Adam is Great, Why Still Miss SGD
Haven’t updated in a while. Recently busy writing graduation thesis, just happened to write the optimization-related part, remembered a great article I bookmarked on Zhihu before. Reading it again still feels very beneficial, specially reposting. Original link here, deleted upon infringement.