Similar question, tho by my read, chain-of-thought seems to be standalone and the primary breakthrough. I asked the author[1]. I've worked in AI&ML since the

Similar question, tho by my read, chain-of-thought seems to be standalone and the primary breakthrough. I asked the author[1]. I've worked in AI&ML since the 90s and both of these do seem much more intuitively promising than simply making larger nets. This seems borne out by the discussion in his previous post, that in Oct '21 OpenAI ran GPT-3 on it, solving 35% and they estimated: "it appears likely that the 175 [billion link neural network] model would require at least two additional orders of magnitude of training data to reach an 80% solve rate.” But instead 85% was achieved within a year using chain-of-thought, not a larger model.[2] [1]https://twitter.com/PMayrgundter/status/1603224294124920832 https://twitter.com/PMayrgundter/status/1603224294124920832 [2]https://yaofu.notion.site/A-Closer-Look-at-Large-Language-Models-Emergent-Abilities-493876b55df5479d80686f68a1abd72f https://yaofu.notion.site/A-Closer-Look-at-Large-Language-Mo...