Scaling alone will certainly not get us to AGI, but that's trivially obvious. The "scale is all you need" argument, from anyone intelligent, assumes that other

Scaling alone will certainly not get us to AGI, but that's trivially obvious. The "scale is all you need" argument, from anyone intelligent, assumes that other obvious deficiencies such as lack of short-term memory (to enable more capable planning/reasoning) will also be taken care of along the way, and I'd not be surprised to see that particular one addressed in upcoming next-gen models. There's a recent paper here from Google suggesting one way to do it, although may other ways too. https://arxiv.org/pdf/2404.07143.pdf https://arxiv.org/pdf/2404.07143.pdf The more interesting question is what other components/capabilities needs to be added to pre-trained transformers to get to AGI, or are some of the missing pieces so fundamental that they require a new approach, and can't just be retrofitted along the way as we continue to scale up?