I think I would put more credence in Yann LeCun's predictions if he had predicted the emergence of few-shot learning and chain-of-thought reasoning as a functio

I think I would put more credence in Yann LeCun's predictions if he had predicted the emergence of few-shot learning and chain-of-thought reasoning as a function of model size/data before they happened. In part, he's arguing that LLMs are not the most efficient path toward intelligence and that some other design will do better, which is probably true, but no one has pushed model size for the (somewhat ironically named) Gato-style multimodal embodied transformers that I think would result in something closer to cat intelligence. I am reasonably certain that further step changes will happen as LLM model and/or data sizes increase. Right now we're achieving a lot of SOTA performance with somewhat smaller models and multimodal pretraining, but not putting those same methods to work in training even larger models on more data.