4 ms·
I agree more with Yann LeCunn's salty reply. Over long run, knowing how autoregressive language models work from scratch will be just one step in having foundat
by LarsDu88 1mo ago
I agree more with Yann LeCunn's salty reply. Over long run, knowing how autoregressive language models work from scratch will be just one step in having foundational understanding, and they might become dated... like knowing how a CRT monitor work. Something of historical interest and good for learning, but not crucial to being well-rounded.
There are other types of models like diffusion models right now that are showing more efficiency and have a higher ceiling for improvement. Understanding math and fundamentals are more important.
- lern_too_spel 1mo agoYann has been consistently wrong about the limits of LLMs.
- LarsDu88 1mo agoHe has, but I think he's right in the sense that there will be better models. It would be premature to think that the current frontier model architectures will last into the future. It would be like making DVD engineering a standard part of high school curricula
- handoflixue 1mo ago> It would be like making DVD engineering a standard part of high school curricula I fail to see how PG suggesting that a 17 year old, today, should study something... is in any way a parallel to formalizing it as part of the curriculum. I would, in fact, expect PG to recommend something completely different in 10 years. The whole point of hacking is to understand the world around you, today
- carbondrone 1mo agoWell good for them, I think their comment is heavily suggesting they're dissatisfied if said limits, and want to see them broken.
- aaron695 1mo ago[dead]