3 ms·
The relevant question is whether Humans + LLMs are much more likely to get to AGI than humans without LLMS. And the answer is pretty obviously yes. I don't thin
by ants_everywhere 1y ago
The relevant question is whether Humans + LLMs are much more likely to get to AGI than humans without LLMS. And the answer is pretty obviously yes. I don't think anyone was arguing that we would get to AGI by just training on more data with exactly the same models. Practically every advance in the last few years has been building additional functionality on top of LLMs, not just scaling up the same architecture to more data.
But zooming out, LLMs are universal approximators, so it's trivially true that they can approximate any function that describes AGI. It's also true that logic (from logos or "word") is about reasoning constrained by language and conversations. So an LLM is the right sort of device you'd expect to achieve general intelligence.
There are arguably non-linguistic forms of intelligence, such as visual intelligence. But those also can operate on written symbols (e.g. the stream of bits from an image file).
The other relevant question is why does Gary Macus always seem so angry? It's draining reading one of his posts.
- nomel 1y agoThe concept of mostly static weights holding the bulk of base intuition/knowledge (foundation if you will ;)) seems like a good bet, since it's how the mammalian brain works (with updates of those long term weights mostly happening while you sleep [1]). I very naively assume the "easy" path will be similar: a very different system that's bolted on/references the foundation models, to enable the realtime/novel reasoning (outside the fixed latent space) bit that isn't possible now. [1] https://animalcare.umich.edu/our-impact/our-impact-monitoring-sleeping-rats-may-solve-puzzle-behind-long-term-memory-processing/ https://animalcare.umich.edu/our-impact/our-impact-monitorin...