6 ms·
This doesn’t seem a surprising or particularly deep result, and perhaps we’re having the wrong arguments about LLMs. I certainly don’t doubt that they’re captur
by thom 4y ago
This doesn’t seem a surprising or particularly deep result, and perhaps we’re having the wrong arguments about LLMs. I certainly don’t doubt that they’re capturing deep knowledge, my only criticism would be that they’re doing so in a lossy and extremely static way.
I’ve come to think of it this way. A simple Markov chain model trained on language would be the simplest example of something picking up a very shallow model of some text. Let’s call that a Layer One model, and we had fun with them for a while.
Machine translation goes one step further - instead of learning just the text it’s given, a model sees that above words are representations of things and ideas, and structures to render those ideas in multiple languages. That feels like Layer Two, but I’m not aware of traditional machine translation models that will take a piece of poetry, for example, and render it in a new language, perhaps using local idioms to capture the essence, if not the exact translation, of some content. So it’s only really one layer deeper.
Large language models are layers beyond this. They are both wider and deeper. They are showing that much more knowledge than we thought can be captured statically - patterns of patterns of patterns of patterns turn out to be enough to store and process a huge amount of what we previously thought was a privileged part of human intelligence. That’s the truly interesting result - the models still seem dumb to me, but boy, being dumb is much more impressive than we used to think.
We’re focused on training LLMs on larger corpora to store more stuff and more parameters and layers to find deeper patterns. But even though they’re clearly learning world models, it’s still just a static snapshot of knowledge. ChatGPT shows over a short span that a model can integrate new information quite well. What’s missing is some mechanism for _constant_ integration of new information, and some degree of internal rumination about past, present and future knowledge. Something algorithmic, temporal and dynamic, not just a static repository of knowledge. I don’t think this actually requires a great deal of magic, just a slight change in direction (although I’ve no doubt this is being worked on and is probably in all sorts of papers already). Of course, there’s still time for us to decide not to do that, but the hour is close at hand.
- typon 4y agoI think to do this at scale and properly we'd need a replacement for backprop
- deleted 4y ago[deleted]
- guerrilla 4y ago> my only criticism would be that they’re doing so in a lossy and extremely static way. Indeed, these are my thoughts too after some interaction. It seems a lot more like a compression algorithm with search as an interface than any kind of intelligence.
- naasking 4y ago> It seems a lot more like a compression algorithm with search as an interface than any kind of intelligence. Intelligence probably is a form of compression with search. GPT just isn't intelligent enough yet, ie. doesn't compress enough or search well enough.
- pyinstallwoes 4y agoIsn’t that basically consciousness though? Language and memory. After all the Bible does start off with “In the beginning was the Word” God is a LLM. :p
- turtleyacht 4y agoInteresting, I haven't yet heard of interpretations from ChatGPT around rabbinic literature. Imagine getting entire summaries of volumes that took lifetimes to ingest.
- pyinstallwoes 4y agoI similarly find it quite interesting. If you find anything and remember this comment, I'd love to know more!
- vladf 4y agoI was hoping for this take—-of course LLMs have an internal model, and we can externally verify how accurate it is statistically. And the more you train it the better it gets. And for discrete spaces you might even get a perfect model eventually with nonzero probability. Since you mentioned different layers, one question to ask is if your poetry example is “just a combination of existing rare patterns,” one that’d be addressed by a bigger model with more data. And as you say the integration of new information does not seem to rely on any fundamentally new principles, just more engineering work. I think then the question to ask would be, well, how much more data to x% more faithful of the rules of Othello? This rate of learning matters, and can be the difference between “in our lifetime” and “never in hundreds of years”. As ML practitioners our jobs are often to improve these rates through better formulation of the learning problem. I think there’s a connection here that amounts to “how quickly you learn”, and it has implications on causality: https://vladfeinberg.com/2019/12/01/metaphysics-of-causality.html https://vladfeinberg.com/2019/12/01/metaphysics-of-causality...