2 ms·

“Language model” just means a probabilistic model of a token sequence. I think what we’re seeing with LLMs is that something that we thought was very hard—appro

“Language model” just means a probabilistic model of a token sequence. I think what we’re seeing with LLMs is that something that we thought was very hard—approximating the true joint probability function of language in this case—might be possible with much higher compression than we expected. (Sure, 170B parameters is a lot, but the structure of Transformers like GPT is so highly redundant that it still seems simpler than I would have expected.) We started out with very simple probabilistic models by making very strong limiting assumptions. A naive Bayes model assumes every token is independent; a Markov model assumes that a token only depends on the single token before it. But ultimately these are just models for the underlying “real” joint distribution, and in theory we should expect that the simple models are bad because they’re bad estimates of that true distribution. I think my experience with these poor approximates limits what I expected that any language model could do, and the success of LLMs is making me recalibrate my expectations. But I think it’s important to keep in mind that the model is still just spitting out the most likely next tokens according to its own estimation of the joint distribution. It’s so good because the model of “most likely next token” is very accurate. It’s easy to fall into the trap of looking at how specific the output is and thinking “wow, out of all the possible outputs, how did it know to give this one? It could have said anything and the right thing is a very low probability event, so it must really understand the question in order to answer properly.” But another way to think about it might be “wow, it’s incredible that the model accurately estimates the probability distribution for this kind of token sequence.” As to your specific example of things like future time stamps, I think I’ve seen folks say in the past that there is some additional input prefix or suffix information in addition to the prompt you send that is passed into the model. So the model is still a static function—it is not learning from experience within an interaction session; it doesn’t experience time or other external contextual factors as part of its execution. The operators update the contextual input to the model as time passes in the world.

etherealG 4y ago
Do you think our own brains when they “understand” are also “just” estimating probability well? If so perhaps we’re quite close to discovery of something important.
- cgearhart 4y ago
  It’s really hard to say. I very much doubt that brains and LLMs operate on the same principle; but that doesn’t mean we aren’t discovering something important. A few years ago Google claimed credit for achieving Quantum Supremacy by building a quantum processor that simulated itself…by running itself. If that sounds tautological, then you see the problem. They fed in a program that described a quantum circuit. When they ran the program, the processor used its physical quantum gates to execute the circuit described by the program and sampled from the resulting distribution of the quantum state. It’s a bit like saying we “simulated” an electrical circuit by physically running the actual circuit. (Their point was that the processor could run many different circuits depending on the input program, so that makes it a “computer”.) Ignoring the task, that processor did exactly what an LLM does: it sampled from a very complicated probability distribution. Did the processor “understand” the program? Did it “understand” quantum physics or mathematics? Did it “understand” the quantum state of the internal wave function? It definitely produced samples that came from the distribution of the wave function for the program under test. But it’s hard to argue that the processor was “understanding” anything—it was doing exactly the thing that it does. If we had enough qbits, then in theory we could approximate the distribution of an LLM like GPT. Would we say then that the processor “understands” what it’s saying? I don’t think the Google processor understands the circuit, so at what point does approximating and sampling from a bigger distribution transition into “understanding”? Like the quantum device, the LLM is just doing exactly the thing that it does: sampling from a probability distribution. In the case of the LLM it’s a distribution where the samples seem to mean something, so it’s tempting to think that the model _intended_ for the output to have that meaning. But in reality the model can’t do anything else. None of that proves that humans do anything different, but it certainly seems like we (and many other animals) are more complex than that.

etherealG 4y ago

Do you think our own brains when they “understand” are also “just” estimating probability well? If so perhaps we’re quite close to discovery of something important.

cgearhart 4y ago
It’s really hard to say. I very much doubt that brains and LLMs operate on the same principle; but that doesn’t mean we aren’t discovering something important. A few years ago Google claimed credit for achieving Quantum Supremacy by building a quantum processor that simulated itself…by running itself. If that sounds tautological, then you see the problem. They fed in a program that described a quantum circuit. When they ran the program, the processor used its physical quantum gates to execute the circuit described by the program and sampled from the resulting distribution of the quantum state. It’s a bit like saying we “simulated” an electrical circuit by physically running the actual circuit. (Their point was that the processor could run many different circuits depending on the input program, so that makes it a “computer”.) Ignoring the task, that processor did exactly what an LLM does: it sampled from a very complicated probability distribution. Did the processor “understand” the program? Did it “understand” quantum physics or mathematics? Did it “understand” the quantum state of the internal wave function? It definitely produced samples that came from the distribution of the wave function for the program under test. But it’s hard to argue that the processor was “understanding” anything—it was doing exactly the thing that it does. If we had enough qbits, then in theory we could approximate the distribution of an LLM like GPT. Would we say then that the processor “understands” what it’s saying? I don’t think the Google processor understands the circuit, so at what point does approximating and sampling from a bigger distribution transition into “understanding”? Like the quantum device, the LLM is just doing exactly the thing that it does: sampling from a probability distribution. In the case of the LLM it’s a distribution where the samples seem to mean something, so it’s tempting to think that the model _intended_ for the output to have that meaning. But in reality the model can’t do anything else. None of that proves that humans do anything different, but it certainly seems like we (and many other animals) are more complex than that.