3 ms·

One reason I hate the “glorified word predictor” phrase, is that predicting the next word involves considering what will come well after that. I saw a research

by swid 4y ago

One reason I hate the “glorified word predictor” phrase, is that predicting the next word involves considering what will come well after that. I saw a research paper where they tested a LLM to predict the word “a” vs “an”. In order to do that, it seems like you need to consider at least 1 word past the next token. The best test for this was: I climbed the pear tree and picked a pear. I climbed the apple tree and picked … That’s a simple example, but the other day, I used ChatGPT to refactor a 2000 word talk to 1000 words and a more engaging voice. I asked for it to make both 500 and 1000 word versions, and it felt to me like it was adhering to the length to determine pacing and delivery of material that signaled it was planning ahead about how much content each fact required. I cannot rectify this with people saying it only looks one word ahead. One word must come next, but to do a good job modeling what that word will be, wouldn’t you need to consider further ahead than that?

usaar333 4y ago
> In order to do that, it seems like you need to consider at least 1 word past the next token. Why? Any large probabilistic model in your example would also predict "an" due to the high attention on the preceding "apple". (In case you are wondering, for the OpenAI GPT3 models, this is consistently handled at the scale of Babbage, which is around 3 billion params). > One word must come next, but to do a good job modeling what that word will be, wouldn’t you need to consider further ahead than that? Well, yes, but GPT isn't a human. That's why it needs so much more data than a human to talk so fluently or "reason".
- swid 4y ago
  > Why? Any large probabilistic model in your example would also predict "an" due to the high attention on apple. I’m not ignoring how the tech works and this is a simple example. But that doesn’t preclude emergent behavior beyond the statistics. Did you catch the GPT Othello paper where researchers show, from a transcript of moves, the model learned to model the board state to make its next move? [0] I’m beginning to think it is reasonable to think of human speech (behavior will come) as a function which these machines are attempting to match. In order to make the best statistically likely response, it should have a model of how different humans speak. I know GPT is not human, but I also don’t know what form intelligence comes in. I am mostly certain you won’t figure out why we are conscious from studying physics and biochemistry (or equivalently the algorithm of an AI, if we had one). I also believe where ever we find intelligence in the universe, we will find some kind of complex network at its core - and I’m doubtful studying that network we will tell us if that network is “intelligent” or “conscious” in a a scientific way - but perhaps we’d say something about it like - “it has a high attention on apple”. [0] https://thegradient.pub/othello/ https://thegradient.pub/othello/
  - usaar333 4y ago
    I agree there is emergent behavior and this is impressive (I'm similarly impressed how well GPT4 can generate SQL statements). That said, even playing Othello is still an example of next-token prediction via pattern recognition. Yah, it might be quasi-building a model of sorts, but that's of course just what non-linear predictors do. Don't get me wrong -- we are also very powerful pattern recognizers.
YeGoblynQueenne 4y ago
>> I cannot rectify this with people saying it only looks one word ahead. One word must come next, but to do a good job modeling what that word will be, wouldn’t you need to consider further ahead than that? No, because you don't predict the probability of a token, you predict the probability of a token _given_ a preceding sequence of tokens. So, to decide whether to follow "I climbed the apple tree and picked" with "a" or "an", you calculate the following (which are conditional probabilities; read p(A|B) as "probability of A given B): P₁ = p(a|I,climbed,the,apple,tree,and,picked) P₂ = p(an|I,clibed,the,apple,tree,and,picked) Now, if P₁ > P₂, you generate "a", otherwise you generate "an". Note that the sentence "I climbed the apple tree and picked" is different than the sentence "I climbed the pear tree and picked" so the following are different probabilities, also: P₃ = p(a|I,climbed,the,pear,tree,and,picked) P₄ = p(an|I,clibed,the,pear,tree,and,picked) So you'll get a different token generated depending on whether P₃ > P₄ or not. In short, yeah, you can choose whether it's going to be "a" or "an" according to the context of the sentence so-far. But even in cases where you can't do that, when you don't have the context of "pear" or "apple", what you _can_ do is generate this string: "I climbed the pear tree and picked a" And then calculate the following probabilities: P₅ = p(ball|I,climbed,the,tree,and,picked,a) P₆ = p(abacus|I,clibed,the,tree,and,picked,a) And if P₅ > P₆ you'll generate "ball", otherwise generate "abacus". So you don't have to know what's coming next, just what you've generated so-far. Does that help?
- swid 4y ago
  It’s just overly reductive. LLMs do not work like Bayesian or Markov models that fail to scale. In order to predict what word a human will say next past a certain accuracy, you actually have to model more than text. You have to model that person’s behavior in a sense, or otherwise you can’t reach that accuracy (the tree example would not require that much - I used it since I had seen it before). I like gwern’s[0] example of a dozen people jockeying for power at a dinner table. Part of the next token prediction includes predicting how these Machiavellian schemes play out and why. Next word prediction may give the model a license and reason to figure all that out. [0]: https://gwern.net/scaling-hypothesis https://gwern.net/scaling-hypothesis
  - 4y ago

by swid 4y ago

usaar333 4y ago

> In order to do that, it seems like you need to consider at least 1 word past the next token. Why? Any large probabilistic model in your example would also predict "an" due to the high attention on the preceding "apple". (In case you are wondering, for the OpenAI GPT3 models, this is consistently handled at the scale of Babbage, which is around 3 billion params). > One word must come next, but to do a good job modeling what that word will be, wouldn’t you need to consider further ahead than that? Well, yes, but GPT isn't a human. That's why it needs so much more data than a human to talk so fluently or "reason".

swid 4y ago
> Why? Any large probabilistic model in your example would also predict "an" due to the high attention on apple. I’m not ignoring how the tech works and this is a simple example. But that doesn’t preclude emergent behavior beyond the statistics. Did you catch the GPT Othello paper where researchers show, from a transcript of moves, the model learned to model the board state to make its next move? [0] I’m beginning to think it is reasonable to think of human speech (behavior will come) as a function which these machines are attempting to match. In order to make the best statistically likely response, it should have a model of how different humans speak. I know GPT is not human, but I also don’t know what form intelligence comes in. I am mostly certain you won’t figure out why we are conscious from studying physics and biochemistry (or equivalently the algorithm of an AI, if we had one). I also believe where ever we find intelligence in the universe, we will find some kind of complex network at its core - and I’m doubtful studying that network we will tell us if that network is “intelligent” or “conscious” in a a scientific way - but perhaps we’d say something about it like - “it has a high attention on apple”. [0] https://thegradient.pub/othello/ https://thegradient.pub/othello/
- usaar333 4y ago
  I agree there is emergent behavior and this is impressive (I'm similarly impressed how well GPT4 can generate SQL statements). That said, even playing Othello is still an example of next-token prediction via pattern recognition. Yah, it might be quasi-building a model of sorts, but that's of course just what non-linear predictors do. Don't get me wrong -- we are also very powerful pattern recognizers.

YeGoblynQueenne 4y ago

>> I cannot rectify this with people saying it only looks one word ahead. One word must come next, but to do a good job modeling what that word will be, wouldn’t you need to consider further ahead than that? No, because you don't predict the probability of a token, you predict the probability of a token _given_ a preceding sequence of tokens. So, to decide whether to follow "I climbed the apple tree and picked" with "a" or "an", you calculate the following (which are conditional probabilities; read p(A|B) as "probability of A given B): P₁ = p(a|I,climbed,the,apple,tree,and,picked) P₂ = p(an|I,clibed,the,apple,tree,and,picked) Now, if P₁ > P₂, you generate "a", otherwise you generate "an". Note that the sentence "I climbed the apple tree and picked" is different than the sentence "I climbed the pear tree and picked" so the following are different probabilities, also: P₃ = p(a|I,climbed,the,pear,tree,and,picked) P₄ = p(an|I,clibed,the,pear,tree,and,picked) So you'll get a different token generated depending on whether P₃ > P₄ or not. In short, yeah, you can choose whether it's going to be "a" or "an" according to the context of the sentence so-far. But even in cases where you can't do that, when you don't have the context of "pear" or "apple", what you _can_ do is generate this string: "I climbed the pear tree and picked a" And then calculate the following probabilities: P₅ = p(ball|I,climbed,the,tree,and,picked,a) P₆ = p(abacus|I,clibed,the,tree,and,picked,a) And if P₅ > P₆ you'll generate "ball", otherwise generate "abacus". So you don't have to know what's coming next, just what you've generated so-far. Does that help?

swid 4y ago
It’s just overly reductive. LLMs do not work like Bayesian or Markov models that fail to scale. In order to predict what word a human will say next past a certain accuracy, you actually have to model more than text. You have to model that person’s behavior in a sense, or otherwise you can’t reach that accuracy (the tree example would not require that much - I used it since I had seen it before). I like gwern’s[0] example of a dozen people jockeying for power at a dinner table. Part of the next token prediction includes predicting how these Machiavellian schemes play out and why. Next word prediction may give the model a license and reason to figure all that out. [0]: https://gwern.net/scaling-hypothesis https://gwern.net/scaling-hypothesis
- 4y ago