4 ms·

>> I cannot rectify this with people saying it only looks one word ahead. One word must come next, but to do a good job modeling what that word will be, wouldn’

>> I cannot rectify this with people saying it only looks one word ahead. One word must come next, but to do a good job modeling what that word will be, wouldn’t you need to consider further ahead than that? No, because you don't predict the probability of a token, you predict the probability of a token _given_ a preceding sequence of tokens. So, to decide whether to follow "I climbed the apple tree and picked" with "a" or "an", you calculate the following (which are conditional probabilities; read p(A|B) as "probability of A given B): P₁ = p(a|I,climbed,the,apple,tree,and,picked) P₂ = p(an|I,clibed,the,apple,tree,and,picked) Now, if P₁ > P₂, you generate "a", otherwise you generate "an". Note that the sentence "I climbed the apple tree and picked" is different than the sentence "I climbed the pear tree and picked" so the following are different probabilities, also: P₃ = p(a|I,climbed,the,pear,tree,and,picked) P₄ = p(an|I,clibed,the,pear,tree,and,picked) So you'll get a different token generated depending on whether P₃ > P₄ or not. In short, yeah, you can choose whether it's going to be "a" or "an" according to the context of the sentence so-far. But even in cases where you can't do that, when you don't have the context of "pear" or "apple", what you _can_ do is generate this string: "I climbed the pear tree and picked a" And then calculate the following probabilities: P₅ = p(ball|I,climbed,the,tree,and,picked,a) P₆ = p(abacus|I,clibed,the,tree,and,picked,a) And if P₅ > P₆ you'll generate "ball", otherwise generate "abacus". So you don't have to know what's coming next, just what you've generated so-far. Does that help?

swid 4y ago
It’s just overly reductive. LLMs do not work like Bayesian or Markov models that fail to scale. In order to predict what word a human will say next past a certain accuracy, you actually have to model more than text. You have to model that person’s behavior in a sense, or otherwise you can’t reach that accuracy (the tree example would not require that much - I used it since I had seen it before). I like gwern’s[0] example of a dozen people jockeying for power at a dinner table. Part of the next token prediction includes predicting how these Machiavellian schemes play out and why. Next word prediction may give the model a license and reason to figure all that out. [0]: https://gwern.net/scaling-hypothesis https://gwern.net/scaling-hypothesis
- YeGoblynQueenne 4y ago
  >> It’s just overly reductive. LLMs do not work like Bayesian or Markov models that fail to scale. Yes, they do. That's how language modelling works. It's the same principle whether it's a small or large language model. Scale only improves predictive accuracy, but it doesn't change what is predicted. In fact, you don't need a large language model to model the use of a/an correctly. An n-gram model will do that just fine. Edit: See Equation 1, page 3, for the pre-training objective in the original GPT; see Equation 4, same page, for the fine-tuning objective: https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf https://cdn.openai.com/research-covers/language-unsupervised... The same objective is used throughout the GPT line. Is that more clear?
  - YeGoblynQueenne 4y ago
    @swid, would you like me to walk you through the notation of the training objectives in the linked paper? It's basically what I showed above. Let me know if you need some pointers to understand the paper's notation.
    - swid 4y ago
      Thanks for the offer, but it feels to me like you are missing the point I would like to be making. Maybe we are all just stochastic parrots in a sense? (FWIW, I do basically understand the notation). I think the universe might be computable like Stephen Wolfram says - if this is the case - in theory someone could eventually walk you through the math and series of computations to make and then say, "see - humans are not intelligent, they are just biological machines without free will. With unlimited time and computation resources, we can run a human on a Turing machine in a virtual universe." Even if the universe is not fully computable to the point we can perfectly simulate it on a Turing machine, we already do a pretty good job simulating lots of things, and it's not clear the loss of fidelity would make much difference. As I said, "intelligence" could be an emergent property of certain highly complex networks. I don't think the substrate of the network matters (digital vs biological) - it's just certain states may come with a feeling for the network. Being able explain the output of the system by math is very powerful - but it fails to capture emergent properties of the universe we would generally prefer to believe in like consciousness. Anyway - we see slime molds act as path optimizers; lots of algorithms we find useful have physical analogies. I don't know how the brain learns, but biologically, there must be some kind of gradient to follow and reinforce some neural connections while reducing others. Math is the language of the universe, but we believe in free will. Explaining the math of LLMs will not bring us into an agreement - this is unfortunately philosophical I think.

swid 4y ago

It’s just overly reductive. LLMs do not work like Bayesian or Markov models that fail to scale. In order to predict what word a human will say next past a certain accuracy, you actually have to model more than text. You have to model that person’s behavior in a sense, or otherwise you can’t reach that accuracy (the tree example would not require that much - I used it since I had seen it before). I like gwern’s[0] example of a dozen people jockeying for power at a dinner table. Part of the next token prediction includes predicting how these Machiavellian schemes play out and why. Next word prediction may give the model a license and reason to figure all that out. [0]: https://gwern.net/scaling-hypothesis https://gwern.net/scaling-hypothesis

YeGoblynQueenne 4y ago
>> It’s just overly reductive. LLMs do not work like Bayesian or Markov models that fail to scale. Yes, they do. That's how language modelling works. It's the same principle whether it's a small or large language model. Scale only improves predictive accuracy, but it doesn't change what is predicted. In fact, you don't need a large language model to model the use of a/an correctly. An n-gram model will do that just fine. Edit: See Equation 1, page 3, for the pre-training objective in the original GPT; see Equation 4, same page, for the fine-tuning objective: https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf https://cdn.openai.com/research-covers/language-unsupervised... The same objective is used throughout the GPT line. Is that more clear?
- YeGoblynQueenne 4y ago
  @swid, would you like me to walk you through the notation of the training objectives in the linked paper? It's basically what I showed above. Let me know if you need some pointers to understand the paper's notation.
  - swid 4y ago
    Thanks for the offer, but it feels to me like you are missing the point I would like to be making. Maybe we are all just stochastic parrots in a sense? (FWIW, I do basically understand the notation). I think the universe might be computable like Stephen Wolfram says - if this is the case - in theory someone could eventually walk you through the math and series of computations to make and then say, "see - humans are not intelligent, they are just biological machines without free will. With unlimited time and computation resources, we can run a human on a Turing machine in a virtual universe." Even if the universe is not fully computable to the point we can perfectly simulate it on a Turing machine, we already do a pretty good job simulating lots of things, and it's not clear the loss of fidelity would make much difference. As I said, "intelligence" could be an emergent property of certain highly complex networks. I don't think the substrate of the network matters (digital vs biological) - it's just certain states may come with a feeling for the network. Being able explain the output of the system by math is very powerful - but it fails to capture emergent properties of the universe we would generally prefer to believe in like consciousness. Anyway - we see slime molds act as path optimizers; lots of algorithms we find useful have physical analogies. I don't know how the brain learns, but biologically, there must be some kind of gradient to follow and reinforce some neural connections while reducing others. Math is the language of the universe, but we believe in free will. Explaining the math of LLMs will not bring us into an agreement - this is unfortunately philosophical I think.