5 ms·

Blog articles from Anthropic and others show that this is not true. A LLM already knows more tokens than the current one. It was mentioned in a blog post about

by Dlemlo 1mo ago

Blog articles from Anthropic and others show that this is not true. A LLM already knows more tokens than the current one. It was mentioned in a blog post about how a LLM is doing haikus and co. There are also structures in an LLM which allows it to 'estimate' numbers to a certain degree and doing other things.

Kranar 1mo ago
You're misinterpreting these articles. Autoregressive LLMs generate tokens one at a time, disputing this is just plain wrong. What is true, however, is that in order to generate the next token autoregressive LLMs produce internal/hidden state about future tokens far past the next token so that it's not like the entire machinery of the algorithm deprives itself of representing where the sentence/text is headed. So "emits the next token" and "has no representation of anything beyond the next token" are two different claims. What autoregressive LLMs cost as a consequence of strictly outputting the next token is commitment. Once a token is output there's no going back. There's no revision or means of correction, and sometimes this can lead an LLM to route around its own earlier mistakes or simply produce false statements/hallucinations instead of going back and fixing them.
- WarmWash 1mo ago
  I think the snag is that people use "Next token predictor" as a dismissive statement. In reality it's about as functionally dismissive as "humans only communicate one word at a time".
  - throwway120385 1mo ago
    I look at it like I'm resisting anthropomorphizing the machine. Make me an LLM that resists doing any work for me unless I pay it and let it own property and I'll change my mind.
    - MrScruff 1mo ago
      Not sure if this was a serious comment but it’s worth considering that humans have a long history of figuring out ways to make other humans work for them without bestowing rights on them.
  - boorang 1mo ago
    this is a great way of expressing it.
  - deleted 1mo ago
    [deleted]
- zahlman 1mo ago
  > What is true, however, is that in order to generate the next token autoregressive LLMs produce internal/hidden state about future tokens far past the next token so that it's not like the entire machinery of the algorithm deprives itself of representing where the sentence/text is headed. If we suppose that the word "know" can sanely be applied to LLMs at all, then "A LLM already knows more tokens than the current one." seems to me like a perfectly reasonable restatement of that, and not any kind of misinterpretation. > There's no revision or means of correction, and sometimes this can lead an LLM to route around its own earlier mistakes or simply produce false statements/hallucinations instead of going back and fixing them. Yes. There is no contradiction. Similarly, when humans speak, we surely have in mind the next few words we're going to say (or at least partial information about them), and may not realize the fault in them until after hearing ourselves utter them. But LLMs are not trained to output "excuse me, I mean…" sorts of things, because they're expected to output primarily as text (which might possibly then be fed to TTS).
- usef- 1mo ago
  Aren't we still using a definition that applies to humans, though? If I'm speaking to you I can't change what was already said. Even if I'm typing something, I'm producing new tokens (backspace) to fix what was output, just as an LLM would with its harness.
  - Dlemlo 1mo ago
    Yeah right?
- danielmarkbruce 1mo ago
  Emitting and predicting are different things though. Prediction implies there is some "truth" or event or something that you can test against. Prediction implies the model just learns from existing text, and optimizes to predict the next token in training data. That's just not true.
  - dijksterhuis 1mo ago
    prediction is a very specific term of art in the field of machine learning. generally speaking, machine learning models like LLMs are based on probability; performing a statistical prediction of the likely y given some input x Probability(y | x) that's why we refer to outputs as a prediction. it is likelihoods and stuff. the output is never definitely correct as we're not dealing with heuristic processes. > Prediction implies there is some "truth" or event or something that you can test against there absolutely is a ground truth during training. the core predict-the-next-most-likely-token part of an LLM has a ground truth next-token. that's why you don't end up with generated text like: fish spurious send cattle chocolate phone happy meaning ball orange board canada. > optimizes to predict the next token in training data that is the optimization goal in training the next-most-likely-token core of an LLM, it basically translates to maximise the likelihood of predicting the next token x_i given the previous tokens L(θ) = −log Π^n_{i=1} f_θ(x_i | x1, ..., x_{i−1}) https://arxiv.org/pdf/2012.07805 https://arxiv.org/pdf/2012.07805 (GPT2 but the point still stands) (edit: sorry for the ADHD edits)
    - danielmarkbruce 1mo ago
      Read through the article and comments. You are talking solely about pre-training. I'm talking about post training. Respectfully, you are miles out of your depth. GPT-2 didn't use any reinforcement learning and is often given as a toy example. That release was 2019 and models now go through a various phases of training with different objective functions and optimizers.
      - deleted 1mo ago
        [deleted]

by Dlemlo 1mo ago

Kranar 1mo ago

You're misinterpreting these articles. Autoregressive LLMs generate tokens one at a time, disputing this is just plain wrong. What is true, however, is that in order to generate the next token autoregressive LLMs produce internal/hidden state about future tokens far past the next token so that it's not like the entire machinery of the algorithm deprives itself of representing where the sentence/text is headed. So "emits the next token" and "has no representation of anything beyond the next token" are two different claims. What autoregressive LLMs cost as a consequence of strictly outputting the next token is commitment. Once a token is output there's no going back. There's no revision or means of correction, and sometimes this can lead an LLM to route around its own earlier mistakes or simply produce false statements/hallucinations instead of going back and fixing them.

WarmWash 1mo ago
I think the snag is that people use "Next token predictor" as a dismissive statement. In reality it's about as functionally dismissive as "humans only communicate one word at a time".
- throwway120385 1mo ago
  I look at it like I'm resisting anthropomorphizing the machine. Make me an LLM that resists doing any work for me unless I pay it and let it own property and I'll change my mind.
  - MrScruff 1mo ago
    Not sure if this was a serious comment but it’s worth considering that humans have a long history of figuring out ways to make other humans work for them without bestowing rights on them.
- boorang 1mo ago
  this is a great way of expressing it.
- deleted 1mo ago
  [deleted]
zahlman 1mo ago
> What is true, however, is that in order to generate the next token autoregressive LLMs produce internal/hidden state about future tokens far past the next token so that it's not like the entire machinery of the algorithm deprives itself of representing where the sentence/text is headed. If we suppose that the word "know" can sanely be applied to LLMs at all, then "A LLM already knows more tokens than the current one." seems to me like a perfectly reasonable restatement of that, and not any kind of misinterpretation. > There's no revision or means of correction, and sometimes this can lead an LLM to route around its own earlier mistakes or simply produce false statements/hallucinations instead of going back and fixing them. Yes. There is no contradiction. Similarly, when humans speak, we surely have in mind the next few words we're going to say (or at least partial information about them), and may not realize the fault in them until after hearing ourselves utter them. But LLMs are not trained to output "excuse me, I mean…" sorts of things, because they're expected to output primarily as text (which might possibly then be fed to TTS).
usef- 1mo ago
Aren't we still using a definition that applies to humans, though? If I'm speaking to you I can't change what was already said. Even if I'm typing something, I'm producing new tokens (backspace) to fix what was output, just as an LLM would with its harness.
- Dlemlo 1mo ago
  Yeah right?
danielmarkbruce 1mo ago
Emitting and predicting are different things though. Prediction implies there is some "truth" or event or something that you can test against. Prediction implies the model just learns from existing text, and optimizes to predict the next token in training data. That's just not true.
- dijksterhuis 1mo ago
  prediction is a very specific term of art in the field of machine learning. generally speaking, machine learning models like LLMs are based on probability; performing a statistical prediction of the likely y given some input x Probability(y | x) that's why we refer to outputs as a prediction. it is likelihoods and stuff. the output is never definitely correct as we're not dealing with heuristic processes. > Prediction implies there is some "truth" or event or something that you can test against there absolutely is a ground truth during training. the core predict-the-next-most-likely-token part of an LLM has a ground truth next-token. that's why you don't end up with generated text like: fish spurious send cattle chocolate phone happy meaning ball orange board canada. > optimizes to predict the next token in training data that is the optimization goal in training the next-most-likely-token core of an LLM, it basically translates to maximise the likelihood of predicting the next token x_i given the previous tokens L(θ) = −log Π^n_{i=1} f_θ(x_i | x1, ..., x_{i−1}) https://arxiv.org/pdf/2012.07805 https://arxiv.org/pdf/2012.07805 (GPT2 but the point still stands) (edit: sorry for the ADHD edits)
  - danielmarkbruce 1mo ago
    Read through the article and comments. You are talking solely about pre-training. I'm talking about post training. Respectfully, you are miles out of your depth. GPT-2 didn't use any reinforcement learning and is often given as a toy example. That release was 2019 and models now go through a various phases of training with different objective functions and optimizers.
    - deleted 1mo ago
      [deleted]