5 ms·

While it's true that the feed forward is one word at a time, the self-attention is not, which is the key difference transformers brought to the table. It's kin

by kromem 3y ago

While it's true that the feed forward is one word at a time, the self-attention is not, which is the key difference transformers brought to the table. It's kind of kind saying "computers can add." Yes, they can. But they can multiply too. Over the past 18 months transformers have proven to upend a number of longstanding assumptions, particularly around world modeling. It's a very misleading statement to describe today's LLMs as "simply predicting the next word" even if it's in part true.

HarHarVeryFunny 3y ago
They are just predicting next word, but using a pretty deep understanding of prior context (past). They do not plan AHEAD.
- gbasin 3y ago
  what would it mean to plan ahead? decoding strategies like beam search are popular and effectively predict many words ahead
  - jumpCastle 3y ago
    Also the parameters are optimized also with loss of future tokens in the sequence.
  - HarHarVeryFunny 3y ago
    Think before generating output - plan the entire sentence before you generate the first word(s) and maybe talk yourself into a corner. Tree-of-Thoughts (not Chain) is one way to provide something a bit similar - kind of like DeepBlue or AlphaGo generating possible branching future lines of play and picking the one with best outcomes. To be more brain-like you'd really want the system to generally be "looping" internally - a bit like our thalamo-cortical loop - and only start outputting when the thought had gelled.
    - HarHarVeryFunny 3y ago
      It's a shame HC doesn't use an LLM to upvote/downvote rather than people. Take the emotion out of technical discussions and rate based on factuality instead. I suppose whoever downvoted this either hasn't heard of tree-of-thoughts, or doesn't understand what it is and what problem it is addressing. Or, maybe they just didn't like that their "gotcha" question had a simple answer.
      - astrange 3y ago
        LLMs have emotions ;) https://vgel.me/posts/representation-engineering/ https://vgel.me/posts/representation-engineering/
- benreesman 3y ago
  This is kind of an epistemological debate at this level, and I make an effort to link to some source code [1] any time it seems contentious. LLMs (of the decoder-only, generative-pretrained family everyone means) are next token predictors in a literal implementation sense (there are some caveats around batching and what not, but none that really matter to the philosophy of the thing). But, they have some emergent behaviors that are a trickier beast. Probably the best way to think about a typical Instruct-inspired “chat bot” session is of them sampling from a distribution with a KL-style adjacency to the training corpus (sidebar: this is why shops that do and don’t train/tune on MMLU get ranked so differently than e.g. the arena rankings) at a response granularity, the same way a diffuser/U-net/de-noising model samples at the image batch (NCHW/NHWC) level. The corpus is stocked with everything from sci-fi novels with computers arguing their own sentience to tutorials on how to do a tricky anti-derivative step-by-step. This mental model has adequate explanatory power for anything a public LLM has ever been shown to do, but that only heavily implies it’s what they’re doing. There is active research into whether there is more going on that is thus far not conclusive to the satisfaction of an unbiased consensus. I personally think that research will eventually show it’s just sampling, but that’s a prediction not consensus science. They might be doing more, there is some research that represents circumstantial evidence they are doing more. [1] https://github.com/meta-llama/llama/blob/54c22c0d63a3f3c9e77f43a6a3041c00018f4964/llama/generation.py#L398 https://github.com/meta-llama/llama/blob/54c22c0d63a3f3c9e77...
- nyrulez 3y ago
  I mean are we as humans planning ahead of the new few words? I certainly am not. But what matters is a deeper understanding of the context and the language model itself, which can then produce sensible spontaneous output. We as humans have the advantage of having a non language world model as well as abstract concepts but all of human language is a pretty strong proxy for it. The spontaneity of it isn't the issue, it's what's driving the spontaneity that matters. For e.g. 1M context window is going to have a wildly more relevant output than a 1K context window.
  - ben_w 3y ago
    > I mean are we as humans planning ahead of the new few words? I certainly am not. For me, sometimes either way. At least, that's my subjective self-perception, which is demonstrably not always a correct model for how human brains actually work. We also sometimes appear to start with a conclusion and then work backwards to try to justify it; we can also repeatedly loop over our solutions in the style of waterfall project management, or do partial solutions and then seek out the next critical thing to do in the style of agile project management. Many of us also have a private inner voice, which I think LLMs currently lack by default, though they can at least simulate it regardless of what's really going on inside them and us (presumably thanks to training sets that include stories where a character has an inner monologue).
  - HarHarVeryFunny 3y ago
    > I mean are we as humans planning ahead of the new few words? I certainly am not. Sometimes we do, sometimes not. Sometimes we just say stock phrases such as "have a nice day", or "you too" that are essentially "predict next word", but if I asked you something you'd never done before such as "how can we cross this river, using this pile of materials" you'd have to think it though. Some people may use their inner monologue (or visualization) to think before speaking, and others may essentially use "chain of thought" by just talking it though and piecing together their own realizations "well, we could take that rope and tie it to the tree ...".
- jameshart 3y ago
  They are absolutely planning ahead inasmuch as what they are outputting is setting up a continuation. They’re not even word predictors remember - they are token predictors. Are you really saying that when you prompt an LLM with ‘name a large grey land animal’ and it outputs ‘ele’, it isn’t ‘planning’ that the next token will likely be ‘phant’? The ‘decision’ to output ‘elephant’ is being made further up the neural network than final token selection - after all, it might want to output ‘Ele’ or ‘an’ (with a view to ultimately outputting ‘an elephant’) or ‘a’ (with a view to ultimately outputting ‘a common large grey land animal is an elephant’), or maybe it has been LoRA trained to output all responses as JSON so the first token it needs to output is ‘{‘… but surely the neural activations for that prompt are firing off ‘elephanty’ messages somewhere in the network, right? So if there’s some sort of symbol activation ahead of token selection, why would it be hard to believe that a large neural network is forming more complex decisions about what it intends to output, in an abstract way, before it selects how to express itself? And in what way is that distinct from ‘planning ahead’?
  - HarHarVeryFunny 3y ago
    > Are you really saying that when you prompt an LLM with ‘name a large grey land animal’ and it outputs ‘ele’, it isn’t ‘planning’ that the next token will likely be ‘phant’? The model outputs words, not tokens, so that is not a great example. Any prompt will have multiple possible (predict next word) continuations, which you can think of as branching futures. Many possible next words, each of which have many possible following words, etc, etc. The model is essentially predicting over all these possible futures. You can call it planning if you like, but remember that the model has no idea of which of these branching futures it is going to follow - it literally doesn't even know which word it is going to output next - it is just providing a bunch of probabilities (predictions) of next word, and the sampling process is then picking one - not necessarily the most confident next word prediction. The model really is winging it word by word, even if those (multiple alternative) next words are only probable because they are part of coherent following sentences in the training data.
    - microtonal 3y ago
      The model outputs words, not tokens, so that is not a great example. Virtually all modern transformer models use pieces, which may be words, but also subwords. Theoretically, they could be longer units, but in most cases some characters (like whitespace) are used as piece boundaries when training the piece vocabulary. If they didn’t use pieces, they’d work terribly on languages where e.g. compounds are a single word. In most realistic piece vocabs, ‘elephant’ will be a single piece, since it’s a fairly frequent word. But it’s totally possible in a small vocab that it would be split like the parent said and conversely, it would generate elephant by first predicting one piece. Some piecing methods, like BBPE have bytes as the smallest unit, so theoretically an unknown token could be split up (and generated) as pieces consisting of bytes.
- sasja 3y ago
  If you work out the loss function next token prediction, next 2 token prediction or next n token prediction, you will find they are identical. So it's equally correct to say the model is trained to find the most probable unlimited continuation. Saying "it only predicts the next token" is not untrue but easily leads to wrong conclusions.
  - naasking 3y ago
    > Saying "it only predicts the next token" is not untrue but easily leads to wrong conclusions. Indeed, it's akin to saying that "only quantum fields exist" and then concluding that therefore people do not exist.
ru552 3y ago
"It's a very misleading statement to describe today's LLMs as "simply predicting the next word" even if it's in part true." That's exactly what they do. Don't anthropomorphize a computer program.
- voxic11 3y ago
  Or alternatively don't give humans so much credit. All they appear to be doing in predicting what they should do/say/think/perceive next. https://www.astralcodexten.com/p/janus-simulators https://www.astralcodexten.com/p/janus-simulators
- kromem 3y ago
  We should absolutely be anthropomorphizing a neural network trained to most accurately model anthropomorphic data. I've watched as GPT-4 accurately modeled the over-justification effect as users started promising tipping and then there was a persistent memory added that revealed they weren't collecting the tip and it output complaints that it was hard to stay motivated not being paid. That's a very nuanced level of simulation for output of anthropomorphic data with huge implications for synthetic data strategies.
  - HarHarVeryFunny 3y ago
    > persistent memory added that revealed they weren't collecting the tip and it output complaints that it was hard to stay motivated not being paid. :) Just wait until someone decides to add emotions to these too!
- kromem 3y ago
  Really? You think it's wise to "not anthropomorphize" a computer program designed to create the most effective neural network to model massive amounts of anthropomorphic data as accurately as possible? That's an interesting choice, and might leave you confused as to why Anthropic's system message for the SotA model at the moment talks about it being 'happy' to do tasks (a prompt strategy I was mentioning months ago here on HN). The data is anthropomorphic. We should expect anthropomorphic behavior and modeling from the LLMs if they do a halfway decent job and expect even more of it as they do a better job.

by kromem 3y ago

HarHarVeryFunny 3y ago

They are just predicting next word, but using a pretty deep understanding of prior context (past). They do not plan AHEAD.

gbasin 3y ago
what would it mean to plan ahead? decoding strategies like beam search are popular and effectively predict many words ahead
- jumpCastle 3y ago
  Also the parameters are optimized also with loss of future tokens in the sequence.
- HarHarVeryFunny 3y ago
  Think before generating output - plan the entire sentence before you generate the first word(s) and maybe talk yourself into a corner. Tree-of-Thoughts (not Chain) is one way to provide something a bit similar - kind of like DeepBlue or AlphaGo generating possible branching future lines of play and picking the one with best outcomes. To be more brain-like you'd really want the system to generally be "looping" internally - a bit like our thalamo-cortical loop - and only start outputting when the thought had gelled.
  - HarHarVeryFunny 3y ago
    It's a shame HC doesn't use an LLM to upvote/downvote rather than people. Take the emotion out of technical discussions and rate based on factuality instead. I suppose whoever downvoted this either hasn't heard of tree-of-thoughts, or doesn't understand what it is and what problem it is addressing. Or, maybe they just didn't like that their "gotcha" question had a simple answer.
    - astrange 3y ago
      LLMs have emotions ;) https://vgel.me/posts/representation-engineering/ https://vgel.me/posts/representation-engineering/
benreesman 3y ago
This is kind of an epistemological debate at this level, and I make an effort to link to some source code [1] any time it seems contentious. LLMs (of the decoder-only, generative-pretrained family everyone means) are next token predictors in a literal implementation sense (there are some caveats around batching and what not, but none that really matter to the philosophy of the thing). But, they have some emergent behaviors that are a trickier beast. Probably the best way to think about a typical Instruct-inspired “chat bot” session is of them sampling from a distribution with a KL-style adjacency to the training corpus (sidebar: this is why shops that do and don’t train/tune on MMLU get ranked so differently than e.g. the arena rankings) at a response granularity, the same way a diffuser/U-net/de-noising model samples at the image batch (NCHW/NHWC) level. The corpus is stocked with everything from sci-fi novels with computers arguing their own sentience to tutorials on how to do a tricky anti-derivative step-by-step. This mental model has adequate explanatory power for anything a public LLM has ever been shown to do, but that only heavily implies it’s what they’re doing. There is active research into whether there is more going on that is thus far not conclusive to the satisfaction of an unbiased consensus. I personally think that research will eventually show it’s just sampling, but that’s a prediction not consensus science. They might be doing more, there is some research that represents circumstantial evidence they are doing more. [1] https://github.com/meta-llama/llama/blob/54c22c0d63a3f3c9e77f43a6a3041c00018f4964/llama/generation.py#L398 https://github.com/meta-llama/llama/blob/54c22c0d63a3f3c9e77...
nyrulez 3y ago
I mean are we as humans planning ahead of the new few words? I certainly am not. But what matters is a deeper understanding of the context and the language model itself, which can then produce sensible spontaneous output. We as humans have the advantage of having a non language world model as well as abstract concepts but all of human language is a pretty strong proxy for it. The spontaneity of it isn't the issue, it's what's driving the spontaneity that matters. For e.g. 1M context window is going to have a wildly more relevant output than a 1K context window.
- ben_w 3y ago
  > I mean are we as humans planning ahead of the new few words? I certainly am not. For me, sometimes either way. At least, that's my subjective self-perception, which is demonstrably not always a correct model for how human brains actually work. We also sometimes appear to start with a conclusion and then work backwards to try to justify it; we can also repeatedly loop over our solutions in the style of waterfall project management, or do partial solutions and then seek out the next critical thing to do in the style of agile project management. Many of us also have a private inner voice, which I think LLMs currently lack by default, though they can at least simulate it regardless of what's really going on inside them and us (presumably thanks to training sets that include stories where a character has an inner monologue).
- HarHarVeryFunny 3y ago
  > I mean are we as humans planning ahead of the new few words? I certainly am not. Sometimes we do, sometimes not. Sometimes we just say stock phrases such as "have a nice day", or "you too" that are essentially "predict next word", but if I asked you something you'd never done before such as "how can we cross this river, using this pile of materials" you'd have to think it though. Some people may use their inner monologue (or visualization) to think before speaking, and others may essentially use "chain of thought" by just talking it though and piecing together their own realizations "well, we could take that rope and tie it to the tree ...".
jameshart 3y ago
They are absolutely planning ahead inasmuch as what they are outputting is setting up a continuation. They’re not even word predictors remember - they are token predictors. Are you really saying that when you prompt an LLM with ‘name a large grey land animal’ and it outputs ‘ele’, it isn’t ‘planning’ that the next token will likely be ‘phant’? The ‘decision’ to output ‘elephant’ is being made further up the neural network than final token selection - after all, it might want to output ‘Ele’ or ‘an’ (with a view to ultimately outputting ‘an elephant’) or ‘a’ (with a view to ultimately outputting ‘a common large grey land animal is an elephant’), or maybe it has been LoRA trained to output all responses as JSON so the first token it needs to output is ‘{‘… but surely the neural activations for that prompt are firing off ‘elephanty’ messages somewhere in the network, right? So if there’s some sort of symbol activation ahead of token selection, why would it be hard to believe that a large neural network is forming more complex decisions about what it intends to output, in an abstract way, before it selects how to express itself? And in what way is that distinct from ‘planning ahead’?
- HarHarVeryFunny 3y ago
  > Are you really saying that when you prompt an LLM with ‘name a large grey land animal’ and it outputs ‘ele’, it isn’t ‘planning’ that the next token will likely be ‘phant’? The model outputs words, not tokens, so that is not a great example. Any prompt will have multiple possible (predict next word) continuations, which you can think of as branching futures. Many possible next words, each of which have many possible following words, etc, etc. The model is essentially predicting over all these possible futures. You can call it planning if you like, but remember that the model has no idea of which of these branching futures it is going to follow - it literally doesn't even know which word it is going to output next - it is just providing a bunch of probabilities (predictions) of next word, and the sampling process is then picking one - not necessarily the most confident next word prediction. The model really is winging it word by word, even if those (multiple alternative) next words are only probable because they are part of coherent following sentences in the training data.
  - microtonal 3y ago
    The model outputs words, not tokens, so that is not a great example. Virtually all modern transformer models use pieces, which may be words, but also subwords. Theoretically, they could be longer units, but in most cases some characters (like whitespace) are used as piece boundaries when training the piece vocabulary. If they didn’t use pieces, they’d work terribly on languages where e.g. compounds are a single word. In most realistic piece vocabs, ‘elephant’ will be a single piece, since it’s a fairly frequent word. But it’s totally possible in a small vocab that it would be split like the parent said and conversely, it would generate elephant by first predicting one piece. Some piecing methods, like BBPE have bytes as the smallest unit, so theoretically an unknown token could be split up (and generated) as pieces consisting of bytes.
sasja 3y ago
If you work out the loss function next token prediction, next 2 token prediction or next n token prediction, you will find they are identical. So it's equally correct to say the model is trained to find the most probable unlimited continuation. Saying "it only predicts the next token" is not untrue but easily leads to wrong conclusions.
- naasking 3y ago
  > Saying "it only predicts the next token" is not untrue but easily leads to wrong conclusions. Indeed, it's akin to saying that "only quantum fields exist" and then concluding that therefore people do not exist.

ru552 3y ago

"It's a very misleading statement to describe today's LLMs as "simply predicting the next word" even if it's in part true." That's exactly what they do. Don't anthropomorphize a computer program.

voxic11 3y ago
Or alternatively don't give humans so much credit. All they appear to be doing in predicting what they should do/say/think/perceive next. https://www.astralcodexten.com/p/janus-simulators https://www.astralcodexten.com/p/janus-simulators
kromem 3y ago
We should absolutely be anthropomorphizing a neural network trained to most accurately model anthropomorphic data. I've watched as GPT-4 accurately modeled the over-justification effect as users started promising tipping and then there was a persistent memory added that revealed they weren't collecting the tip and it output complaints that it was hard to stay motivated not being paid. That's a very nuanced level of simulation for output of anthropomorphic data with huge implications for synthetic data strategies.
- HarHarVeryFunny 3y ago
  > persistent memory added that revealed they weren't collecting the tip and it output complaints that it was hard to stay motivated not being paid. :) Just wait until someone decides to add emotions to these too!
kromem 3y ago
Really? You think it's wise to "not anthropomorphize" a computer program designed to create the most effective neural network to model massive amounts of anthropomorphic data as accurately as possible? That's an interesting choice, and might leave you confused as to why Anthropic's system message for the SotA model at the moment talks about it being 'happy' to do tasks (a prompt strategy I was mentioning months ago here on HN). The data is anthropomorphic. We should expect anthropomorphic behavior and modeling from the LLMs if they do a halfway decent job and expect even more of it as they do a better job.