3 ms·

No, they were correct. In fact an LLM stitches together stuff it observed in its training data. That scales up way better than a lot of us expected, but it's

• mike_hearn 3mo ago
  It's not correct. Please read some more research papers, this isn't what people working in AI believe at all. You can prove with experiments that different human languages get translated to the same abstract conceptual space in the middle layers, for example. It's why interpretability is so difficult. The claim is odd in another way: you can train a person on non-working code and they'll produce nonsense. That doesn't mean people are just stitching together words they've previously seen.
  • deleted 3mo ago
    [deleted]
  • deleted 3mo ago
    [deleted]
  • deleted 3mo ago
    [deleted]
• dTal 3mo ago
  They do not "stitch together" anything. Neither on a technical level, nor a philosophical one. It "scales better than you expected" because your mental model is wrong. And, not to insult you, but it's quite obviously wrong. As a mental model it fails to explain basic capabilities. How can an LLM follow elaborate instructions? How can it respond appropriately to user input, when the user input doesn't match any previously seen text? Hell - how does it even balance parentheses? There is no way to explain any of this without conceding that the LLM has semantic understanding. It knows that this comes after that, but "this" and "that" can be at an arbitrary level of abstraction. Sure - they generate text "like" text they've seen before. That "like" does a ton of heavy lifting.
  • nyeah 3mo ago
    Balancing parentheses requires semantic understanding? Look, some folks are more impassioned about this stuff than I am. Maybe that's a good thing. But LLMs do in fact just try to predict the next token, using a very big training set. They're very impressive (at tasks the training set prepares them for). But that's how they work.
    • mike_hearn 3mo ago
      It requires some level of semantic understanding, like what a paren is and what it means to balance them. The issue in this discussion is that "predict the next token" is a problematically reductive description of what's going on. It's like saying compilers are programs that emit bytes or that humans are mammals that make sounds. It's not strictly false but it's not capturing the depth of what's happening either. A simple way to see this is to ask: predicting the next token of what? The obvious answer - predicting the next token that would be found in the training set - isn't correct. If that's what it were doing then it would yield no prediction or random predictions for any prefix not found in that training set, but it isn't what happens. We see generalization and reasoning. They can answer questions never asked before. And once post-training kicks in the question of what it's predicting becomes even harder. It becomes more like predicting what this specific AI assistant would say next, which is a circular definition.
      • deleted 3mo ago
        [deleted]
      • nyeah 3mo ago
        In fact LLMs are trained to predict the next token in the training set. Of course sometimes a new text input doesn't match the training set, or it matches two or more places in the training set. LLMs use a neural network to interpolate, so that's fine. Please look this up if you have any doubts. Ok. Now. I think you're adding something to the description above. Maybe what you're describing is something "emergent," or maybe it's basically just word vectors that were built in on purpose. You may be adding something correct, or something incorrect. Fine. But it's not reasonable to say that the "reductive" description above is a "lie". It's not. It's more like a recipe. If you look at correct instructions for making steak, and you call the author a "liar" then you are missing something important.