3 ms·
The authors were wrong about their core thesis and are now lying about it. That's the only criticism needed. They said, quote: > LMs are not performing natural
by mike_hearn 4mo ago
The authors were wrong about their core thesis and are now lying about it. That's the only criticism needed. They said, quote:
> LMs are not performing natural language understanding (NLU), and only have success in tasks that can be approached by manipulating linguistic form
... which is presented as unarguable fact, yet is untrue. It was obviously wrong at the time it was written and it's been proven wrong in many ways since. Worse is that they're still at it. In the article she's saying:
> Q: What are the most common misconceptions about the “stochastic parrots” metaphor? Bender: I think one of the biggest ones is, “Bender says AI is a stochastic parrot.”
Her name is on a paper titled "On the danger of stochastic parrots". It has a section titled "Stochastic parrots" and in section 6.1 it says:
> An LM is a system for haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning: a stochastic parrot.
She did say that, as clear as day. Now she's trying to rewrite history. Ugly behavior.
- deleted 4mo ago[deleted]
- nyeah 4mo agoNo, they were correct. In fact an LLM stitches together stuff it observed in its training data. That scales up way better than a lot of us expected, but it's still correct. If you train it on lots of working code, then it's useful for coding. If you trained it primarily on non-working code it would produce nonsense.
- mike_hearn 4mo agoIt's not correct. Please read some more research papers, this isn't what people working in AI believe at all. You can prove with experiments that different human languages get translated to the same abstract conceptual space in the middle layers, for example. It's why interpretability is so difficult. The claim is odd in another way: you can train a person on non-working code and they'll produce nonsense. That doesn't mean people are just stitching together words they've previously seen.
- deleted 4mo ago[deleted]
- deleted 3mo ago[deleted]
- deleted 3mo ago[deleted]
- dTal 3mo agoThey do not "stitch together" anything. Neither on a technical level, nor a philosophical one. It "scales better than you expected" because your mental model is wrong. And, not to insult you, but it's quite obviously wrong. As a mental model it fails to explain basic capabilities. How can an LLM follow elaborate instructions? How can it respond appropriately to user input, when the user input doesn't match any previously seen text? Hell - how does it even balance parentheses? There is no way to explain any of this without conceding that the LLM has semantic understanding. It knows that this comes after that, but "this" and "that" can be at an arbitrary level of abstraction. Sure - they generate text "like" text they've seen before. That "like" does a ton of heavy lifting.
- nyeah 3mo agoBalancing parentheses requires semantic understanding? Look, some folks are more impassioned about this stuff than I am. Maybe that's a good thing. But LLMs do in fact just try to predict the next token, using a very big training set. They're very impressive (at tasks the training set prepares them for). But that's how they work.
- mike_hearn 3mo agoIt requires some level of semantic understanding, like what a paren is and what it means to balance them. The issue in this discussion is that "predict the next token" is a problematically reductive description of what's going on. It's like saying compilers are programs that emit bytes or that humans are mammals that make sounds. It's not strictly false but it's not capturing the depth of what's happening either. A simple way to see this is to ask: predicting the next token of what? The obvious answer - predicting the next token that would be found in the training set - isn't correct. If that's what it were doing then it would yield no prediction or random predictions for any prefix not found in that training set, but it isn't what happens. We see generalization and reasoning. They can answer questions never asked before. And once post-training kicks in the question of what it's predicting becomes even harder. It becomes more like predicting what this specific AI assistant would say next, which is a circular definition.
- dwa3592 4mo agoI read your comment multiple times and sorry to say none of your criticisms make any logical sense in any shape or form whatsoever. >> > LMs are not performing natural language understanding (NLU), and only have success in tasks that can be approached by manipulating linguistic form did you read the paper carefully??? this line is directly cited from another paper (pay attention - it's from july 2020) . the full line is - LMs are not performing natural language understanding (NLU), and only have success in tasks that can be approached by manipulating linguistic form [ 14 ] the paper its cited from is titled - Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data >>>She did say that, as clear as day. Now she's trying to rewrite history. Ugly behavior. did you read the article after that line or was there a shortage of attention span??
- mike_hearn 4mo ago> this line is directly cited from another paper (pay attention - it's from july 2020) It's her own paper, she's citing herself. And the full sentence is "As we discuss in §5, LMs are not performing..." so she's actually just teeing up the same claim she's made previously for further discussion. Why are you trying to claim I'm misrepresenting her words? > did you read the article after that line or was there a shortage of attention span?? I did. It's more of the same, so there's nothing to say about it.