4 ms·

The human stochastic parrots (GP, not you) spouting these 2023 talking points really need to update their weights. I’m guessing this way of thinking has a stick

The human stochastic parrots (GP, not you) spouting these 2023 talking points really need to update their weights. I’m guessing this way of thinking has a stickiness because thinking of an LLM as “just a fancy markov chain” makes them feel less threatening to some people (we’re past the point where it could be good faith reasoning). Like, I hear people say things like that (or that coding agents can only do web development, or that they can only write code from their training data), and then I look at Claude Code on my computer, currently debugging embedded code on a peripheral while also troubleshooting the app it’s connected to, and I’m struck by how clearly out of touch with reality a lot of the LLM cope is. People need to stop obsessing over “the out of control hype” and reckon with the thing that’s sitting in front of them.

teucris 1y ago
I think there’s a bit of parroting going around but LLMs are predictive and there’s a lot you can inuit a lot about how they behave just on that fact alone. Sure, calling it “token” prediction is oversimplifying things, but stating that, by their nature, LLMs are guessing at the next most likely thing in the scenario (next data structure needing to be coded up, next step in a process, next concept to cover in a paragraph, etc.) is a very useful mental model.
- bt1a 1y ago
  I would challenge the utility of this mental model as again they're not simply tracing a "most likely" path unless your sampling methods are trivially greedy. I don't know of a better way to model it, and I promise I'm not trying to be anal here
  - teucris 1y ago
    “All models are wrong, but some are useful.” Agreed - I picked certain words to be intentionally ambiguous eg “most likely” since it provides an effective intuitive grasp of what’s going on, even if it’s more complicated than that.
- Uehreka 1y ago
  Honestly, I think the best way to reason about LLM behavior is to abandon any sort of white-box mental model (where you start from things you “know” about their internal mechanisms). Treat them as a black box, observe their behavior in many situations and over a long period of time, draw conclusions from the patterns you observe and test if your conclusions have predictive weight. Of course, if someone is predisposed to incuriosity about LLMs and refuses to use them, they won’t be able to participate in that approach. However I don’t think there’s an alternative.
  - libraryofbabel 1y ago
    This is precisely what I recommend to people starting out with LLMs: do not start with the architecture, start with their behavior - use them for a while as a black box and then circle back and learn about transformers and cross entropy loss functions and whatever. Bottom-up approaches to learning work well in other areas of computing, but not this - there is nothing in the architecture to suggest the emergent behavior that we see.
    - teucris 1y ago
      This is more or less how I came to the mental model I have that I refer to above. It helps me tremendously in knowing what to expect from every model I’ve used.
  - anthem2025 1y ago
    So just ignore everything you actually know until you can fool yourself into thinking fancy auto complete is totally real intelligence? Why not apply that to computers in general and then we can all worship the magic boxes.
libraryofbabel 1y ago
You’re being downvoted, perhaps because your tone is a little harsh, but you’re not wrong: people really are still making versions of the “stochastic parrots” argument. It comes up again and again, on hacker news and elsewhere. And yet a few months ago an LLM got gold on the Mathematical Olympiad. “Stochastic parrots” just isn’t a useful metaphor anymore. I find AI hype as annoying as anyone, and LLMs do have all sorts of failure modes, some of which are related to how they are trained. But at this point they are doing things that many people (including me) would have flatly denied was possible with this architecture 3 years ago during the initial ChatGPT hype. When the facts change we need to change our opinions, and like you say, reckon anew with the thing that’s sitting in front of us.
anthem2025 1y ago
Nah it can still be entirely on good faith. Not everyone is an easily impressed and convinced that fancy autocomplete is going to suddenly spontaneously develop intelligence.

teucris 1y ago

I think there’s a bit of parroting going around but LLMs are predictive and there’s a lot you can inuit a lot about how they behave just on that fact alone. Sure, calling it “token” prediction is oversimplifying things, but stating that, by their nature, LLMs are guessing at the next most likely thing in the scenario (next data structure needing to be coded up, next step in a process, next concept to cover in a paragraph, etc.) is a very useful mental model.

bt1a 1y ago
I would challenge the utility of this mental model as again they're not simply tracing a "most likely" path unless your sampling methods are trivially greedy. I don't know of a better way to model it, and I promise I'm not trying to be anal here
- teucris 1y ago
  “All models are wrong, but some are useful.” Agreed - I picked certain words to be intentionally ambiguous eg “most likely” since it provides an effective intuitive grasp of what’s going on, even if it’s more complicated than that.
Uehreka 1y ago
Honestly, I think the best way to reason about LLM behavior is to abandon any sort of white-box mental model (where you start from things you “know” about their internal mechanisms). Treat them as a black box, observe their behavior in many situations and over a long period of time, draw conclusions from the patterns you observe and test if your conclusions have predictive weight. Of course, if someone is predisposed to incuriosity about LLMs and refuses to use them, they won’t be able to participate in that approach. However I don’t think there’s an alternative.
- libraryofbabel 1y ago
  This is precisely what I recommend to people starting out with LLMs: do not start with the architecture, start with their behavior - use them for a while as a black box and then circle back and learn about transformers and cross entropy loss functions and whatever. Bottom-up approaches to learning work well in other areas of computing, but not this - there is nothing in the architecture to suggest the emergent behavior that we see.
  - teucris 1y ago
    This is more or less how I came to the mental model I have that I refer to above. It helps me tremendously in knowing what to expect from every model I’ve used.
- anthem2025 1y ago
  So just ignore everything you actually know until you can fool yourself into thinking fancy auto complete is totally real intelligence? Why not apply that to computers in general and then we can all worship the magic boxes.

libraryofbabel 1y ago

You’re being downvoted, perhaps because your tone is a little harsh, but you’re not wrong: people really are still making versions of the “stochastic parrots” argument. It comes up again and again, on hacker news and elsewhere. And yet a few months ago an LLM got gold on the Mathematical Olympiad. “Stochastic parrots” just isn’t a useful metaphor anymore. I find AI hype as annoying as anyone, and LLMs do have all sorts of failure modes, some of which are related to how they are trained. But at this point they are doing things that many people (including me) would have flatly denied was possible with this architecture 3 years ago during the initial ChatGPT hype. When the facts change we need to change our opinions, and like you say, reckon anew with the thing that’s sitting in front of us.

anthem2025 1y ago

Nah it can still be entirely on good faith. Not everyone is an easily impressed and convinced that fancy autocomplete is going to suddenly spontaneously develop intelligence.