3 ms·

If someone can look at that reasoning trace and see a stochastic parrot next word prediction machine, we don't understand those words in the same way.

sneak 1mo ago
Yeah, it has been clear for a long time that there is reasoning and mental modeling going on here. The other option is that you do understand those words the same way, and the people making these (now nonsensical) anti-AI claims simply aren’t talking about the same programs/models we are. Their idea of SOTA is when chatgpt.com launched. If you took a point sample pre-Opus, and didn’t write a good prompt, of course you would think all AI programming was worthless slop.
- deleted 1mo ago
  [deleted]
- negura 1mo ago
  Maybe get yourself checked for chatbot psychosis. I am using current models productively, every day, and have 0 (and I mean precisely, literally 0) issue with calling it a stochastic parrot, one which lacks any kind of mentality whatsoever. There is not a shred of doubt in my mind that this is purely a statistical model, generating sequences of words, that happen to make sense in our actual mentality.
  - sneak 1mo ago
    What about my proposed psychosis would make the tests go green when it implements something complex in one shot? In any case, thanks for validating my previous assumption that people like this still exist even in the age of Opus/Fable.
    - negura 1mo ago
      > What about my proposed psychosis would make the tests go green when it implements something complex in one shot? Nothing. That's why it's a psychosis. It has nothing to do with the reality of how language models generate words without mental reasoning.
      - sneak 1mo ago
        The “something complex” is critical here - I mean specifically an implementation that requires an advanced and reasoned mental model to write. How does it oneshot those and pass comprehensive tests if it isn’t doing advanced reasoning?
        negura 1mo ago
        Because it's a statistical model of the words used in similar problem domains.
- weego 1mo ago
  it has been clear for a long time that there is reasoning and mental modeling going on here There is not. No one from these products is even claiming that's the case and they're so desperate to make the next big claim to re-ignite investment they'd be shouting it from every rooftop. It's just breaking out all the reasonable probabilities around what it's been tasked with and structuring them in a way that is designed to actively look human, and then feed it back to itself. Fundamentally that's the easiest way to iterate new features when the underlying architecture of LLMs is largely "fixed" right now. The fact it is output in a way that appears to reason through each is just a technical decision that creates an illusion of reasoning.
  - gjm11 1mo ago
    What, as precisely as you can say, is the difference between an illusion of reasoning and reasoning? (I am not claiming that there is none. But I personally would define "reasoning" in terms of its structure and its results, and it looks to me as if the best LLMs' "illusion of reasoning" has enough similarities in structure and results to much human reasoning that I don't see why we shouldn't also call it reasoning; if your opinion differs then I'm curious about where the disagreements lie. E.g., do we have different beliefs about what sort of thing LLMs' schmeasoning is able to accomplish, or does your notion of "reasoning" specifically require that it be done by humans, or what?)
dnautics 1mo ago
The stochastic parrot epithet is so 4 months ago
0xfaded 1mo ago
I still call them stochastic parrots, but believe what they are revealing is that we are all stochastic parrots to some extent. I simply don't see how biological computation (i.e. thinking) can be anything else. Similar to the reveal in west world, we are likely much simpler than we give ourselves credit for. A "train of thought" can be seen as a trace of a depth first search where the preceding trace is used to guide termination and next expansion decisions. A similar concept, "taboo search", exists in classical constraint optimization where previous solutions are fit to a model that guides future expansion (but as the name "taboo" implies, away from uninteresting solutions). We also have harnesses that perform breath first search. If I tried to describe what it means to "think deeply", I would probably say a combination of both. Ultimately I believe that we will surpass human capabilities but fail with alignment. Handing the world's resources over to stochastic systems that can evolve faster than we can reason about them simply leaves too many "interesting" outcomes that do not end well. I also expect the failure modes will be totally non-obvious.
- vasco 1mo ago
  As long as there's enough of them with different goals it doesn't matter, they'll keep each other in check. The worlds resources are already handed over to the worst people and we're still doing fine and none of the billionaires are "aligned with society". They just align with their own belly but because they want different things it all kinda works.
  - sujzhsbnwjek 1mo ago
    These “worst people” need you. They physically need you alive to perform labor for them and to give them money (and status). That’s the reason we are “doing fine”. Once they stop needing you.. Also, both our comments brush over the generational struggles for fairness over the centuries. We have fought to be “fine”, it did not just happen. Without fairness being introduced by force you and I would be slaving away in some sweatshop getting paid nickels as was the norm not so long ago. Edit: That’s also assuming you are Caucasian. If you are of a different ethnicity.. well, historically, all bets are off. You could also be the literal possession of some of these “worst people” with not even your own children considered yours.
pasteleft 1mo ago
LLM is "stochastic parrot next word prediction machine"; it's just that this "stochastic parrot next word prediction machine" have proven to be smarter than most people. I mean, this already happened with AlphaGo too.
- BoredomIsFun 1mo ago
  > to be smarter than most people. Hell no. In very narrow tasks - yes, in vast majority, esp. involving state tracking (board games) and spatial reasoning - they are awful.
slopinthebag 1mo ago
Why can't a next token prediction machine not predict a train of reasoning?

sneak 1mo ago

Yeah, it has been clear for a long time that there is reasoning and mental modeling going on here. The other option is that you do understand those words the same way, and the people making these (now nonsensical) anti-AI claims simply aren’t talking about the same programs/models we are. Their idea of SOTA is when chatgpt.com launched. If you took a point sample pre-Opus, and didn’t write a good prompt, of course you would think all AI programming was worthless slop.

deleted 1mo ago
[deleted]
negura 1mo ago
Maybe get yourself checked for chatbot psychosis. I am using current models productively, every day, and have 0 (and I mean precisely, literally 0) issue with calling it a stochastic parrot, one which lacks any kind of mentality whatsoever. There is not a shred of doubt in my mind that this is purely a statistical model, generating sequences of words, that happen to make sense in our actual mentality.
- sneak 1mo ago
  What about my proposed psychosis would make the tests go green when it implements something complex in one shot? In any case, thanks for validating my previous assumption that people like this still exist even in the age of Opus/Fable.
  - negura 1mo ago
    > What about my proposed psychosis would make the tests go green when it implements something complex in one shot? Nothing. That's why it's a psychosis. It has nothing to do with the reality of how language models generate words without mental reasoning.
    - sneak 1mo ago
      The “something complex” is critical here - I mean specifically an implementation that requires an advanced and reasoned mental model to write. How does it oneshot those and pass comprehensive tests if it isn’t doing advanced reasoning?
      - negura 1mo ago
        Because it's a statistical model of the words used in similar problem domains.
weego 1mo ago
it has been clear for a long time that there is reasoning and mental modeling going on here There is not. No one from these products is even claiming that's the case and they're so desperate to make the next big claim to re-ignite investment they'd be shouting it from every rooftop. It's just breaking out all the reasonable probabilities around what it's been tasked with and structuring them in a way that is designed to actively look human, and then feed it back to itself. Fundamentally that's the easiest way to iterate new features when the underlying architecture of LLMs is largely "fixed" right now. The fact it is output in a way that appears to reason through each is just a technical decision that creates an illusion of reasoning.
- gjm11 1mo ago
  What, as precisely as you can say, is the difference between an illusion of reasoning and reasoning? (I am not claiming that there is none. But I personally would define "reasoning" in terms of its structure and its results, and it looks to me as if the best LLMs' "illusion of reasoning" has enough similarities in structure and results to much human reasoning that I don't see why we shouldn't also call it reasoning; if your opinion differs then I'm curious about where the disagreements lie. E.g., do we have different beliefs about what sort of thing LLMs' schmeasoning is able to accomplish, or does your notion of "reasoning" specifically require that it be done by humans, or what?)

dnautics 1mo ago

The stochastic parrot epithet is so 4 months ago

0xfaded 1mo ago

I still call them stochastic parrots, but believe what they are revealing is that we are all stochastic parrots to some extent. I simply don't see how biological computation (i.e. thinking) can be anything else. Similar to the reveal in west world, we are likely much simpler than we give ourselves credit for. A "train of thought" can be seen as a trace of a depth first search where the preceding trace is used to guide termination and next expansion decisions. A similar concept, "taboo search", exists in classical constraint optimization where previous solutions are fit to a model that guides future expansion (but as the name "taboo" implies, away from uninteresting solutions). We also have harnesses that perform breath first search. If I tried to describe what it means to "think deeply", I would probably say a combination of both. Ultimately I believe that we will surpass human capabilities but fail with alignment. Handing the world's resources over to stochastic systems that can evolve faster than we can reason about them simply leaves too many "interesting" outcomes that do not end well. I also expect the failure modes will be totally non-obvious.

vasco 1mo ago
As long as there's enough of them with different goals it doesn't matter, they'll keep each other in check. The worlds resources are already handed over to the worst people and we're still doing fine and none of the billionaires are "aligned with society". They just align with their own belly but because they want different things it all kinda works.
- sujzhsbnwjek 1mo ago
  These “worst people” need you. They physically need you alive to perform labor for them and to give them money (and status). That’s the reason we are “doing fine”. Once they stop needing you.. Also, both our comments brush over the generational struggles for fairness over the centuries. We have fought to be “fine”, it did not just happen. Without fairness being introduced by force you and I would be slaving away in some sweatshop getting paid nickels as was the norm not so long ago. Edit: That’s also assuming you are Caucasian. If you are of a different ethnicity.. well, historically, all bets are off. You could also be the literal possession of some of these “worst people” with not even your own children considered yours.

pasteleft 1mo ago

LLM is "stochastic parrot next word prediction machine"; it's just that this "stochastic parrot next word prediction machine" have proven to be smarter than most people. I mean, this already happened with AlphaGo too.

BoredomIsFun 1mo ago
> to be smarter than most people. Hell no. In very narrow tasks - yes, in vast majority, esp. involving state tracking (board games) and spatial reasoning - they are awful.

slopinthebag 1mo ago

Why can't a next token prediction machine not predict a train of reasoning?