3 ms·

What humanity has achieved here is incredible. We couldn’t even build an idiot for decades. What you’re referring to is popular opinion. AI has become so perva

What humanity has achieved here is incredible. We couldn’t even build an idiot for decades. What you’re referring to is popular opinion. AI has become so pervasive in our lives that we are used to it and the magnitude of achievement has been lost on us. The fact that it went from stochastic parrot to idiot savant to crappy consultant is from people in denial about reality and then slowly coming to terms with it. In the beginning literally everyone on HN called it a stochastic parrot with the authority of an expert. Clearly they were all wrong.

SketchySeaBeast 1y ago
Oh, it's still a stochastic parrot. What changed is that people realized it didn't have the authority of an expert. What's a stochastic parrot with dubious authority? It's a crappy consultant.
- ninetyninenine 1y ago
  It’s not. People in academia are not using this term anymore because it’s utterly clear it can output knowledge that doesn’t exist.
  - SketchySeaBeast 1y ago
    > output knowledge that doesn’t exist So can parrots. They'll gladly generate neologisms. I'm interested in how academics define "knowledge that doesn't exist".
    - ninetyninenine 1y ago
      Of course parrots can output knowledge that doesn’t exist. Stochastic parrot is a different term. > "knowledge that doesn't exist". I said that term. So there’s no official definition but you already know that. Basically it’s clear among everyone academics included that LLMs can rudimentarily do what humans do. That means composing knowledge and working things out to form new knowledge that doesn’t exist.
      - SketchySeaBeast 1y ago
        "Stochastic Parrot" implies that the thing producing the noise doesn't understand it. I'm not sure how that's currently disproven. Even acting as an agent, it is my understanding that it's just acting on it's own messages in the exact same way it'd act on one of ours. > That means composing knowledge and working things out to form new knowledge that doesn’t exist. That's not a terribly useful criteria, though. A Markov chain can produce novel sentences, hell a bingo machine can if you write words on the balls. "Knowledge" is kind of meaningless but also seemingly profound.
        ninetyninenine 1y ago
        > That's not a terribly useful criteria, though. A Markov chain can produce novel sentences, hell a bingo machine can if you write words on the balls. "Knowledge" is kind of meaningless but also seemingly profound. I don’t know why you came up with this pedantic example. Perhaps you’re autistic? If so then I apologize for assuming you aren’t. Everyone knows that we are talking about more than just knowledge consisting of a random sting of letters. We are talking about actual useful knowledge.
        SketchySeaBeast 1y ago
        I don't know if you appreciate how this argument has gone from arguing how "everyone academic" understands LLMs to arguing things that "everyone knows", and now you're othering me when I don't find you're increasingly tenuous arguments untenable. A certain amount of pedantry is required for these discussions, otherwise we're left in a place where we can't define "actual useful knowledge". At this moment I assume you're defining "actual useful knowledge" as simply anything you find convincing, which is a criteria that could be easily gamed. How are you determining that knowledge is actually novel?
        ninetyninenine 1y ago
        I’m not willing to go into that level of “pedantry”. I like to assume I’m talking with people that have relevant context so we don’t have to go into stupid detail and assume random data generated by a freaking random number generator constitutes as knowledge.
bee_rider 1y ago
Were they wrong to call it a stochastic parrot, or was there some wrong implication about the usefulness of such a parrot?
- daveguy 1y ago
  Turns out, sometimes you want the string "polly-want-a-cracker"* in your codebase. * where "polly-want-a-cracker" is some form of existing, common fizz-buzz-ish code.
- ninetyninenine 1y ago
  The usefulness of an LLM is similar to the usefulness of a baby, but higher. The term stochastic parrot has nothing to do with usefulness and everything to do with the existential meaning of whether this ai is repeating what it is taught or creatively forming new knowledge from logic and composition from previous knowledge. It is categorically unequivocal that LLMs do not just parrot previous knowledge stochastically. They form new ideas from scratch.
deleted 1y ago
[deleted]

SketchySeaBeast 1y ago

Oh, it's still a stochastic parrot. What changed is that people realized it didn't have the authority of an expert. What's a stochastic parrot with dubious authority? It's a crappy consultant.

ninetyninenine 1y ago
It’s not. People in academia are not using this term anymore because it’s utterly clear it can output knowledge that doesn’t exist.
- SketchySeaBeast 1y ago
  > output knowledge that doesn’t exist So can parrots. They'll gladly generate neologisms. I'm interested in how academics define "knowledge that doesn't exist".
  - ninetyninenine 1y ago
    Of course parrots can output knowledge that doesn’t exist. Stochastic parrot is a different term. > "knowledge that doesn't exist". I said that term. So there’s no official definition but you already know that. Basically it’s clear among everyone academics included that LLMs can rudimentarily do what humans do. That means composing knowledge and working things out to form new knowledge that doesn’t exist.
    - SketchySeaBeast 1y ago
      "Stochastic Parrot" implies that the thing producing the noise doesn't understand it. I'm not sure how that's currently disproven. Even acting as an agent, it is my understanding that it's just acting on it's own messages in the exact same way it'd act on one of ours. > That means composing knowledge and working things out to form new knowledge that doesn’t exist. That's not a terribly useful criteria, though. A Markov chain can produce novel sentences, hell a bingo machine can if you write words on the balls. "Knowledge" is kind of meaningless but also seemingly profound.
      - ninetyninenine 1y ago
        > That's not a terribly useful criteria, though. A Markov chain can produce novel sentences, hell a bingo machine can if you write words on the balls. "Knowledge" is kind of meaningless but also seemingly profound. I don’t know why you came up with this pedantic example. Perhaps you’re autistic? If so then I apologize for assuming you aren’t. Everyone knows that we are talking about more than just knowledge consisting of a random sting of letters. We are talking about actual useful knowledge.
        SketchySeaBeast 1y ago
        I don't know if you appreciate how this argument has gone from arguing how "everyone academic" understands LLMs to arguing things that "everyone knows", and now you're othering me when I don't find you're increasingly tenuous arguments untenable. A certain amount of pedantry is required for these discussions, otherwise we're left in a place where we can't define "actual useful knowledge". At this moment I assume you're defining "actual useful knowledge" as simply anything you find convincing, which is a criteria that could be easily gamed. How are you determining that knowledge is actually novel?
        ninetyninenine 1y ago
        I’m not willing to go into that level of “pedantry”. I like to assume I’m talking with people that have relevant context so we don’t have to go into stupid detail and assume random data generated by a freaking random number generator constitutes as knowledge.

bee_rider 1y ago

Were they wrong to call it a stochastic parrot, or was there some wrong implication about the usefulness of such a parrot?

daveguy 1y ago
Turns out, sometimes you want the string "polly-want-a-cracker"* in your codebase. * where "polly-want-a-cracker" is some form of existing, common fizz-buzz-ish code.
ninetyninenine 1y ago
The usefulness of an LLM is similar to the usefulness of a baby, but higher. The term stochastic parrot has nothing to do with usefulness and everything to do with the existential meaning of whether this ai is repeating what it is taught or creatively forming new knowledge from logic and composition from previous knowledge. It is categorically unequivocal that LLMs do not just parrot previous knowledge stochastically. They form new ideas from scratch.

deleted 1y ago

[deleted]