8 ms·

Author here -- I think it is a stochastic parrot, don't get me wrong. But my argument is that humans are also, at some level, sufficiently advanced stochastic

Author here -- I think it is a stochastic parrot, don't get me wrong. But my argument is that humans are also, at some level, sufficiently advanced stochastic parrots. At least, as it pertains to many creative endeavors. You're always building on the back of something that comes before. We're all just remixing ideas we've heard before. I mean even this comment -- none of the words I'm saying are new, and many of the word combinations I'm using have been used time and time again. The ideas I'm expressing have merit -- but are they wholly original? Not really. We all need eachother's creative energies to do our best work.

HervalFreire 4y ago
yeah, but colloquially when people start using the term "stochastic parrot" they're referring to something far stupider. From a Technical and pedantic standpoint it could be said ALL intelligence is isomorphic to a stochastic parrot so it's not reasonable to assume in common communication that people are referring to the technical definition when they talk. Your point in the article is basically saying that chatGPT is not as stupid as people think and also suggesting the fact that humans can be stupid in a similar way to chatGPT as well.
- BoiledCabbage 4y ago
  > yeah, but colloquially when people start using the term "stochastic parrot" they're referring to something far stupider. From a Technical and pedantic standpoint it could be said ALL intelligence is isomorphic to a stochastic parrot I mean that's somewhat his point (if I understand him correctly). All intelligence could effectively be described as a "stochastic parrot" as a result, calling something a "stochastic parrot" is a roughly meaningless insult that is really a thinly-veiled way of saying "no it's dumber than me" without actually using those words. And his point is, it's essentially a defense mechanism in people when challenged by something that is much smarter than expected to overly emphasis it's mistakes and fall back to "no it'd dumber than me" than evaluate it for what it actually is. I think that's the issue. It's a defense mechanism. If you replace "stochastic parrot" in most of those comments with "it's dumber than me" you see what the comment essentially is. "It's dumber than me. I'm smarter than it, I don't need to be worried about it".
  - HervalFreire 4y ago
    >I mean that's somewhat his point His point is humans and GPT are both stochastic parrots. I'm saying ALL intelligence even the one in an ant or a chess AI is a stochastic parrot. Thus it's pointless to use this term to compare intelligence. Therefore people must not actually be referring to the technical definition when they use the word "stochastic parrot." >If you replace "stochastic parrot" in most of those comments with "it's dumber than me" you see what the comment essentially is. "It's dumber than me. I'm smarter than it, I don't need to be worried about it". Yes this is Exactly What I was saying in the post you replied to. You and I are in agreement.
    - RHSman2 4y ago
      I’m dumber than you
  - Jensson 4y ago
    A person who does understand how new technology works will see it as a dumb contraption. A person who doesn't understand how it works will see it as magic.
    - famouswaffles 4y ago
      Lol no. That's just what you tell yourself to feel smart. They're are many who understand that don't think of it as dumb. "I think GPT-3 is artificial general intelligence, AGI. I think GPT-3 is as intelligent as a human. And I think that it is probably more intelligent than a human in a restricted way… in many ways it is more purely intelligent than humans are. I think humans are approximating what GPT-3 is doing, not vice versa.” — Connor Leahy, co-founder of EleutherAI, creator of GPT-J (November 2020)
      - Jensson 4y ago
        A founder trying to sell a product by telling you the product is magic? Yeah, very trustworthy source there.
eternalban 4y ago
> none of the words I'm saying are new, and many of the word combinations I'm using have been used time and time again. The ideas I'm expressing have merit -- but are they wholly original? Then why are you saying it? But besides that query, your "word combination" bit elicited a 'so what?' > We're all just remixing ideas we've heard before. Something wrong with the grammar in that sentence. (Or do you go by we/us?)
jmcgeeney 4y ago
The difference between humans and sophisticated stochastic parrots is reason. The most common kinds of mistakes that chatGPT currently makes are when it says things that are not simply wrong, but don't make sense. Perhaps it will be possible to emulate reason with enough data, training, parameters, etc, but without some representation of the ability to understand what you don't know, what you know, and what follows from those things consistently, I wonder if these kinds of models will ever become truly reliable.
- ldhough 4y ago
  Although even in this example it does say some things that are simply wrong: "Some programming languages like TypeScript and Swift use a colon for both variable type annotations and function return types." This is incorrect for Swift, which uses the same arrows as "TenetLang" for function return types. Actually the first thing I thought looking at the example code was "looks swifty but not quite as well designed."
- galaxyLogic 4y ago
  > some representation of the ability to understand what you don't know, what you know, and what follows from those things consistently Right, that is truly what the chatbots are lacking. They can fool some of the people some of the time, but they can't fool all of the people all of the time. Their creators - not the chatbots themselves - are trying to "fool" people into believing that the chatbots would have as you say "ability to understand what you don't know, what you know, and what follows from those things consistently".
- HervalFreire 4y ago
  Except the fact that it does both. It commits mistakes AND it also comes up with remarkable, novel output that makes sense. Perhaps you just haven't seen an example of it that's convincing enough. https://www.engraved.blog/building-a-virtual-machine-inside/ https://www.engraved.blog/building-a-virtual-machine-inside/ Here's another one.
- raincole 4y ago
  It's totally possible that humans don't do reason. It's possible that the parrot in our brain makes the decision, and then the "frontend" of our mind makes fake reasons that sound logical enough. But it's just a possibility, and I don't find it's particularly convincing.
testrun 4y ago
Nope, we parrot mostly, but not always. When we do not, we create something new, and then we started parroting again. Mostly. The question is, can ai do something new, understand context of the new thing, understand when and where it will be usable and when it make sense to apply it.
- marviel 4y ago
  When we're creating something "new" -- are you positing that it isn't built on top of other ideas, being combined in new ways?
  - testrun 4y ago
    New can be 1. something without a predecessor, 2. synthesis of other ideas that creates totally new application or theory
    - marviel 4y ago
      Can you provide an example of 1? I think there are definitely some out there that feel almost "singular" in the way you describe, but each time I try to find a non-predecessor item I end up feeling "nope there are still dependencies"
      - testrun 4y ago
        Relativity for instance. New does not exclude dependencies or partial predecessors. New is something that is totally different from before.
        vidarh 4y ago
        Relativity comes from exploring the problem of the incompatibilities of pre-existing theories and experiments. I don't see anything that suggests it requires anything more than synthesis and a stochastic process to explore the gaps and inconsistencies. In terms of timing, they're the culmination of decades of pre-existing work, with e.g. the Lorentz transformations named the same year Einstein published his paper on special relativity.
galaxyLogic 4y ago
> The ideas I'm expressing have merit -- but are they wholly original? I think the difference is that your intention is to write sentences which you think and believe are correct and which can help other people understand the subject. Chatbot has no intentions of its own, it only imitates texts it has read from the internet. As far as it is concerned they might be totally wrong. You on the other hand have the ability -- and the desire -- to reason about what you are saying and think whether it is actually true or not.
- marviel 4y ago
  I'm not sure I can support the idea that the agent doesn't have the ability to reason -- try to give it any complex (text-based) puzzle you can think of and it'll do just about as well as an average person, oftentimes much better. And I think you may be wrong to say the chatbot doesn't have intentions -- it's intentions based on its training are to accurately predict the next character. It doesn't care in the same way we do, sure -- but you could make a case (and it will be made in courts in the next few years, I don't doubt) that these agents do have desires that are analogous to our own, by the nature of their training process. I don't know where that leaves us to be honest, but it's an interesting topic to discuss.
  - galaxyLogic 4y ago
    You say > it's intentions based on its training are to accurately predict the next character. then you say: > these agents do have desires that are analogous to our own I think its very in-human to have a single desire which is to predict the next character. That is not analogous to our desires. And the intention to predict the next character is not the intention of the chatbot, it is the intention of whoever created the chatbot or whoever is using it for that purpose. AI is a tool created by humans to fulfill the intentions of those humans. Is it the intention of a gun to kill people? No, it is the intention of whichever human who uses a gun for such a purpose. Is it the intention of AI to predict the next character? No that is the intention of the human who uses AI for such a purpose.
    - 4y ago
okaleniuk 4y ago
I would go even further. People are not even that advanced stochastic parrots. In the similar task, we failed just as spectacularly as the GPT-4 did right now. Rust, Zig, Julia, Kotlin, and dozens smaller "new" languages fail to bear a smear of originality in them. Their authors remixed old ideas in different proportions and got different grammars as a result. That's something an LLM can do and much faster too. But there is also Unison. That one new language that stands out, and the very fact that it does makes it improbable for an LLM to generate. LM is a language model, it avoids marginality and originality altogether by its design. Language models can and are useful if you specifically want to avoid marginality. To reduce noise, to remove errors. There is huge potential in them. If the problem was to design the most average programming language with no purpose, no market niche, and no technological context - then GPT-4 is clearly a winner.
- visarga 4y ago
  Small LMs do tend to avoid originality, but large LMs can use the context as a starting point to walk outside their training data distribution. They can do that by being better at composing concepts and using context-conditioning, which is also known as in-context learning.
- laserbeam 4y ago
  > Rust, Zig, Julia, Kotlin, and dozens smaller "new" languages fail to bear a smear of originality in them. To be fair, most programming languages don't try to be original. They try to solve problems. They also try to keep syntax close enough to other established programming treds so people have an easier time learning them.
- marviel 4y ago
  Thanks for pointing out Unison -- I'm definitely going to check it out. There is another language in this space I've had my eye on as well, but I can't recall the name at the moment.
- wruza 4y ago
  I would go even further. People are not even that advanced stochastic parrots. In the similar task, we failed just as spectacularly as the GPT-4 did right now. Rust, Zig, Julia, Kotlin, and dozens smaller "new" languages fail to bear a smear of originality in them. Their authors remixed old ideas in different proportions and got different grammars as a result. It wasn’t an originality contest, so there’s at least one non-sequitur in that statement. Everyone can create an original syntax, given modern character sets and formatting abilities. The key issue with that is the amount of developers who want to learn and switch between these is low.
travisgriggs 4y ago
I think the difference is that LLMs are essentially just stochastic parrots (SP) Humans are stochastic parrots and more. In some ways the SP behavior of LLMs already far outstrips the SP abilities of humans. And there are some leaps that humans' SP ablities will be hard to mimic. But at the end of the day, a human is still more than just an LLM. What exactly, we'll be debating that til the end of time likely. We know that an LLM can probably give us a pretty condensed summary of mankinds philosophical wranglings on the subject. :D
dirkt 4y ago
> But my argument is that humans are also, at some level, sufficiently advanced stochastic parrots. But humans have a feedback loop called "consciousness" that sits above the stochastic parrot level, can reason about it, and can separate the wheat from the chaff. AI still has to reach this level. ChatGPT cannot judge if its answers are right or wrong. It just claims things confidently, and as long as it is parroting correctly, it is right, and that's why humans get fooled into thinking that maybe they can trust it. (That's also why humans get fooled by human bullshitters who claim things confidently). So if you don't use ChatGPT as what it is, a lookup-tool trained on the content of the internet, which can't even tell you where it got the information from, and the onus is on you to verify if the information is correct or not, and if instead try to use it as some kind of expert you can trust (which I have seen quite a few people do), then you are doing it wrong.
- marviel 4y ago
  I don't disagree we're still better in a number of ways, but if GPT-4 was given a database, and a "while" loop -- do you think it would gain consciousness?
chimprich 4y ago
> We're all just remixing ideas we've heard before. We're largely remixing ideas we've heard before, but not just I think. If you're just remixing ideas that came before... where did those ideas come from? I think you can get quite far remixing ideas but for more novel concepts to emerge you have to actually create new stuff. In biology it seems that evolution has found it worthwhile not to minimise mutations too much. In genetic programming you need both mutation and crossover for evolution. I don't think LLMs really have that mutation of ideas (at the moment!), and I suspect humans create genuinely new concepts in a much more sophisticated way than mere random mutation.

HervalFreire 4y ago

yeah, but colloquially when people start using the term "stochastic parrot" they're referring to something far stupider. From a Technical and pedantic standpoint it could be said ALL intelligence is isomorphic to a stochastic parrot so it's not reasonable to assume in common communication that people are referring to the technical definition when they talk. Your point in the article is basically saying that chatGPT is not as stupid as people think and also suggesting the fact that humans can be stupid in a similar way to chatGPT as well.

BoiledCabbage 4y ago
> yeah, but colloquially when people start using the term "stochastic parrot" they're referring to something far stupider. From a Technical and pedantic standpoint it could be said ALL intelligence is isomorphic to a stochastic parrot I mean that's somewhat his point (if I understand him correctly). All intelligence could effectively be described as a "stochastic parrot" as a result, calling something a "stochastic parrot" is a roughly meaningless insult that is really a thinly-veiled way of saying "no it's dumber than me" without actually using those words. And his point is, it's essentially a defense mechanism in people when challenged by something that is much smarter than expected to overly emphasis it's mistakes and fall back to "no it'd dumber than me" than evaluate it for what it actually is. I think that's the issue. It's a defense mechanism. If you replace "stochastic parrot" in most of those comments with "it's dumber than me" you see what the comment essentially is. "It's dumber than me. I'm smarter than it, I don't need to be worried about it".
- HervalFreire 4y ago
  >I mean that's somewhat his point His point is humans and GPT are both stochastic parrots. I'm saying ALL intelligence even the one in an ant or a chess AI is a stochastic parrot. Thus it's pointless to use this term to compare intelligence. Therefore people must not actually be referring to the technical definition when they use the word "stochastic parrot." >If you replace "stochastic parrot" in most of those comments with "it's dumber than me" you see what the comment essentially is. "It's dumber than me. I'm smarter than it, I don't need to be worried about it". Yes this is Exactly What I was saying in the post you replied to. You and I are in agreement.
  - RHSman2 4y ago
    I’m dumber than you
- Jensson 4y ago
  A person who does understand how new technology works will see it as a dumb contraption. A person who doesn't understand how it works will see it as magic.
  - famouswaffles 4y ago
    Lol no. That's just what you tell yourself to feel smart. They're are many who understand that don't think of it as dumb. "I think GPT-3 is artificial general intelligence, AGI. I think GPT-3 is as intelligent as a human. And I think that it is probably more intelligent than a human in a restricted way… in many ways it is more purely intelligent than humans are. I think humans are approximating what GPT-3 is doing, not vice versa.” — Connor Leahy, co-founder of EleutherAI, creator of GPT-J (November 2020)
    - Jensson 4y ago
      A founder trying to sell a product by telling you the product is magic? Yeah, very trustworthy source there.

eternalban 4y ago

> none of the words I'm saying are new, and many of the word combinations I'm using have been used time and time again. The ideas I'm expressing have merit -- but are they wholly original? Then why are you saying it? But besides that query, your "word combination" bit elicited a 'so what?' > We're all just remixing ideas we've heard before. Something wrong with the grammar in that sentence. (Or do you go by we/us?)

jmcgeeney 4y ago

The difference between humans and sophisticated stochastic parrots is reason. The most common kinds of mistakes that chatGPT currently makes are when it says things that are not simply wrong, but don't make sense. Perhaps it will be possible to emulate reason with enough data, training, parameters, etc, but without some representation of the ability to understand what you don't know, what you know, and what follows from those things consistently, I wonder if these kinds of models will ever become truly reliable.

ldhough 4y ago
Although even in this example it does say some things that are simply wrong: "Some programming languages like TypeScript and Swift use a colon for both variable type annotations and function return types." This is incorrect for Swift, which uses the same arrows as "TenetLang" for function return types. Actually the first thing I thought looking at the example code was "looks swifty but not quite as well designed."
galaxyLogic 4y ago
> some representation of the ability to understand what you don't know, what you know, and what follows from those things consistently Right, that is truly what the chatbots are lacking. They can fool some of the people some of the time, but they can't fool all of the people all of the time. Their creators - not the chatbots themselves - are trying to "fool" people into believing that the chatbots would have as you say "ability to understand what you don't know, what you know, and what follows from those things consistently".
HervalFreire 4y ago
Except the fact that it does both. It commits mistakes AND it also comes up with remarkable, novel output that makes sense. Perhaps you just haven't seen an example of it that's convincing enough. https://www.engraved.blog/building-a-virtual-machine-inside/ https://www.engraved.blog/building-a-virtual-machine-inside/ Here's another one.
raincole 4y ago
It's totally possible that humans don't do reason. It's possible that the parrot in our brain makes the decision, and then the "frontend" of our mind makes fake reasons that sound logical enough. But it's just a possibility, and I don't find it's particularly convincing.

testrun 4y ago

Nope, we parrot mostly, but not always. When we do not, we create something new, and then we started parroting again. Mostly. The question is, can ai do something new, understand context of the new thing, understand when and where it will be usable and when it make sense to apply it.

marviel 4y ago
When we're creating something "new" -- are you positing that it isn't built on top of other ideas, being combined in new ways?
- testrun 4y ago
  New can be 1. something without a predecessor, 2. synthesis of other ideas that creates totally new application or theory
  - marviel 4y ago
    Can you provide an example of 1? I think there are definitely some out there that feel almost "singular" in the way you describe, but each time I try to find a non-predecessor item I end up feeling "nope there are still dependencies"
    - testrun 4y ago
      Relativity for instance. New does not exclude dependencies or partial predecessors. New is something that is totally different from before.
      - vidarh 4y ago
        Relativity comes from exploring the problem of the incompatibilities of pre-existing theories and experiments. I don't see anything that suggests it requires anything more than synthesis and a stochastic process to explore the gaps and inconsistencies. In terms of timing, they're the culmination of decades of pre-existing work, with e.g. the Lorentz transformations named the same year Einstein published his paper on special relativity.

galaxyLogic 4y ago

> The ideas I'm expressing have merit -- but are they wholly original? I think the difference is that your intention is to write sentences which you think and believe are correct and which can help other people understand the subject. Chatbot has no intentions of its own, it only imitates texts it has read from the internet. As far as it is concerned they might be totally wrong. You on the other hand have the ability -- and the desire -- to reason about what you are saying and think whether it is actually true or not.

marviel 4y ago
I'm not sure I can support the idea that the agent doesn't have the ability to reason -- try to give it any complex (text-based) puzzle you can think of and it'll do just about as well as an average person, oftentimes much better. And I think you may be wrong to say the chatbot doesn't have intentions -- it's intentions based on its training are to accurately predict the next character. It doesn't care in the same way we do, sure -- but you could make a case (and it will be made in courts in the next few years, I don't doubt) that these agents do have desires that are analogous to our own, by the nature of their training process. I don't know where that leaves us to be honest, but it's an interesting topic to discuss.
- galaxyLogic 4y ago
  You say > it's intentions based on its training are to accurately predict the next character. then you say: > these agents do have desires that are analogous to our own I think its very in-human to have a single desire which is to predict the next character. That is not analogous to our desires. And the intention to predict the next character is not the intention of the chatbot, it is the intention of whoever created the chatbot or whoever is using it for that purpose. AI is a tool created by humans to fulfill the intentions of those humans. Is it the intention of a gun to kill people? No, it is the intention of whichever human who uses a gun for such a purpose. Is it the intention of AI to predict the next character? No that is the intention of the human who uses AI for such a purpose.
  - 4y ago

okaleniuk 4y ago

I would go even further. People are not even that advanced stochastic parrots. In the similar task, we failed just as spectacularly as the GPT-4 did right now. Rust, Zig, Julia, Kotlin, and dozens smaller "new" languages fail to bear a smear of originality in them. Their authors remixed old ideas in different proportions and got different grammars as a result. That's something an LLM can do and much faster too. But there is also Unison. That one new language that stands out, and the very fact that it does makes it improbable for an LLM to generate. LM is a language model, it avoids marginality and originality altogether by its design. Language models can and are useful if you specifically want to avoid marginality. To reduce noise, to remove errors. There is huge potential in them. If the problem was to design the most average programming language with no purpose, no market niche, and no technological context - then GPT-4 is clearly a winner.

visarga 4y ago
Small LMs do tend to avoid originality, but large LMs can use the context as a starting point to walk outside their training data distribution. They can do that by being better at composing concepts and using context-conditioning, which is also known as in-context learning.
laserbeam 4y ago
> Rust, Zig, Julia, Kotlin, and dozens smaller "new" languages fail to bear a smear of originality in them. To be fair, most programming languages don't try to be original. They try to solve problems. They also try to keep syntax close enough to other established programming treds so people have an easier time learning them.
marviel 4y ago
Thanks for pointing out Unison -- I'm definitely going to check it out. There is another language in this space I've had my eye on as well, but I can't recall the name at the moment.
wruza 4y ago
I would go even further. People are not even that advanced stochastic parrots. In the similar task, we failed just as spectacularly as the GPT-4 did right now. Rust, Zig, Julia, Kotlin, and dozens smaller "new" languages fail to bear a smear of originality in them. Their authors remixed old ideas in different proportions and got different grammars as a result. It wasn’t an originality contest, so there’s at least one non-sequitur in that statement. Everyone can create an original syntax, given modern character sets and formatting abilities. The key issue with that is the amount of developers who want to learn and switch between these is low.

travisgriggs 4y ago

I think the difference is that LLMs are essentially just stochastic parrots (SP) Humans are stochastic parrots and more. In some ways the SP behavior of LLMs already far outstrips the SP abilities of humans. And there are some leaps that humans' SP ablities will be hard to mimic. But at the end of the day, a human is still more than just an LLM. What exactly, we'll be debating that til the end of time likely. We know that an LLM can probably give us a pretty condensed summary of mankinds philosophical wranglings on the subject. :D

dirkt 4y ago

> But my argument is that humans are also, at some level, sufficiently advanced stochastic parrots. But humans have a feedback loop called "consciousness" that sits above the stochastic parrot level, can reason about it, and can separate the wheat from the chaff. AI still has to reach this level. ChatGPT cannot judge if its answers are right or wrong. It just claims things confidently, and as long as it is parroting correctly, it is right, and that's why humans get fooled into thinking that maybe they can trust it. (That's also why humans get fooled by human bullshitters who claim things confidently). So if you don't use ChatGPT as what it is, a lookup-tool trained on the content of the internet, which can't even tell you where it got the information from, and the onus is on you to verify if the information is correct or not, and if instead try to use it as some kind of expert you can trust (which I have seen quite a few people do), then you are doing it wrong.

marviel 4y ago
I don't disagree we're still better in a number of ways, but if GPT-4 was given a database, and a "while" loop -- do you think it would gain consciousness?

chimprich 4y ago

> We're all just remixing ideas we've heard before. We're largely remixing ideas we've heard before, but not just I think. If you're just remixing ideas that came before... where did those ideas come from? I think you can get quite far remixing ideas but for more novel concepts to emerge you have to actually create new stuff. In biology it seems that evolution has found it worthwhile not to minimise mutations too much. In genetic programming you need both mutation and crossover for evolution. I don't think LLMs really have that mutation of ideas (at the moment!), and I suspect humans create genuinely new concepts in a much more sophisticated way than mere random mutation.