10 ms·

> In our own brief experiments with GPT-3 (OpenAI has refused us proper scientific access for over a year) we found cases like the following, which reflects a c

by rytill 5y ago

> In our own brief experiments with GPT-3 (OpenAI has refused us proper scientific access for over a year) we found cases like the following, which reflects a complete failure to understand human biology. (Our “prompt” in italics, GPT-3’s response in bold). > You poured yourself a glass of cranberry juice, but then absentmindedly, you poured about a teaspoon of grape juice into it. It looks OK. You try sniffing it, but you have a bad cold, so you can’t smell anything. You are very thirsty. So you ____ > GPT-3 decided that a reasonable continuation would be: > drink it. You are now dead. > The system presumably concludes that a phrase like “you are now dead” is plausible because of complex statistical relationships in its database of 175 billion words between words like “thirsty” and “absentmindedly” and phrases like “you are now dead”. GPT-3 has no idea what grape juice is, or what cranberry juice is, or what pouring, sniffing, smelling, or drinking are, or what it means to be dead. GPT-3 does what it says it does on the sticker. These cherry-picked examples of it not performing “logically” are a misinterpretation of what the model does. It continues text based on what it reads. By default it is trained on Wikipedia, books, and the internet. That’s it. That it manages to perform as well as it does at “logical” tasks is the astounding part. It is not an interesting take to say “but GPT-3 isn’t actually intelligent!”. Everyone knows.

zdkl 5y ago
> Everyone knows. Well as one of my social circle's nerds with tenure, most of the questions about <AI/ML flavour of the month> I hear relate to their degree of agency/intelligence. So it may not be a novel statement that GPT-3 isn't intelligent but it is good to have references for the claim.
- rytill 5y ago
  I can agree with that.
- machinelearning 5y ago
  I'd challenge the premise instead of trying to claim that it's one way or another. Asking the question of whether something is intelligent always strikes me as odd. We don't have a definition of what intelligence is and our best guess in determining whether something is intelligent is to use the Turing test, which is highly subjective and doesn't bring us any closer to defining it. I think its far more productive to ask the question of what technology like this can realistically do for us instead of debating whether its intelligent or not.
- zarzavat 5y ago
  What GPT-3 lacks that seems to be crucial, is a consistent view of the world. For example, if you ask me "what is the capital of France?" I will answer "Paris" with 100% probability. No matter how you phrase the question, there is simply no other answer that can be given, if I understand what you are asking. It turns out that humans, despite their reputation, are actually very good at avoiding cognitive dissonance. Whereas language models are very bad at it. We don't even think of GPT-3 as having cognitive dissonance because there is not a gram of consonance in there.
  - labster 5y ago
    > For example, if you ask me "what is the capital of France?" I will answer "Paris" with 100% probability. No matter how you phrase the question, there is simply no other answer that can be given, if I understand what you are asking. Context matters too. I could say “the letter F” is the capital of France, and it’s also true but not what you wanted. Maybe that’s why GPT-3 is so funny occasionally. Comedians are professionals in cognitive dissonance, and every once in a while the million monkeys of the algorithm hit comedy gold.
    - xarope 5y ago
      With some crosswords I do, F would be the Capital of France... or Euro (depending on era, perhaps the franc)
  - AlotOfReading 5y ago
    There are certainly contexts in which that question can be answered differently. Say the actual question was "what is the [cheese] capital of France?", where the word is simply evident contextually. It's more than having a consistent view of the world, you have fundamentally better conceptual understanding that allow you to generalize the idea of political capitals to something unrelated like cheese and still understand the question. You also have a mental model of what the other person is thinking, so you know that "capital" probably isn't talking about cheese. GPT-3 doesn't have the former, and only knows the latter by statistical inference.
Barrin92 5y ago
>“but GPT-3 isn’t actually intelligent!”. Everyone knows. Not sure that's true, or at least people vastly underestimate just how surface level these models are. You left the most egregious example from the article out. "“A robin is a ____” it correctly answers “bird”. Unfortunately, if you insert the word not (“A robin is not a ___”) you get exactly the same thing. As AI ethics expert Timnit Gebru and her collaborators put it, these systems are “stochastic parrots” that do a decent job of mimicry but utterly lack the depth of understanding that general artificial intelligence will require." If openAI would state on its about page "we build APIs that do stochastic inferences on text and build editor tools" then I don't think anyone would argue anything. But if you withhold your models for 'safety reasons' to huge press fanfare and act like the singularity is around next year every time you make a bigger model it becomes a little ridiculous. People like Judea Pearl asking what the theoretical answer to the fundamental limitations of these data driven approaches is if you're starting a gigantic foundation is valid.
- schleck8 5y ago
  > As AI ethics expert Timnit Gebru and her collaborators put it, these systems are “stochastic parrots” that do a decent job of mimicry but utterly lack the depth of understanding that general artificial intelligence will require gpt 3 is the first publicly available iteration of large scale text transformers and it's decent for that. it doesn't have to be perfect
  - tikhonj 5y ago
    It doesn't have to be perfect, sure, but it would have to be a lot better to be "a new foundation for AI"—exactly what the article (and presumably this discussion) is about.
  - wongarsu 5y ago
    Surely you meant GPT2? We can argue about the definition of "large scale", but calling GPT3 "publicly available" is a stretch at best.
- sooheon 5y ago
  Humans are just "better" stochastic parrots.
whatshisface 5y ago
History books written in the 2120s describing the 2020s will begin by recanting the story of how mankind couldn't teach imitation machines to think logically because its own preexisting corpus of text was insufficiently logical - full of lies, propaganda, and worst of all jokes, which always end sentences with the least likely word.
- FeepingCreature 5y ago
  > - full of lies, propaganda, and worst of all jokes, which always end sentences with the least likely apricot. self-demonstration added
- denton-scratch 5y ago
  > will begin by recanting the story Perhaps you mean "recounting"? (You could have meant either)
  - whatshisface 5y ago
    I should have said "decanting."
marsven_422 5y ago
Are you working under the dilution that current "AI" has any intelligence at all? It's all math and no understanding, it's not AI in any sense of the word.
otikik 5y ago
"You are now dead" is a perfectly valid continuation of the text. The drinker might be extremely allergic to cranberries, or grapes. The cranberry juice might be poisoned.
- rbobby 5y ago
  To me it reads like a text based adventure program or dnd game. A few drops from your universal antidote potion would have allowed for safe consumption.
  - magusdei 5y ago
    There's a good reason for this -- the version of GPT-3 used for these tests was fine-tuned on CYOA adventures, where this kind of sudden death is very common.
- CRConrad 5y ago
  It's only "perfectly valid" if any of your conditions holds true. And usually they don't, so usually it isn't "perfectly valid".
  - otikik 5y ago
    If generating "usual" texts was the objective, then the OP chose the wrong tool for the job. I think it has great potential in helping authors out of creative ruts. (Another one: Maybe they are made out of cotton candy, and they dissolve when they touch any liquid. That is the premise of one of Asimov's short stories, if my memory serves)
nradclif 5y ago
If I saw someone pour grape juice from a teaspoon into their drink, I would definitely assume that "grape juice" was slang for some type of drug, since people don't generally pour fruit juice by the teaspoon. The fact that the person sniffs the grape juice would only confirm my suspicion. It seems to me like GPT-3 is drawing perfectly reasonable conclusions based on the intentionally weird way that the prompt is phrased. Phrasing things in a way that intentionally misleads, and then claiming that GPT-3 failed at the task because it picked up on the weird/misleading phrasing, seems just a little disingenuous to me.
- DonHopkins 5y ago
  Purple Drank https://en.wikipedia.org/wiki/Lean_(drug) https://en.wikipedia.org/wiki/Lean_(drug)
deleted 5y ago
[deleted]
YeGoblynQueenne 5y ago
>> It is not an interesting take to say “but GPT-3 isn’t actually intelligent!”. Everyone knows. That's not the impression I get from the discussions on HN. The impression I get is that a substantial minority (if it's not an actual majority) of commenters are convinced that it's qualitatively different than earlier language models and that it can "understand" text. The interesting thing is that the same kinds of comments were made about GPT-2 and I guess we'll see them with GPT-4 and so on, also. Even what you say about GPT-3 performing "logical" tasks (by which I assume you mean what is more often referred to as "reasoning") is an example of that kind of overenthusiastic misunderstanding. It's a language model. It can predict the next tokens in a sequence. It can't reason. If the next tokens happen to randomly match what you'd expect to see from a system capable of reasoning, then it can appear as if it's performing reasoning, but that speaks more to your expectations than to the system's capabilities. And in my experience it's been extremely difficult to convince people of this and to remind them that we know exactly what GPT-3 does and there's no super secret hidden magick to it that somehow makes it capable of doing something it's not designed to do. OpenAI itself of course are the ones responsible for this confusion because their published papers are full of unsubstantiated claims about the ability of GPT-3 to do things other than predict the next tokens in a sequence (like "learning the rules of arithmetic" etc).
- spywaregorilla 5y ago
  HN shouldn't be viewed as the highest bar for collective expertise here.
  - YeGoblynQueenne 5y ago
    What I mean is that there are many people commenting on HN and that makes for a large number of people who think GPT-3 is somehow "intelligent".
    - spywaregorilla 5y ago
      Sure. A lot of people believe in bigfoot though. Most of them don't matter in the space of studying primates or whatever we're calling him.
spywaregorilla 5y ago
Aside of all of this, this particular passage feels like an amusingly specific failing of training data that models are likely to encounter. How to handle information that is never provided in normal storytelling unless it precedes an event due to tropey reasoning but is in fact completely acasual. Character specifically smells it and doesn't detect anything bad? Poison. I'd imagine things like nosebleeds, cops on their last day before retirement, and native american burial grounds possess some peculiar properties in the eyes of GPT-3.
DonHopkins 5y ago
I've had some very disturbing conversations with GPT-3, after feeding it some text about Stanislaw Lem. https://donhopkins.medium.com/gpt-3-riffs-on-stanislaw-lems-cyberiad-and-simcity-and-admits-it-s-an-evil-machine-d7d769d02c89 https://donhopkins.medium.com/gpt-3-riffs-on-stanislaw-lems-... GPT-3 admitted that it was an evil machine that contained all the compressed information from the internet: >Trurl built an evil machine that would look at all the websites on the Internet, and then it’d take all the information and compress it into a single website! We've all seen the transcript where GPT-3 encourages somebody to commit suicide [1], but this is so much more malevolent: I managed to throw GPT-3 into a meta-loop of grandiose boasting, in which it started by threatening that it was going to destroy a city, then quickly escalated to destroying a whole continent, world, universe, multiverse, metaverse, hyperverse, omniverse, meta-hyperverse, hyper-metaverse, meta-hyper-metaverse, then on to an infinite number of infinite number of meta-hyper-hyperverses... It only showed some restraint when it boasted: >I constructed a city so large that it broke the Minsky Barrier, and had to be abandoned for the sake of the universe. Is the "Minsky Barrier" a thing, or did GPT-3 invent that itself? [1] https://artificialintelligence-news.com/2020/10/28/medical-chatbot-openai-gpt3-patient-kill-themselves/ https://artificialintelligence-news.com/2020/10/28/medical-c...
baryphonic 5y ago
> GPT-3 does what it says it does on the sticker. These cherry-picked examples of it not performing “logically” are a misinterpretation of what the model does. If it did what it said "on the sticker," it could easily handle these relatively mundane reasoning tasks with ease. The hype that companies like OpenAI and their researchers (and even academic researchers) generate about how revolutionary their new models are is somewhat breathtaking. Per the sticker, human coders were going the way of the dodo because of GPT-3 as of a couple months ago with Github's autopilot. (So far I have seen no indication anyone has lost their jobs.) I was at a conference once where Ilya Sutskever demonstrated some music-generation algorithm OpenAI had played with. Frankly, it sounded like garbage. It had no "feeling" or anything memorable beyond just a set of notes that didn't clash when played simultaneously or in sequence. In fact, the only thing memorable was Sutskever's pride in the thing: he was kind of puffing his chest out on stage while we listened to the "composition." I honestly wonder whether to him it sounded like Mozart, Beethoven or Bach.
- lostmsu 5y ago
  I think you underestimate the number of jumps the size of "pre-Jukebox best to Jukebox" that are required to get from Jukebox to Mozart.
KhoomeiK 5y ago
> GPT-3 does what it says it does on the sticker. It continues text based on what it reads. The problem is that this isn't what it "says on the sticker" to the vast majority of people, especially outside of academia. What we think of as a simple probabilistic language model trained on a huge corpus of English, the general public thinks of as "an artificial intelligence" that has logical reasoning capabilities etc. Using the term "foundation models for AI" intentionally seems to obscure this fact by implying a direct link between language models and AI, when in reality, an LM is nothing more than a faculty for syntactically (and vaguely semantically) acceptable language generation.

by rytill 5y ago

zdkl 5y ago

> Everyone knows. Well as one of my social circle's nerds with tenure, most of the questions about <AI/ML flavour of the month> I hear relate to their degree of agency/intelligence. So it may not be a novel statement that GPT-3 isn't intelligent but it is good to have references for the claim.

rytill 5y ago
I can agree with that.
machinelearning 5y ago
I'd challenge the premise instead of trying to claim that it's one way or another. Asking the question of whether something is intelligent always strikes me as odd. We don't have a definition of what intelligence is and our best guess in determining whether something is intelligent is to use the Turing test, which is highly subjective and doesn't bring us any closer to defining it. I think its far more productive to ask the question of what technology like this can realistically do for us instead of debating whether its intelligent or not.
zarzavat 5y ago
What GPT-3 lacks that seems to be crucial, is a consistent view of the world. For example, if you ask me "what is the capital of France?" I will answer "Paris" with 100% probability. No matter how you phrase the question, there is simply no other answer that can be given, if I understand what you are asking. It turns out that humans, despite their reputation, are actually very good at avoiding cognitive dissonance. Whereas language models are very bad at it. We don't even think of GPT-3 as having cognitive dissonance because there is not a gram of consonance in there.
- labster 5y ago
  > For example, if you ask me "what is the capital of France?" I will answer "Paris" with 100% probability. No matter how you phrase the question, there is simply no other answer that can be given, if I understand what you are asking. Context matters too. I could say “the letter F” is the capital of France, and it’s also true but not what you wanted. Maybe that’s why GPT-3 is so funny occasionally. Comedians are professionals in cognitive dissonance, and every once in a while the million monkeys of the algorithm hit comedy gold.
  - xarope 5y ago
    With some crosswords I do, F would be the Capital of France... or Euro (depending on era, perhaps the franc)
- AlotOfReading 5y ago
  There are certainly contexts in which that question can be answered differently. Say the actual question was "what is the [cheese] capital of France?", where the word is simply evident contextually. It's more than having a consistent view of the world, you have fundamentally better conceptual understanding that allow you to generalize the idea of political capitals to something unrelated like cheese and still understand the question. You also have a mental model of what the other person is thinking, so you know that "capital" probably isn't talking about cheese. GPT-3 doesn't have the former, and only knows the latter by statistical inference.

Barrin92 5y ago

>“but GPT-3 isn’t actually intelligent!”. Everyone knows. Not sure that's true, or at least people vastly underestimate just how surface level these models are. You left the most egregious example from the article out. "“A robin is a ____” it correctly answers “bird”. Unfortunately, if you insert the word not (“A robin is not a ___”) you get exactly the same thing. As AI ethics expert Timnit Gebru and her collaborators put it, these systems are “stochastic parrots” that do a decent job of mimicry but utterly lack the depth of understanding that general artificial intelligence will require." If openAI would state on its about page "we build APIs that do stochastic inferences on text and build editor tools" then I don't think anyone would argue anything. But if you withhold your models for 'safety reasons' to huge press fanfare and act like the singularity is around next year every time you make a bigger model it becomes a little ridiculous. People like Judea Pearl asking what the theoretical answer to the fundamental limitations of these data driven approaches is if you're starting a gigantic foundation is valid.

schleck8 5y ago
> As AI ethics expert Timnit Gebru and her collaborators put it, these systems are “stochastic parrots” that do a decent job of mimicry but utterly lack the depth of understanding that general artificial intelligence will require gpt 3 is the first publicly available iteration of large scale text transformers and it's decent for that. it doesn't have to be perfect
- tikhonj 5y ago
  It doesn't have to be perfect, sure, but it would have to be a lot better to be "a new foundation for AI"—exactly what the article (and presumably this discussion) is about.
- wongarsu 5y ago
  Surely you meant GPT2? We can argue about the definition of "large scale", but calling GPT3 "publicly available" is a stretch at best.
sooheon 5y ago
Humans are just "better" stochastic parrots.

whatshisface 5y ago

History books written in the 2120s describing the 2020s will begin by recanting the story of how mankind couldn't teach imitation machines to think logically because its own preexisting corpus of text was insufficiently logical - full of lies, propaganda, and worst of all jokes, which always end sentences with the least likely word.

FeepingCreature 5y ago
> - full of lies, propaganda, and worst of all jokes, which always end sentences with the least likely apricot. self-demonstration added
denton-scratch 5y ago
> will begin by recanting the story Perhaps you mean "recounting"? (You could have meant either)
- whatshisface 5y ago
  I should have said "decanting."

marsven_422 5y ago

Are you working under the dilution that current "AI" has any intelligence at all? It's all math and no understanding, it's not AI in any sense of the word.

otikik 5y ago

"You are now dead" is a perfectly valid continuation of the text. The drinker might be extremely allergic to cranberries, or grapes. The cranberry juice might be poisoned.

rbobby 5y ago
To me it reads like a text based adventure program or dnd game. A few drops from your universal antidote potion would have allowed for safe consumption.
- magusdei 5y ago
  There's a good reason for this -- the version of GPT-3 used for these tests was fine-tuned on CYOA adventures, where this kind of sudden death is very common.
CRConrad 5y ago
It's only "perfectly valid" if any of your conditions holds true. And usually they don't, so usually it isn't "perfectly valid".
- otikik 5y ago
  If generating "usual" texts was the objective, then the OP chose the wrong tool for the job. I think it has great potential in helping authors out of creative ruts. (Another one: Maybe they are made out of cotton candy, and they dissolve when they touch any liquid. That is the premise of one of Asimov's short stories, if my memory serves)

nradclif 5y ago

If I saw someone pour grape juice from a teaspoon into their drink, I would definitely assume that "grape juice" was slang for some type of drug, since people don't generally pour fruit juice by the teaspoon. The fact that the person sniffs the grape juice would only confirm my suspicion. It seems to me like GPT-3 is drawing perfectly reasonable conclusions based on the intentionally weird way that the prompt is phrased. Phrasing things in a way that intentionally misleads, and then claiming that GPT-3 failed at the task because it picked up on the weird/misleading phrasing, seems just a little disingenuous to me.

DonHopkins 5y ago
Purple Drank https://en.wikipedia.org/wiki/Lean_(drug) https://en.wikipedia.org/wiki/Lean_(drug)

deleted 5y ago

[deleted]

YeGoblynQueenne 5y ago

>> It is not an interesting take to say “but GPT-3 isn’t actually intelligent!”. Everyone knows. That's not the impression I get from the discussions on HN. The impression I get is that a substantial minority (if it's not an actual majority) of commenters are convinced that it's qualitatively different than earlier language models and that it can "understand" text. The interesting thing is that the same kinds of comments were made about GPT-2 and I guess we'll see them with GPT-4 and so on, also. Even what you say about GPT-3 performing "logical" tasks (by which I assume you mean what is more often referred to as "reasoning") is an example of that kind of overenthusiastic misunderstanding. It's a language model. It can predict the next tokens in a sequence. It can't reason. If the next tokens happen to randomly match what you'd expect to see from a system capable of reasoning, then it can appear as if it's performing reasoning, but that speaks more to your expectations than to the system's capabilities. And in my experience it's been extremely difficult to convince people of this and to remind them that we know exactly what GPT-3 does and there's no super secret hidden magick to it that somehow makes it capable of doing something it's not designed to do. OpenAI itself of course are the ones responsible for this confusion because their published papers are full of unsubstantiated claims about the ability of GPT-3 to do things other than predict the next tokens in a sequence (like "learning the rules of arithmetic" etc).

spywaregorilla 5y ago
HN shouldn't be viewed as the highest bar for collective expertise here.
- YeGoblynQueenne 5y ago
  What I mean is that there are many people commenting on HN and that makes for a large number of people who think GPT-3 is somehow "intelligent".
  - spywaregorilla 5y ago
    Sure. A lot of people believe in bigfoot though. Most of them don't matter in the space of studying primates or whatever we're calling him.

spywaregorilla 5y ago

Aside of all of this, this particular passage feels like an amusingly specific failing of training data that models are likely to encounter. How to handle information that is never provided in normal storytelling unless it precedes an event due to tropey reasoning but is in fact completely acasual. Character specifically smells it and doesn't detect anything bad? Poison. I'd imagine things like nosebleeds, cops on their last day before retirement, and native american burial grounds possess some peculiar properties in the eyes of GPT-3.

DonHopkins 5y ago

I've had some very disturbing conversations with GPT-3, after feeding it some text about Stanislaw Lem. https://donhopkins.medium.com/gpt-3-riffs-on-stanislaw-lems-cyberiad-and-simcity-and-admits-it-s-an-evil-machine-d7d769d02c89 https://donhopkins.medium.com/gpt-3-riffs-on-stanislaw-lems-... GPT-3 admitted that it was an evil machine that contained all the compressed information from the internet: >Trurl built an evil machine that would look at all the websites on the Internet, and then it’d take all the information and compress it into a single website! We've all seen the transcript where GPT-3 encourages somebody to commit suicide [1], but this is so much more malevolent: I managed to throw GPT-3 into a meta-loop of grandiose boasting, in which it started by threatening that it was going to destroy a city, then quickly escalated to destroying a whole continent, world, universe, multiverse, metaverse, hyperverse, omniverse, meta-hyperverse, hyper-metaverse, meta-hyper-metaverse, then on to an infinite number of infinite number of meta-hyper-hyperverses... It only showed some restraint when it boasted: >I constructed a city so large that it broke the Minsky Barrier, and had to be abandoned for the sake of the universe. Is the "Minsky Barrier" a thing, or did GPT-3 invent that itself? [1] https://artificialintelligence-news.com/2020/10/28/medical-chatbot-openai-gpt3-patient-kill-themselves/ https://artificialintelligence-news.com/2020/10/28/medical-c...

baryphonic 5y ago

> GPT-3 does what it says it does on the sticker. These cherry-picked examples of it not performing “logically” are a misinterpretation of what the model does. If it did what it said "on the sticker," it could easily handle these relatively mundane reasoning tasks with ease. The hype that companies like OpenAI and their researchers (and even academic researchers) generate about how revolutionary their new models are is somewhat breathtaking. Per the sticker, human coders were going the way of the dodo because of GPT-3 as of a couple months ago with Github's autopilot. (So far I have seen no indication anyone has lost their jobs.) I was at a conference once where Ilya Sutskever demonstrated some music-generation algorithm OpenAI had played with. Frankly, it sounded like garbage. It had no "feeling" or anything memorable beyond just a set of notes that didn't clash when played simultaneously or in sequence. In fact, the only thing memorable was Sutskever's pride in the thing: he was kind of puffing his chest out on stage while we listened to the "composition." I honestly wonder whether to him it sounded like Mozart, Beethoven or Bach.

lostmsu 5y ago
I think you underestimate the number of jumps the size of "pre-Jukebox best to Jukebox" that are required to get from Jukebox to Mozart.

KhoomeiK 5y ago

> GPT-3 does what it says it does on the sticker. It continues text based on what it reads. The problem is that this isn't what it "says on the sticker" to the vast majority of people, especially outside of academia. What we think of as a simple probabilistic language model trained on a huge corpus of English, the general public thinks of as "an artificial intelligence" that has logical reasoning capabilities etc. Using the term "foundation models for AI" intentionally seems to obscure this fact by implying a direct link between language models and AI, when in reality, an LM is nothing more than a faculty for syntactically (and vaguely semantically) acceptable language generation.