4 ms·

I thought it was well known that GPT-3 is pretty good at producing incoherent bullshit. No surprise here. Take this for example: > At the party, I poured myse

I thought it was well known that GPT-3 is pretty good at producing incoherent bullshit. No surprise here. Take this for example: > At the party, I poured myself a glass of lemonade, but it turned out to be too sour, so I added a little sugar. I didn’t see a spoon handy, so I stirred it with a cigarette. But that turned out to be a bad idea because it kept falling on the floor. That’s when he decided to start the Cremation Association of North America, which has become a major cremation provider with 145 locations. What?

shadowmore 6y ago
Sounds like the way dreams are usually structured. Seemingly coherent overarching structure with completely arbitrary zig-zags at the micro level.
ohgodplsno 6y ago
I mean, switch a few words for more tech related ones, and you've got the pitch of quite a few YCombinator candidates.
skatesor 6y ago
GPT doesn't have an 'understanding' class or a 'reasoning' function or whatever. It's a really well put together piece of statistics and sentences like these show it doesn't really have a concept of 'making sense'. You can use your much more advanced human brain to visibly see where it put in random variables (cigarette) and where it borrowed pieces of sentences (but it turned out to be too sour). You can see it made no connection between those two things that wasn't based on pure probability, and got it wrong anyway. I'm not trying to be reductive, i like the model, it's just good to know the limitations of the tools you are using and to remember that it's not an independent thinker.
- hackinthebochs 6y ago
  >It's a really well put together piece of statistics But why think "statistics" precludes it from having genuine understanding to some degree. After all, there is a statistical description the human brain but that doesn't seem to preclude understanding. I keep asking this whenever I see dismissive responses of this sort, and I never get a reply.
  - ssivark 6y ago
    Statistics doesn’t preclude understanding, but statistics are definitely not enough. For example, uncertainties/probabilities/statistics is original to whether the model incorporates causal/reasoning structure. Any tractable amount of data with the former can’t approximate an ounce of the latter. All breakages will be attributed to “distribution shifts” of the underlying statistical distribution, or other pretty words we can come up with... but that basically makes purely statistical approaches “stupid”.
    - hackinthebochs 6y ago
      >Any tractable amount of data with the former [statistics] can’t approximate an ounce of the latter [causal/reasoning structure]. I don't know why you think this is true. If statistically B follows A to a high degree, then a sufficiently advanced statistical model will represent "A then B" in some manner. In a predictive language model, at some point the best way to model a text corpus that indirectly references the "A then B" causal structure is to just model that structure and reference it as needed.
      - ClumsyPilot 6y ago
        Because if you have a working concept of time, space, and modes of transport, you are aware that a a person has been driving for 2 hours, you can easily deduce the handful of possible towns they might arrive at. Indeed we have software that does that. The statistical model will die to combinatorial explosion between billions of possible combinations of locations, times, and modes of transport. In various literature in 2 hours you might have travelled across town, across continents, or to the moon. Statistical approach to such problems is dumb.
        hackinthebochs 6y ago
        But this isn't pointing to a fundamental limitation of statistical models, only a limitation of the text corpus. If you had a billion pages of text written about some town and the text included descriptions of travel distances and locations, the model should eventually develop a good representation of the town and relative locations. But of course without such a seed of spatial information, it will just make up plausible data. A human would behave similarly when forced to write a story while lacking critical information. >Statistical approach to such problems is dumb. Well, expecting your model to extract a spatial representation of the world from text is a dumb approach indeed. We interact with the spatial information much more directly. But our ability to navigate is fundamentally just a process of capturing regularities in our sensory input.
- nullc 6y ago
  I think you're drawing an inappropriate conclusion. Imagine someone offered you a million dollars per contiguous accurate word you produce in a continuation of some text they randomly drew from the internet. You were allowed to have a team of experts help you and have a month to construct your best response (but not to go look up the text itself). If the text they gave you was some weird non-sequitur thing are you going to make your continuation an earnest reply? No. People don't usually write that way. Instead, you might decide that the source material was a children's story or a joke site and continue in kind. You want the millions of dollars, after all. So when prompted with some "natural reasoning" question without any other context-- involving a farcical situation with a bathing suit and a court appearance of course you are going to continue it with a story about wearing a bathing suit to court. Because, duh. This is the task GPT3 is accomplishing. But instead of merely being incentivized by millions of dollars, GPT3 was built from the ground up to exist for only this purpose and it does it reasonably well. Would we assume that you didn't understand the world, didn't have a concept of 'making sense', or were only capable of madlibs because when asked to accurately continue text from the internet and prompted with a bit of narrative about a bathing suit and a court you recognized the genre? No, we would not. Quite the opposite. As far as I can tell GPT3 understands the world better than some MIT Technology Review writers and HN posters.
- dTal 6y ago
  The cigarette and the too-sour part were all part of the prompt and were written by a human. I guess somebody had better tell them the bad news...

shadowmore 6y ago

Sounds like the way dreams are usually structured. Seemingly coherent overarching structure with completely arbitrary zig-zags at the micro level.

ohgodplsno 6y ago

I mean, switch a few words for more tech related ones, and you've got the pitch of quite a few YCombinator candidates.

skatesor 6y ago

GPT doesn't have an 'understanding' class or a 'reasoning' function or whatever. It's a really well put together piece of statistics and sentences like these show it doesn't really have a concept of 'making sense'. You can use your much more advanced human brain to visibly see where it put in random variables (cigarette) and where it borrowed pieces of sentences (but it turned out to be too sour). You can see it made no connection between those two things that wasn't based on pure probability, and got it wrong anyway. I'm not trying to be reductive, i like the model, it's just good to know the limitations of the tools you are using and to remember that it's not an independent thinker.

hackinthebochs 6y ago
>It's a really well put together piece of statistics But why think "statistics" precludes it from having genuine understanding to some degree. After all, there is a statistical description the human brain but that doesn't seem to preclude understanding. I keep asking this whenever I see dismissive responses of this sort, and I never get a reply.
- ssivark 6y ago
  Statistics doesn’t preclude understanding, but statistics are definitely not enough. For example, uncertainties/probabilities/statistics is original to whether the model incorporates causal/reasoning structure. Any tractable amount of data with the former can’t approximate an ounce of the latter. All breakages will be attributed to “distribution shifts” of the underlying statistical distribution, or other pretty words we can come up with... but that basically makes purely statistical approaches “stupid”.
  - hackinthebochs 6y ago
    >Any tractable amount of data with the former [statistics] can’t approximate an ounce of the latter [causal/reasoning structure]. I don't know why you think this is true. If statistically B follows A to a high degree, then a sufficiently advanced statistical model will represent "A then B" in some manner. In a predictive language model, at some point the best way to model a text corpus that indirectly references the "A then B" causal structure is to just model that structure and reference it as needed.
    - ClumsyPilot 6y ago
      Because if you have a working concept of time, space, and modes of transport, you are aware that a a person has been driving for 2 hours, you can easily deduce the handful of possible towns they might arrive at. Indeed we have software that does that. The statistical model will die to combinatorial explosion between billions of possible combinations of locations, times, and modes of transport. In various literature in 2 hours you might have travelled across town, across continents, or to the moon. Statistical approach to such problems is dumb.
      - hackinthebochs 6y ago
        But this isn't pointing to a fundamental limitation of statistical models, only a limitation of the text corpus. If you had a billion pages of text written about some town and the text included descriptions of travel distances and locations, the model should eventually develop a good representation of the town and relative locations. But of course without such a seed of spatial information, it will just make up plausible data. A human would behave similarly when forced to write a story while lacking critical information. >Statistical approach to such problems is dumb. Well, expecting your model to extract a spatial representation of the world from text is a dumb approach indeed. We interact with the spatial information much more directly. But our ability to navigate is fundamentally just a process of capturing regularities in our sensory input.
nullc 6y ago
I think you're drawing an inappropriate conclusion. Imagine someone offered you a million dollars per contiguous accurate word you produce in a continuation of some text they randomly drew from the internet. You were allowed to have a team of experts help you and have a month to construct your best response (but not to go look up the text itself). If the text they gave you was some weird non-sequitur thing are you going to make your continuation an earnest reply? No. People don't usually write that way. Instead, you might decide that the source material was a children's story or a joke site and continue in kind. You want the millions of dollars, after all. So when prompted with some "natural reasoning" question without any other context-- involving a farcical situation with a bathing suit and a court appearance of course you are going to continue it with a story about wearing a bathing suit to court. Because, duh. This is the task GPT3 is accomplishing. But instead of merely being incentivized by millions of dollars, GPT3 was built from the ground up to exist for only this purpose and it does it reasonably well. Would we assume that you didn't understand the world, didn't have a concept of 'making sense', or were only capable of madlibs because when asked to accurately continue text from the internet and prompted with a bit of narrative about a bathing suit and a court you recognized the genre? No, we would not. Quite the opposite. As far as I can tell GPT3 understands the world better than some MIT Technology Review writers and HN posters.
dTal 6y ago
The cigarette and the too-sour part were all part of the prompt and were written by a human. I guess somebody had better tell them the bad news...