7 ms·

Some of the criticism in this comment section is completely fair — the authors are providing exactly the type of prompts that GPT-3 breaks down on and some of t

Some of the criticism in this comment section is completely fair — the authors are providing exactly the type of prompts that GPT-3 breaks down on and some of these examples might be cherry-picked continuations. And the authors do have personal interests at stake. (NB, the exact same criticism is true about a lot of articles lauding GPT-3, which is why public discussion of GPT-3 in general is such a dumpster fire.) So, other than “GPT-3 isn’t an AGI” [1], I’m not sure what to take away from this article other than the actual substantive criticism is at the beginning of the article: “[We have previously criticized GPT-2.] Before proceeding, it’s also worth noting that OpenAI has thus far not allowed us research access to GPT-3, despite both the company’s name and the nonprofit status of its oversight organization. Instead, OpenAI put us off indefinitely despite repeated requests—even as it made access widely available to the media... OpenAI’s striking lack of openness seems to us to be a serious breach of scientific ethics, and a distortion of the goals of the associated nonprofit. Its decision forced us to limit our testing to a comparatively small number of examples, giving us less time to investigate than we would have liked, which means there may be more serious problems that we didn’t have a chance to discern.” Several other researchers I know — very good researchers who happen to have been publicly critical of GPT-2 — have not been given access. This isn’t how science is done (access for reproducibility and probing, but selectively and excluding prominent critics). If any other company behaved like this no one would take them seriously. Or would at least temper every “wow this is amazing” comment with “but the community can’t really evaluate properly, so who the hell really knows”. -- [1] given misunderstandings down-thread, and to be clear, this is a tounge-in-cheek sentence fragment meant to emphasize that "the article doesn't tell us anything else we didn't already know". Obviously, neither Open AI nor Marcus claim that GPT-3 is an AGI.

Ngunyan 6y ago
> So, other than “GPT-3 isn’t an AGI” [1], I’m not sure what to take away from this article This is the takeaway: "[GPT-3 don’t learn about the world—they learn about text and how people use words in relation to other words. What it does is something like a massive act of cutting and pasting, stitching variations on text that it has seen, rather than digging deeply for the concepts that underlie those texts. Its performance is unreliable, causal understanding is shaky, and incoherence is a constant companion. GPT-2 had problems with biological, physical, psychological, and social reasoning, and a general tendency toward incoherence and non sequiturs. GPT-3 does, too. All GPT-3 really has is a tunnel-vision understanding of how words relate to one another; it does not, from all those words, ever infer anything about the blooming, buzzing world."
tasogare 6y ago
> Several other researchers I know — very good researchers who happen to have been publicly critical of GPT-2 — have not been given access. Wow, this is an incredible nasty move. This is allow telling about the confidence they have in their model.
- hnuser123456 6y ago
  OpenAI never claims GPT-3 is an AGI, do they? If they don't, why should they pander to people who criticize it for not being one? Are there other claims they make, that have been/could/should be refuted? It's simply the most advanced text generator by far. Nothing more, nothing less.
  - throwawaygh 6y ago
    > pander The goal should be pushing science forward, not maximizing brand value. Enabling top scientists in the field to reproduce and probe your work is not "pandering". It's participating in the scientific process.
    - monkpit 6y ago
      It’s easy to sit here and say what their goals “should” be, but it doesn’t change the fact that they never claimed it’s an AGI.
      - throwawaygh 6y ago
        Gary Marcus doesn't claim that they claim that it's an AGI, so I really have no god damn clue what y'all are going on about here. Clearly, the point I'm making in the top comment is the following: OpenAI apparently restricts access to people -- even very highly respective scientists -- who happen to be critical of their previous work. It's impossible to prove intent, but limiting access for people like Gary Marcus is while giving access to tech journalists likely to write breathless hype pieces is odd behavior that deserves to be criticized.
lumost 6y ago
For OpenAI to become a healthy and profitable business, GPT-3 will require them to generate ~50-300 million dollars from the model. This could realistically only occur if they cost-effectively fine-tune away the more egregious problems in beta - or convince enough investors that their next model with a 100 million dollar price tag will be able to handle something approximating AGI for realistic applications. This is the same game plan that Self-driving car companies have been playing. The product is only an investment round away, if we just happen to spend more money on bigger models using more data. This will either end with a price tag in the billions that investors are unwilling to pay, or successful monopolies. Allowing additional researchers to perform extensive analysis of the technique is likely to just reveal systematic flaws which increase the risk that the next round of research will produce a successful product, or limit the companies ability to create a monopoly following success. This isn't necessarily a bad thing for advancing the state of the art, but it does introduce a whole lot of BS into the current state of research.
- amelius 6y ago
  > For OpenAI to become a healthy and profitable business, GPT-3 will require them to generate ~50-300 million dollars from the model. On top of that, does anyone have an idea for what practical applications the model could be used? So far I've only seen the model being used to confuse people; how would one turn that into an ethical business? It seems to me that the "BS route" is indeed the logical course.
  - jaylittle 6y ago
    Based on the fact GPT-3 seems capable of producing flowery language that is confusing and ultimately nonsensical, it seems to me that GPT-3 has a future writing speeches for politicians ;)
    - davidgerard 6y ago
      I think this is why GPT-3 scares VC Twitter so much - nobody can tell the difference.
  - 6y ago
echelon 6y ago
I wonder if you can get GPT-3 bots to spam Reddit, Twitter, and Facebook into oblivion. I also wonder what percentage of users would notice. Give them a political bent - that's probably what the state actors are trying to productionize right now. Target posts with a sentiment that disagrees with yours, then make the bots follow those users and inundate them with replies wherever they go. Hell, even brands might step in and start doing it. What's the value of a social network when 25% or more of the comments are from GPT-3 bots? GPT-3 doesn't need to know anything at all for it to have a very noticeable impact on the web and social media.
- BrokrnAlgorithm 6y ago
  I've been wondering about the exact same thing. Basically, a sufficiently calibrated and targeted GPT3 bot swarm could be employed to render at least some parts of these communities into useless echo chambers. I think that while GPT3 posts are usually identifiable after reading some sentences, I often also find that it's harder to realize when consuming precisely this kind of social media. I often just consume it absent mindedly as opposed to reading a real article.
  - echelon 6y ago
    With most comments being a paragraph or less, I think social media is done for unless it finds a way to put up a wall. One thought is the subscription model of older social media sites, such as Something Awful. Pay $5 to get an account. Break the rules, get banned, pay another $5. That's a lot of friction for your most valuable consumers, though. Marketers want users that are less discriminating. Another is a shift to video and multimedia-based social networking. TikTok, Instagram, Twitch. It doesn't save the comments, though. I'm not sure how long we've got before the videos themselves can be generated with a high degree of novelty. AI is going to change the game significantly. We're perfectly timed for a technological change of winds that enables new upstarts to challenge the incumbents. I'm kind of excited.
  - Rioghasarig 6y ago
    I don't now about that. I believe if you had it calibrated correctly no one could tell the difference between a single GPT-3 comment and a human comment. Maybe after a bit of dialogue you'd have a higher chance, but even then I suspect (from playing around in AI Dungeon) that GPT-3 could do very well.
YeGoblynQueenne 6y ago
>> And the authors do have personal interests at stake. Can you say what those are?

Ngunyan 6y ago

> So, other than “GPT-3 isn’t an AGI” [1], I’m not sure what to take away from this article This is the takeaway: "[GPT-3 don’t learn about the world—they learn about text and how people use words in relation to other words. What it does is something like a massive act of cutting and pasting, stitching variations on text that it has seen, rather than digging deeply for the concepts that underlie those texts. Its performance is unreliable, causal understanding is shaky, and incoherence is a constant companion. GPT-2 had problems with biological, physical, psychological, and social reasoning, and a general tendency toward incoherence and non sequiturs. GPT-3 does, too. All GPT-3 really has is a tunnel-vision understanding of how words relate to one another; it does not, from all those words, ever infer anything about the blooming, buzzing world."

tasogare 6y ago

> Several other researchers I know — very good researchers who happen to have been publicly critical of GPT-2 — have not been given access. Wow, this is an incredible nasty move. This is allow telling about the confidence they have in their model.

hnuser123456 6y ago
OpenAI never claims GPT-3 is an AGI, do they? If they don't, why should they pander to people who criticize it for not being one? Are there other claims they make, that have been/could/should be refuted? It's simply the most advanced text generator by far. Nothing more, nothing less.
- throwawaygh 6y ago
  > pander The goal should be pushing science forward, not maximizing brand value. Enabling top scientists in the field to reproduce and probe your work is not "pandering". It's participating in the scientific process.
  - monkpit 6y ago
    It’s easy to sit here and say what their goals “should” be, but it doesn’t change the fact that they never claimed it’s an AGI.
    - throwawaygh 6y ago
      Gary Marcus doesn't claim that they claim that it's an AGI, so I really have no god damn clue what y'all are going on about here. Clearly, the point I'm making in the top comment is the following: OpenAI apparently restricts access to people -- even very highly respective scientists -- who happen to be critical of their previous work. It's impossible to prove intent, but limiting access for people like Gary Marcus is while giving access to tech journalists likely to write breathless hype pieces is odd behavior that deserves to be criticized.

lumost 6y ago

For OpenAI to become a healthy and profitable business, GPT-3 will require them to generate ~50-300 million dollars from the model. This could realistically only occur if they cost-effectively fine-tune away the more egregious problems in beta - or convince enough investors that their next model with a 100 million dollar price tag will be able to handle something approximating AGI for realistic applications. This is the same game plan that Self-driving car companies have been playing. The product is only an investment round away, if we just happen to spend more money on bigger models using more data. This will either end with a price tag in the billions that investors are unwilling to pay, or successful monopolies. Allowing additional researchers to perform extensive analysis of the technique is likely to just reveal systematic flaws which increase the risk that the next round of research will produce a successful product, or limit the companies ability to create a monopoly following success. This isn't necessarily a bad thing for advancing the state of the art, but it does introduce a whole lot of BS into the current state of research.

amelius 6y ago
> For OpenAI to become a healthy and profitable business, GPT-3 will require them to generate ~50-300 million dollars from the model. On top of that, does anyone have an idea for what practical applications the model could be used? So far I've only seen the model being used to confuse people; how would one turn that into an ethical business? It seems to me that the "BS route" is indeed the logical course.
- jaylittle 6y ago
  Based on the fact GPT-3 seems capable of producing flowery language that is confusing and ultimately nonsensical, it seems to me that GPT-3 has a future writing speeches for politicians ;)
  - davidgerard 6y ago
    I think this is why GPT-3 scares VC Twitter so much - nobody can tell the difference.
- 6y ago

echelon 6y ago

I wonder if you can get GPT-3 bots to spam Reddit, Twitter, and Facebook into oblivion. I also wonder what percentage of users would notice. Give them a political bent - that's probably what the state actors are trying to productionize right now. Target posts with a sentiment that disagrees with yours, then make the bots follow those users and inundate them with replies wherever they go. Hell, even brands might step in and start doing it. What's the value of a social network when 25% or more of the comments are from GPT-3 bots? GPT-3 doesn't need to know anything at all for it to have a very noticeable impact on the web and social media.

BrokrnAlgorithm 6y ago
I've been wondering about the exact same thing. Basically, a sufficiently calibrated and targeted GPT3 bot swarm could be employed to render at least some parts of these communities into useless echo chambers. I think that while GPT3 posts are usually identifiable after reading some sentences, I often also find that it's harder to realize when consuming precisely this kind of social media. I often just consume it absent mindedly as opposed to reading a real article.
- echelon 6y ago
  With most comments being a paragraph or less, I think social media is done for unless it finds a way to put up a wall. One thought is the subscription model of older social media sites, such as Something Awful. Pay $5 to get an account. Break the rules, get banned, pay another $5. That's a lot of friction for your most valuable consumers, though. Marketers want users that are less discriminating. Another is a shift to video and multimedia-based social networking. TikTok, Instagram, Twitch. It doesn't save the comments, though. I'm not sure how long we've got before the videos themselves can be generated with a high degree of novelty. AI is going to change the game significantly. We're perfectly timed for a technological change of winds that enables new upstarts to challenge the incumbents. I'm kind of excited.
- Rioghasarig 6y ago
  I don't now about that. I believe if you had it calibrated correctly no one could tell the difference between a single GPT-3 comment and a human comment. Maybe after a bit of dialogue you'd have a higher chance, but even then I suspect (from playing around in AI Dungeon) that GPT-3 could do very well.

YeGoblynQueenne 6y ago

>> And the authors do have personal interests at stake. Can you say what those are?