4 ms·

This is horrifying and whenever someone (in this thread and many others) exclaim how this is "cool" and "exciting" I picture a 13 year old boy out with his mate

by tedeh 6y ago

This is horrifying and whenever someone (in this thread and many others) exclaim how this is "cool" and "exciting" I picture a 13 year old boy out with his mate in the woods saying that after firing three 9mm rounds into a tree just after stealing his fathers gun. That is not to disparage these posters, this is quite obviously in a naive sense a "cool" piece of technology but the ramifications in todays already extremely polarized society of seeing this technology end up in the hands of "the wrong people" and used for all kinds of nefarious purposes makes me feel quite uneasy. Now we can't trust text, the most trusted medium in human history, and then what? Someone else in this thread doubted that we would see AI within their lifetime, but now think there is a 50/50 shot it will happen in the next decade due to GPT-3. I wasn't expecting to see the disintegration of society during my lifetime, and while I don't think there is a 50 % chance of THAT happening in the next decade, these days that just feels much more likely than before. OpenAI should stop whatever they are doing and create a searchable repository of every piece text generated by this model so that we have a quick way of checking automatically provenance at the very least.

Trasmatta 6y ago
> Now we can't trust text, the most trusted medium in human history, and then what? I don't understand what you mean by this. Isn't text the least trusted medium? Anyone can easily try to impersonate anyone else in text. Going by your argument of trust, shouldn't we be more afraid of deepfake videos? Or even more accurately, deepfake videos paired with something like GPT-3.
- atq2119 6y ago
  Yes, we should also be afraid of deepfake. With GPT-3, the issue isn't so much being able to impersonate somebody, but the ability to generate human-seeming text at scale. This allows you to create a false perception of public sentiment by maintaining fake accounts on online forums, writing fake letters to the editor, to politicians, and so on. On the flip side, this is already possible today with content farms, and perhaps GPT-3 can save us by being the thing that finally erodes our trust in these signals. One thing is sure though, it drives yet another arms race, and arms races are always a net loss for society.
ipnon 6y ago
We must return, regretably, to judging the quality of a text on the content within itself, instead of by appeal to the authority who wrote it. This will all change if we can agree GPT-{3,4,...,n} is the greatest author on this planet. To be honest I find it hard to believe that won't be the case within this century.
- kortilla 6y ago
  Right, when judging an individual piece of text you should evaluate the merits of the argument. Where this becomes dangerous though is in building what looks like consensus among thousands of users of a service. Text like this makes it feasible to build huge puppet networks that can get upvotes/karma from real users quite easy by playing into their echo chambers. This type of technology could easily be the undoing of anonymous upvote driven forums like reddit/hn.
visarga 6y ago
If society put a lid on OpenAI do you think others are going to stop doing language model research? Luddite responses don't fare well with new tech. Society will adapt, there is already a lot of garbage text online, and we'll find uses for LMs.
zerocrates 6y ago
What I've never quite gotten is, what's really the risk people are seeing? GPT-2 specifically I remember a great deal of handwringing (or hype) about how dangerous it was. I feel like I even asked this same question here earlier: What's the danger? I hear about "polarization" and so on, but what's this supposed to enable that the bots and trolls and just good old regular people of today don't? Is it just a matter of scale? A widespread ability to pretty convincingly fake more difficult things like photos and video seems much more relevant or "disruptive" than anything in the realm of text generation. I just don't really know what "fake" text does at the end of the day.
- jtolmar 6y ago
  Software engineers naturally focus on what's a new capability for software. GPT-3 looks nearly capable of shoveling out the sort of low-quality content that's primarily curated by other algorithms, like clickbait news or content marketing, which was still firmly beyond software capabilities last year. Software engineers also frequently underestimate what can already be done with a heap of poorly paid workers. So while it's a new capability for software, it's not a new capability for business. You're right that this can already be done with people.
  - killerstorm 6y ago
    Have you ever hired human text writers? I have. And I can tell you even with not-so-poorly-paid people you often get craptacular results. Writing is really damn hard, especially writing something which require domain knowledge. From examples I've seen, it looks like GPT-3 works a lot better than your average human writer, for a fraction of cost, for a fraction of time. A lot of low-quality content you see on the web is more of a copy-paste job than actual writing. Arguable, GPT-3 is also doing a transformative copy-paste, but it remixes content of thousands of sources in a coherent way. Most people can't do that.
- samvher 6y ago
  I rely a lot on text for obtaining information and shaping my opinion, and in many cases short form text plays an important role (e.g. here or on reddit). I’m sure I’m not alone in that. This technology can at the very least waste my time, confuse me and hide the content that I’m actually looking for. It looks like it can feasibly generate 2-3 sentence comments that make sense in context, but in an automated way, with the purpose of injecting a specific sentiment into a comment section. I already didn’t like that sometimes it seems comments I think are written by humans might not be (or they might not be sincere). This kind of technology can make that problem a lot larger. It could flood the internet with so much crap, that is so hard to filter out, that the internet becomes a much less usable source for obtaining reliable information. I think that’s pretty scary.
killerstorm 6y ago
The cat is out of bag already -- people know what one can accomplish if you scale up a GPT-2-like model. OpenAI has no control over it -- anyone with enough hardware and a bit of knowledge can make his own GPT-3, perhaps specialized for his needs. The scariest thing is that GPT-3 is completely unoptimized. If you want to generate indistinguishable articles in a particular field, you can o it with fine-tuning, or, perhaps, adding GAN on top of it.
losvedir 6y ago
Nah, I think this might look worse than it is to "always online" types. And sure, it might be the death knell for Twitter and reddit. But it fortunately has zero impact on face to face interactions. I guess the 90s, before social media, before going to the internet for entertainment, before Yelp and reddit for buying advice, were before many commenters here were aware, but they weren't so bad. A reversion to that time period, but with today's ability to video chat with friends and family and the compute power available for, say, drug discovery, doesn't sound so bad to me. In fact, I'm struggling to think how GPT-3 affects me at all, given my lifestyle.

by tedeh 6y ago

Trasmatta 6y ago

> Now we can't trust text, the most trusted medium in human history, and then what? I don't understand what you mean by this. Isn't text the least trusted medium? Anyone can easily try to impersonate anyone else in text. Going by your argument of trust, shouldn't we be more afraid of deepfake videos? Or even more accurately, deepfake videos paired with something like GPT-3.

atq2119 6y ago
Yes, we should also be afraid of deepfake. With GPT-3, the issue isn't so much being able to impersonate somebody, but the ability to generate human-seeming text at scale. This allows you to create a false perception of public sentiment by maintaining fake accounts on online forums, writing fake letters to the editor, to politicians, and so on. On the flip side, this is already possible today with content farms, and perhaps GPT-3 can save us by being the thing that finally erodes our trust in these signals. One thing is sure though, it drives yet another arms race, and arms races are always a net loss for society.

ipnon 6y ago

We must return, regretably, to judging the quality of a text on the content within itself, instead of by appeal to the authority who wrote it. This will all change if we can agree GPT-{3,4,...,n} is the greatest author on this planet. To be honest I find it hard to believe that won't be the case within this century.

kortilla 6y ago
Right, when judging an individual piece of text you should evaluate the merits of the argument. Where this becomes dangerous though is in building what looks like consensus among thousands of users of a service. Text like this makes it feasible to build huge puppet networks that can get upvotes/karma from real users quite easy by playing into their echo chambers. This type of technology could easily be the undoing of anonymous upvote driven forums like reddit/hn.

visarga 6y ago

If society put a lid on OpenAI do you think others are going to stop doing language model research? Luddite responses don't fare well with new tech. Society will adapt, there is already a lot of garbage text online, and we'll find uses for LMs.

zerocrates 6y ago

What I've never quite gotten is, what's really the risk people are seeing? GPT-2 specifically I remember a great deal of handwringing (or hype) about how dangerous it was. I feel like I even asked this same question here earlier: What's the danger? I hear about "polarization" and so on, but what's this supposed to enable that the bots and trolls and just good old regular people of today don't? Is it just a matter of scale? A widespread ability to pretty convincingly fake more difficult things like photos and video seems much more relevant or "disruptive" than anything in the realm of text generation. I just don't really know what "fake" text does at the end of the day.

jtolmar 6y ago
Software engineers naturally focus on what's a new capability for software. GPT-3 looks nearly capable of shoveling out the sort of low-quality content that's primarily curated by other algorithms, like clickbait news or content marketing, which was still firmly beyond software capabilities last year. Software engineers also frequently underestimate what can already be done with a heap of poorly paid workers. So while it's a new capability for software, it's not a new capability for business. You're right that this can already be done with people.
- killerstorm 6y ago
  Have you ever hired human text writers? I have. And I can tell you even with not-so-poorly-paid people you often get craptacular results. Writing is really damn hard, especially writing something which require domain knowledge. From examples I've seen, it looks like GPT-3 works a lot better than your average human writer, for a fraction of cost, for a fraction of time. A lot of low-quality content you see on the web is more of a copy-paste job than actual writing. Arguable, GPT-3 is also doing a transformative copy-paste, but it remixes content of thousands of sources in a coherent way. Most people can't do that.
samvher 6y ago
I rely a lot on text for obtaining information and shaping my opinion, and in many cases short form text plays an important role (e.g. here or on reddit). I’m sure I’m not alone in that. This technology can at the very least waste my time, confuse me and hide the content that I’m actually looking for. It looks like it can feasibly generate 2-3 sentence comments that make sense in context, but in an automated way, with the purpose of injecting a specific sentiment into a comment section. I already didn’t like that sometimes it seems comments I think are written by humans might not be (or they might not be sincere). This kind of technology can make that problem a lot larger. It could flood the internet with so much crap, that is so hard to filter out, that the internet becomes a much less usable source for obtaining reliable information. I think that’s pretty scary.

killerstorm 6y ago

The cat is out of bag already -- people know what one can accomplish if you scale up a GPT-2-like model. OpenAI has no control over it -- anyone with enough hardware and a bit of knowledge can make his own GPT-3, perhaps specialized for his needs. The scariest thing is that GPT-3 is completely unoptimized. If you want to generate indistinguishable articles in a particular field, you can o it with fine-tuning, or, perhaps, adding GAN on top of it.

losvedir 6y ago

Nah, I think this might look worse than it is to "always online" types. And sure, it might be the death knell for Twitter and reddit. But it fortunately has zero impact on face to face interactions. I guess the 90s, before social media, before going to the internet for entertainment, before Yelp and reddit for buying advice, were before many commenters here were aware, but they weren't so bad. A reversion to that time period, but with today's ability to video chat with friends and family and the compute power available for, say, drug discovery, doesn't sound so bad to me. In fact, I'm struggling to think how GPT-3 affects me at all, given my lifestyle.