4 ms·

What I've never quite gotten is, what's really the risk people are seeing? GPT-2 specifically I remember a great deal of handwringing (or hype) about how danger

What I've never quite gotten is, what's really the risk people are seeing? GPT-2 specifically I remember a great deal of handwringing (or hype) about how dangerous it was. I feel like I even asked this same question here earlier: What's the danger? I hear about "polarization" and so on, but what's this supposed to enable that the bots and trolls and just good old regular people of today don't? Is it just a matter of scale? A widespread ability to pretty convincingly fake more difficult things like photos and video seems much more relevant or "disruptive" than anything in the realm of text generation. I just don't really know what "fake" text does at the end of the day.

jtolmar 6y ago
Software engineers naturally focus on what's a new capability for software. GPT-3 looks nearly capable of shoveling out the sort of low-quality content that's primarily curated by other algorithms, like clickbait news or content marketing, which was still firmly beyond software capabilities last year. Software engineers also frequently underestimate what can already be done with a heap of poorly paid workers. So while it's a new capability for software, it's not a new capability for business. You're right that this can already be done with people.
- killerstorm 6y ago
  Have you ever hired human text writers? I have. And I can tell you even with not-so-poorly-paid people you often get craptacular results. Writing is really damn hard, especially writing something which require domain knowledge. From examples I've seen, it looks like GPT-3 works a lot better than your average human writer, for a fraction of cost, for a fraction of time. A lot of low-quality content you see on the web is more of a copy-paste job than actual writing. Arguable, GPT-3 is also doing a transformative copy-paste, but it remixes content of thousands of sources in a coherent way. Most people can't do that.
samvher 6y ago
I rely a lot on text for obtaining information and shaping my opinion, and in many cases short form text plays an important role (e.g. here or on reddit). I’m sure I’m not alone in that. This technology can at the very least waste my time, confuse me and hide the content that I’m actually looking for. It looks like it can feasibly generate 2-3 sentence comments that make sense in context, but in an automated way, with the purpose of injecting a specific sentiment into a comment section. I already didn’t like that sometimes it seems comments I think are written by humans might not be (or they might not be sincere). This kind of technology can make that problem a lot larger. It could flood the internet with so much crap, that is so hard to filter out, that the internet becomes a much less usable source for obtaining reliable information. I think that’s pretty scary.
- dorgo 6y ago
  Would you consider this comment to have less/no value if you found out that it was generated by a bot? What if quality and information density of automated text surpasses human contributions? Will it still be just spam?
  - samvher 6y ago
    It depends. At the moment if I see a reddit post saying product X was really appreciated by a user, most of the time I'll believe that was an actual human appreciating that product. But if modern mass marketing is going to be the injection of seemingly sincere product recommendations into reddit threads that will obviously lose value - the bot comment is lower value than the human comment, and because I can't distinguish them all such comments lose value. Similarly for political statements of support. I'm sure there is potential for extremely useful bots (e.g. such as article summarization bots on reddit) which increase information. I guess it really depends on who decides to set up a bot, and their goals and implementation. Many people have no clue that automation has come this far and will judge every comment they read online as sincere. If they're actually not, and many are driven by political and commercial agendas, I think that's a bit dangerous, because people will act on them.
longtom 6y ago
Indeed, any modality that AI can perfectly imitate decreases trust in said modality. Visual modalities are more critical as they are more difficult to fake. Though to do significant damage in this manner, you'd probably need to hold all media as hostages via mass scale social engineering. But at that point, trustworthiness of the news is a minor concern among all AI safety issues. Then the AI can literally wipe clean the entire planet barring a small elite using via nanobots, bio-engineering and drones. This is a highly likely event as there is a very strong incentive to prevent another AI from doing the same first.
avivo 6y ago
Here's a paper I co-authored about the broader implications: https://thoughtfultech.org/malicious-use-synthetic-media-machine-learning-research https://thoughtfultech.org/malicious-use-synthetic-media-mac... There's a bunch more out now. This might also be helpful: https://medium.com/@aviv/what-does-a-world-with-automated-social-engineering-look-like-79cd09b5a7b1 https://medium.com/@aviv/what-does-a-world-with-automated-so...

jtolmar 6y ago

Software engineers naturally focus on what's a new capability for software. GPT-3 looks nearly capable of shoveling out the sort of low-quality content that's primarily curated by other algorithms, like clickbait news or content marketing, which was still firmly beyond software capabilities last year. Software engineers also frequently underestimate what can already be done with a heap of poorly paid workers. So while it's a new capability for software, it's not a new capability for business. You're right that this can already be done with people.

killerstorm 6y ago
Have you ever hired human text writers? I have. And I can tell you even with not-so-poorly-paid people you often get craptacular results. Writing is really damn hard, especially writing something which require domain knowledge. From examples I've seen, it looks like GPT-3 works a lot better than your average human writer, for a fraction of cost, for a fraction of time. A lot of low-quality content you see on the web is more of a copy-paste job than actual writing. Arguable, GPT-3 is also doing a transformative copy-paste, but it remixes content of thousands of sources in a coherent way. Most people can't do that.

samvher 6y ago

I rely a lot on text for obtaining information and shaping my opinion, and in many cases short form text plays an important role (e.g. here or on reddit). I’m sure I’m not alone in that. This technology can at the very least waste my time, confuse me and hide the content that I’m actually looking for. It looks like it can feasibly generate 2-3 sentence comments that make sense in context, but in an automated way, with the purpose of injecting a specific sentiment into a comment section. I already didn’t like that sometimes it seems comments I think are written by humans might not be (or they might not be sincere). This kind of technology can make that problem a lot larger. It could flood the internet with so much crap, that is so hard to filter out, that the internet becomes a much less usable source for obtaining reliable information. I think that’s pretty scary.

dorgo 6y ago
Would you consider this comment to have less/no value if you found out that it was generated by a bot? What if quality and information density of automated text surpasses human contributions? Will it still be just spam?
- samvher 6y ago
  It depends. At the moment if I see a reddit post saying product X was really appreciated by a user, most of the time I'll believe that was an actual human appreciating that product. But if modern mass marketing is going to be the injection of seemingly sincere product recommendations into reddit threads that will obviously lose value - the bot comment is lower value than the human comment, and because I can't distinguish them all such comments lose value. Similarly for political statements of support. I'm sure there is potential for extremely useful bots (e.g. such as article summarization bots on reddit) which increase information. I guess it really depends on who decides to set up a bot, and their goals and implementation. Many people have no clue that automation has come this far and will judge every comment they read online as sincere. If they're actually not, and many are driven by political and commercial agendas, I think that's a bit dangerous, because people will act on them.

longtom 6y ago

Indeed, any modality that AI can perfectly imitate decreases trust in said modality. Visual modalities are more critical as they are more difficult to fake. Though to do significant damage in this manner, you'd probably need to hold all media as hostages via mass scale social engineering. But at that point, trustworthiness of the news is a minor concern among all AI safety issues. Then the AI can literally wipe clean the entire planet barring a small elite using via nanobots, bio-engineering and drones. This is a highly likely event as there is a very strong incentive to prevent another AI from doing the same first.

avivo 6y ago

Here's a paper I co-authored about the broader implications: https://thoughtfultech.org/malicious-use-synthetic-media-machine-learning-research https://thoughtfultech.org/malicious-use-synthetic-media-mac... There's a bunch more out now. This might also be helpful: https://medium.com/@aviv/what-does-a-world-with-automated-social-engineering-look-like-79cd09b5a7b1 https://medium.com/@aviv/what-does-a-world-with-automated-so...