3 ms·

OpenAI is run by people who believe and spread horror stories about how AI can totally change society, for better or worse. I think this is reflective of a sub

OpenAI is run by people who believe and spread horror stories about how AI can totally change society, for better or worse. I think this is reflective of a substantial disconnect and polarisation in society, because OpenAI's position here was driven by a very fundamental intuition about human nature which is not universally shared. Namely, their rationale was something like this: the world is dominated by people whose opinions are fundamentally shaped by what others are saying, and moreover, by how frequently other people seem to be saying it. They don't really think about things or rely on their own experience: they're just mimics. In such a world being able to auto-generate fake messages of support for some political position or another at scale would give you immense power, because the population would automatically swing behind you based merely on the perception that everyone was swinging behind you. So that's their fear. But is it realistic? Well, this is where we get into the polarisation. "You aren't smart enough to have an opinion" is the sort of viewpoint that leads to an elitist vs populist conflict. There have been reams of analysis about this and it's not really AI specific - e.g. did people vote for Brexit because of Twitter, or because of things they saw in the press, or did they vote based on their own experiences, or what their close friends/family thought, or what mix of "all of the above" is the truest mix? When they refused to release GPT-2 the OpenAI researchers took an extreme on that spectrum, asserting that essentially they had built a mind control device. But of course which didn't work on themselves, only on lesser minds. Not surprisingly this was very controversial, albeit I think the Valley/Hacker News set would be surprised at just how controversial it would have been if anyone outside the AI world had really noticed. You can't tell most of the world their opinions aren't really their own and not expect pushback. Personally I don't share those intuitions at all. The controversy over DeepFakes is similar. Photoshop has existed for decades and hasn't led to a dystopia, but suddenly DeepFakes is going to create fundamental social change? I don't think so. AI researchers, especially in academia, need big narratives to keep up the funding and suggest they're on a mission to save the world vs e.g. making slightly better playlist recommendations. GPT-2 seems like a classic case of this being taken to absurdity.

Veedrac 7y ago
> When they refused to release GPT-2 the OpenAI researchers took an extreme on that spectrum, asserting that essentially they had built a mind control device. Having read everything I could find from OpenAI on this topic, I have no recollection at all of such claims by OpenAI. Sources would be appropriate.
- repolfx 7y ago
  They said they wouldn't release the biggest model because it could be abused to make bots that sway politics. Consider the chain of reasoning required to believe that.
  - Veedrac 7y ago
    Again, I suggest you give direct sources for your arguments.
    - repolfx 7y ago
      Are you sure you read their announcement blog? You seem to believe that they didn't say what they spent an entire blog post saying. Here it is again. https://openai.com/blog/better-language-models/ https://openai.com/blog/better-language-models/ They said "Due to concerns about large language models being used to generate deceptive, biased, or abusive language at scale, we are only releasing a much smaller version of GPT-2 along with sampling code." And in case it wasn't clear what they mean by "abusive language", they clarified: "Today, malicious actors—some of which are political in nature—have already begun to target the shared online commons, using things like “robotic tools, fake accounts and dedicated teams to troll individuals with hateful commentary or smears that make them afraid to speak, or difficult to be heard or believed”" "These findings, combined with earlier results on synthetic imagery, audio, and video, imply that technologies are reducing the cost of generating fake content and waging disinformation campaigns. The public at large will need to become more skeptical of text they find online." Or summarised: • We think people believe things they read online today. • We think there are political "disinformation campaigns" being waged, by the sort of people who would use AI to wage them more effectively. • We think GPT-2 would let them create "deceptive, biased or abusive language" automatically, and this is too dangerous to release because people aren't skeptical enough of what they read online. • We think the entire population needs to change its behaviour for GPT-2 to be safe to release. I think that directly supports my arguments. Their whole worldview assumes the population is naive, easily manipulated by text and more importantly the volume of it ... they also draw a bogus analogy to DeepFakes. It's bogus because nothing stops anyone writing words today that OpenAI staff might find "biased" or "deceptive", literally anyone can do that. But substituting a celebrities face onto porn would require very advanced video editing skills most people don't have: DeepFakes is a truly new capability, whereas GPT-2 is merely a capability of scale. To believe GPT-2 changes anything you must believe the majority of people make up their minds on political issues based purely on volumes of anonymous online comments. Hence my argument.
paggle 7y ago
Why does “the world is dominated by people whose opinions are fundamentally shaped by what others are saying, and moreover, by how frequently other people seem to be saying it“ imply an elitist vs populist class divide, rather than applying to EVERYONE? I can honestly say I have never met a person whose thoughts were not primarily a construct of their environment. Take everyone I’ve ever known, and have them born 2000 years ago in Greece, and they’d probably believe that slavery is fine, child sexual slavery is a perk of being rich, and that it’s perfectly normal to burn heretics. Anyone whose thoughts weren’t dominated by societal cues would probably end up in jail.
- AgentME 7y ago
  Yeah, there's very few of my opinions I hold now that I would've been able to invent in a vacuum. I adopted pretty much all of my opinions from seeing them online among others and reasoning my way to decide between the options. If I had never seen the opinions I now hold, then I probably wouldn't have picked them. If these good opinions that I ended up picking had been lost in the noise of a sheer flood of bad opinions, then I would've become a different person easily. The idea of a good text-generating AI possibly being able to flood forums is a little scary, and the idea that even the little bit of caution OpenAI has done is elitist is ridiculous.
- repolfx 7y ago
  You're conflating environment and reading of written opinions. People are products of their experiences, sure. That experience is much wider than "possibly bot authored comments on social media", but OpenAI is acting as if it isn't. Most people won't agree with their stance. It does however seem to have some weight in certain social circles, the sort of circles that assume most people aren't educated enough to make good decisions.
derefr 7y ago
> asserting that essentially they had built a mind control device. But of course which didn't work on themselves, only on lesser minds More like asserting that they had built a nuclear reactor, which someone else could easily turn into a nuclear bomb. They haven't turned it into a nuclear bomb themselves, so the fact that they haven't been blown up by it doesn't require any special justification.
- Judgmentality 7y ago
  My real issue is they started as a non-profit supposedly developing AI for the benefit of humanity, then quickly became a for-profit with technology so dangerous only they are allowed to wield it. If they just said "we are developing AI because AI is awesome" I would have a lot more respect for them than I currently do. I wish they would at least change their name. As it stands, it's hard for me to see their release process as truly an experiment and not as just a clever marketing ploy - because people are talking about them a lot more now than before.

Veedrac 7y ago

> When they refused to release GPT-2 the OpenAI researchers took an extreme on that spectrum, asserting that essentially they had built a mind control device. Having read everything I could find from OpenAI on this topic, I have no recollection at all of such claims by OpenAI. Sources would be appropriate.

repolfx 7y ago
They said they wouldn't release the biggest model because it could be abused to make bots that sway politics. Consider the chain of reasoning required to believe that.
- Veedrac 7y ago
  Again, I suggest you give direct sources for your arguments.
  - repolfx 7y ago
    Are you sure you read their announcement blog? You seem to believe that they didn't say what they spent an entire blog post saying. Here it is again. https://openai.com/blog/better-language-models/ https://openai.com/blog/better-language-models/ They said "Due to concerns about large language models being used to generate deceptive, biased, or abusive language at scale, we are only releasing a much smaller version of GPT-2 along with sampling code." And in case it wasn't clear what they mean by "abusive language", they clarified: "Today, malicious actors—some of which are political in nature—have already begun to target the shared online commons, using things like “robotic tools, fake accounts and dedicated teams to troll individuals with hateful commentary or smears that make them afraid to speak, or difficult to be heard or believed”" "These findings, combined with earlier results on synthetic imagery, audio, and video, imply that technologies are reducing the cost of generating fake content and waging disinformation campaigns. The public at large will need to become more skeptical of text they find online." Or summarised: • We think people believe things they read online today. • We think there are political "disinformation campaigns" being waged, by the sort of people who would use AI to wage them more effectively. • We think GPT-2 would let them create "deceptive, biased or abusive language" automatically, and this is too dangerous to release because people aren't skeptical enough of what they read online. • We think the entire population needs to change its behaviour for GPT-2 to be safe to release. I think that directly supports my arguments. Their whole worldview assumes the population is naive, easily manipulated by text and more importantly the volume of it ... they also draw a bogus analogy to DeepFakes. It's bogus because nothing stops anyone writing words today that OpenAI staff might find "biased" or "deceptive", literally anyone can do that. But substituting a celebrities face onto porn would require very advanced video editing skills most people don't have: DeepFakes is a truly new capability, whereas GPT-2 is merely a capability of scale. To believe GPT-2 changes anything you must believe the majority of people make up their minds on political issues based purely on volumes of anonymous online comments. Hence my argument.

paggle 7y ago

Why does “the world is dominated by people whose opinions are fundamentally shaped by what others are saying, and moreover, by how frequently other people seem to be saying it“ imply an elitist vs populist class divide, rather than applying to EVERYONE? I can honestly say I have never met a person whose thoughts were not primarily a construct of their environment. Take everyone I’ve ever known, and have them born 2000 years ago in Greece, and they’d probably believe that slavery is fine, child sexual slavery is a perk of being rich, and that it’s perfectly normal to burn heretics. Anyone whose thoughts weren’t dominated by societal cues would probably end up in jail.

AgentME 7y ago
Yeah, there's very few of my opinions I hold now that I would've been able to invent in a vacuum. I adopted pretty much all of my opinions from seeing them online among others and reasoning my way to decide between the options. If I had never seen the opinions I now hold, then I probably wouldn't have picked them. If these good opinions that I ended up picking had been lost in the noise of a sheer flood of bad opinions, then I would've become a different person easily. The idea of a good text-generating AI possibly being able to flood forums is a little scary, and the idea that even the little bit of caution OpenAI has done is elitist is ridiculous.
repolfx 7y ago
You're conflating environment and reading of written opinions. People are products of their experiences, sure. That experience is much wider than "possibly bot authored comments on social media", but OpenAI is acting as if it isn't. Most people won't agree with their stance. It does however seem to have some weight in certain social circles, the sort of circles that assume most people aren't educated enough to make good decisions.

derefr 7y ago

> asserting that essentially they had built a mind control device. But of course which didn't work on themselves, only on lesser minds More like asserting that they had built a nuclear reactor, which someone else could easily turn into a nuclear bomb. They haven't turned it into a nuclear bomb themselves, so the fact that they haven't been blown up by it doesn't require any special justification.

Judgmentality 7y ago
My real issue is they started as a non-profit supposedly developing AI for the benefit of humanity, then quickly became a for-profit with technology so dangerous only they are allowed to wield it. If they just said "we are developing AI because AI is awesome" I would have a lot more respect for them than I currently do. I wish they would at least change their name. As it stands, it's hard for me to see their release process as truly an experiment and not as just a clever marketing ploy - because people are talking about them a lot more now than before.