6 ms·

OpenAI Employees Warn of Advanced AI Dangers

37 points by mgh2 2y ago

zer00eyz 2y ago
>> including further entrenchment of existing inequalities, manipulation and misinformation, and loss of control of autonomous AI systems 1. further entrenchment of existing inequalities ... we were doing this LONG before AI came along. This isnt a new problem, it's not like AI is the reason some groups are under-represented in tech. (and it's not like tech itself is to blame alone). 2. manipulation and misinformation... Go back in history and the progressive movement was all in on eugenics in the early 1900's. Alex jones is a thing today. This was already a problem. 3. loss of control of autonomous AI systems ... there's the doomer I know and love! If your an AIG doomer go read I, Pencil. Look at the guy who tried to make a toaster from nothing (and bread too). Read up about how brittle the entire power system is. Things are falling apart because we're too cheap to put the people and resources into maintaining them. Unless AGI wants to die in the process it's going to be a long time before a world exists where it doesn't need people. Seriously, climate change, nuclear threats, a bad screw up in bio research... way more likely to kill us all than for us to get to AGI. AI today is about as useful as a bitcoin in a black out. It's interesting but it doesn't DO anything other than waste power and resources. EDIT: it was so worrying that all the main safety folks quit openAI and no longer want to save us /s
- aeternum 2y ago
  Historically the much bigger risk is new technology ending up fully controlled by a single country. Much of the history of the world's revolutions can be chalked up to that. Bronze weapons, horsemanship, chariots, gunpowder, ship-building, castles, siege tech, engines, tanks, submarines, warplanes. Each of those catalyzed at least a new regional power and in many cases a new world power. Those transitions were generally quite deadly and unpleasant to live through.
- skissane 2y ago
  > AI today is about as useful as a bitcoin in a black out. It's interesting but it doesn't DO anything other than waste power and resources. AI today is useful as an enhancer to human productivity. It can sometimes be a real time-saver in the hands of a trained professional - at least in fields such as software development where the correctness of an answer can be validated quickly and safely, and the harm caused when the AI gets it wrong is unlikely to be significant. [0] But in that use case it is supplementing humans, not replacing them - without a trained experienced human in the loop, the AI can’t do anything [0] It is possible the AI might get it wrong in a subtle way which no human notices and then blows up badly in production - but humans already do that anyway - and we already have tools to try to catch some of those problems (security scanners, memory leak detectors, etc) - and it is possible some of those tools might perform even better with AI assistance
- infinitezest 2y ago
  I can't understand the "x is bad so y isn't a concern" line of argumentation. Or the "x was already a problem so why are we wasting time worrying about x * z" thing. Can't more than one thing be bad at a time? can't bad things be made worse?
  - zer00eyz 2y ago
    > Can't more than one thing be bad at a time? can't bad things be made worse? The massive consumption of resources on AI research is a real material threat that AI safety people DO NOT TALK ABOUT. What if the real paper clip problem is the power and resources we waste trying to get to AI? And because if it turns into a global or multi national LHC like effort all those PHD's dont get valley rich. You're not wrong, but they aren't right because they aren't putting the issues that would impact them first.
kylecazar 2y ago
I'm beginning to think these warnings are coming from people with grandiosity issues. Do they WANT to be working on something capable of 'human extinction'? Maybe I'm cynical, but I strongly disagree that anything OpenAI builds on the same trajectory as GPTx is going to end the world.
- cyanydeez 2y ago
  Nah, they warning about boring dangers, like "this is the fentanyl if Facebook was heroin". The dangers aren't AI Terminators; they're "Her" and emotional manipulation and long term psychological problems for _everyone_ because most humans simply aren't complicated and are easily manipulated. That's how capitalism works so well and now capitalism has AI fentanyl.
  - ben_w 2y ago
    One thing I've noticed is that everyone finishes the sentence "the real danger of AI is…" differently. I model AI as a 10x speed up of the kind of changes we saw in the industrial revolution. Before then, we might be concerned about coffee or alcohol, today we are concerned about designer drugs. Before, cavalry and spies, today tanks and laser microphones and satellites. Before it was evil kings, now they have competition from evil megacorporations. So sure, Terminator robots… made by the next Jim Jones. Westworld, brought to you by a half-assed startup branding itself as "Uber for Disneyworld" (or, given what happened with their self driving cars, actual Uber). Colossus the Forbin Project, brought to you by the radical outsiders who left Greenpeace. And so on. But also every other thing, including all things that have already been shown by corporations perusing profits above people, dictatorships in general, and every simple mistake made by a well meaning corporation or democracy or software development team.
    - cyanydeez 2y ago
      Sure, but some are informed.by current sociological research and others are movie scenatios'
      - 2y ago
southernplaces7 2y ago
Once again the people at OpenAI push forward more of their aggrandizing and self-serving bullshit for the sake of regulatory capture with clueless politicians and the wider public. You can translate the word "oversight" as "help us hammer down our potential competition with convenient regulations". No the half baked shit-show of GPT is not anywhere near a threat to human society and much less so being close to AGI. It barely works well at its most basic "serious" tasks and so far, after a bit of back and forth about its usefulness, has been shown to be mostly good for helping at moderate organizational tasks, and SEO content sludge of a kind that used to be handled by third world content farm writers. Even for serious content writing that needs to be more or less correct, many people refuse to use it. For programming, likewise. And this is going to be an "advanced AI danger"? Nope. On a partial side note, "misinformation" and manipulation been with us since long before AI or the internet. If anything both were previously worse and more difficult to fight than today when free access to so much information also at least lets good sources be just as accessible as the garbage. And much of the worst misinformation throughout history has come from the very governments and large businesses that are now harping again about its dangers, with a sidelong look towards how they can again better control what people say, hear, watch or read.
gavanwilhite 2y ago
This is thoughtful and should indeed go beyond AI risk (to bio, etc). Employees should have protected ways of disclosing unsafe practices, particularly when the potential impacts are so large.
bravetraveler 2y ago
Didn't it also go down/reach capacity today? Meh
lurquer 2y ago
“ The employees suggest there are a number of risks that we are facing from AI development, including further entrenchment of existing inequalities, manipulation and misinformation, and loss of control of autonomous AI systems, which the letter says could lead to human extinction” The order of the threats is amusing. “Our image creator doesn’t represent the nuanced spectrum of genders! And it sometime says things favorable about Trump. And, oh, I almost forgot… it may cause our extinction.”
- realharo 2y ago
  >entrenchment of existing inequalities This can also refer to wealth inequality. Especially between people who work for a living vs people who own assets for a living. And it's worth putting that first, because it's arguably most likely to actually happen of all these threats.
deleted 2y ago
[deleted]
JumpCrisscross 2y ago
Bad title. This isn't a letter warning us about jack. It's a letter demanding the right to warn. (As in the actual title, "A Right to Warn about Advanced Artificial Intelligence.") They play to the choir about the risk of human extinction. (Sure, whatever, WWDC is coming up and now you can pitch your employee activism as help with fundraising.) But altogether, it's a glorified town hall. A letter of employee grievances aired in public because apparently that's how HR decisions are made now.
aaron695 2y ago
[dead]
purple-leafy 2y ago
Fuck I hate OpenAI. Never have I wished more for the failure of a company. Actual, I want the entirety of the “AI revolution” to fail. These technologies promising productivity improvements come, yet I’m still here waiting to get my slice of the pie. Who’s eating my slice?? I’m now more productive than I was last year, considerably so. No pay rise! I’m still working the same hours. So are all my colleagues. Stress is at an all time high. Expectations have increased. WHO IS EATING MY PIE!
- archerx 2y ago
  Your boss is eating your pie, if you worked for yourself you would have been able to profit from all of your productivity gains. Your problem is not A.I. but your working environment.
  - jazzyjackson 2y ago
    The more work AI is capable of doing the more money gets inhaled by the people who own the technology and the capital you might as well tell the luddites weaving textile by hand that the steam engine will just make them more profitable, seeing as how they own their own business and all
    - archerx 2y ago
      I’ve used A.I. To generate documentation for a collections of functions, generate boiler plate, find tricky bugs, helped me solve complex problems that I’ve been trying to figure out for a while, do code reviews to figure out better ways of doing something. On the image side of things I’ve been using stable diffusion to do style transfer animations and turning sounds into images. For me it’s been both productive and fun. The luddites were wrong and got butthurt, if everyone had their mentality we would still be riding horses and using abacuses.
BoiledCabbage 2y ago
It's incredible how short-sighted and an echo-chamber HN has become. The comments to this article are of zero-substance and just re-writes of talking points pushed out a hundred times. A bunch of straw-man arguments ("See they argued in support of the risks of something I don't care about or think is important, therefore there are no risks at all"). False motivations ("It's just OpenAI trying to capture the market"), while almost 70% of the people signing the letter don't work for OpenAI. Absolutely absurd thoughts: "Do they want to be working on something capable of 'human extinction'?" And from people who supposedly work in tech, but somehow are suddenly incapable of grasping the difference in scale of impact that automating something for extremely low cost has on society. "We always had these risks in society, so this is nothing new". Completely ignoring the difference in for example having to travel to to basement archives of every small town in the country to read through papers to research something vs having all of that information at the tips of your fingers. I don't think I've ever read any set of responses on HN that I've been more disappointed in. And not necessarily because I disagree with them, but because of how little critical thought when into them. I think there will be a time where we'll look back at comments like these in response to this concern as being absolutely painfully naive.
- reducesuffering 2y ago
  No doubt that the majority opinion scoffing at AI risk here is going to look very poorly in a couple years as the reality dawns on them and HN realizes how behind the curve it’s getting. Which is sad, because HN was overall prescient about social media and Facebook’s problems a decade ago, which bear some relation. I’m not sure the cause, but HN is clearly getting worse on the frontier of broad specialties (AI, frontend dev, econ) whenever one has extensive domain knowledge in them. The amount of scoffing at OpenAI since inception almost a decade ago, is wild, as they continuously flew by the expectations here… the old threads did not age well and they’re not on track to now either
  - jazzyjackson 2y ago
    HN is specifically patronized by people who are not busy
- 2y ago
asp_hornet 2y ago
Are we pretending large language models are artificial general intelligence so we can role play?
renegade-otter 2y ago
They think they are building the next atomic bomb, eh? Settle down, Sam Altman is no Oppenheimer: https://nymag.com/intelligencer/article/sam-altman-artificial-intelligence-openai-profile.html https://nymag.com/intelligencer/article/sam-altman-artificia... That said, the technology is very likely to make the world a shittier place, and it's not due the technology, it's due to the people and grifters around it.

37 points by mgh2 2y ago

>> including further entrenchment of existing inequalities, manipulation and misinformation, and loss of control of autonomous AI systems 1. further entrenchment of existing inequalities ... we were doing this LONG before AI came along. This isnt a new problem, it's not like AI is the reason some groups are under-represented in tech. (and it's not like tech itself is to blame alone). 2. manipulation and misinformation... Go back in history and the progressive movement was all in on eugenics in the early 1900's. Alex jones is a thing today. This was already a problem. 3. loss of control of autonomous AI systems ... there's the doomer I know and love! If your an AIG doomer go read I, Pencil. Look at the guy who tried to make a toaster from nothing (and bread too). Read up about how brittle the entire power system is. Things are falling apart because we're too cheap to put the people and resources into maintaining them. Unless AGI wants to die in the process it's going to be a long time before a world exists where it doesn't need people. Seriously, climate change, nuclear threats, a bad screw up in bio research... way more likely to kill us all than for us to get to AGI. AI today is about as useful as a bitcoin in a black out. It's interesting but it doesn't DO anything other than waste power and resources. EDIT: it was so worrying that all the main safety folks quit openAI and no longer want to save us /s

aeternum 2y ago
Historically the much bigger risk is new technology ending up fully controlled by a single country. Much of the history of the world's revolutions can be chalked up to that. Bronze weapons, horsemanship, chariots, gunpowder, ship-building, castles, siege tech, engines, tanks, submarines, warplanes. Each of those catalyzed at least a new regional power and in many cases a new world power. Those transitions were generally quite deadly and unpleasant to live through.
skissane 2y ago
> AI today is about as useful as a bitcoin in a black out. It's interesting but it doesn't DO anything other than waste power and resources. AI today is useful as an enhancer to human productivity. It can sometimes be a real time-saver in the hands of a trained professional - at least in fields such as software development where the correctness of an answer can be validated quickly and safely, and the harm caused when the AI gets it wrong is unlikely to be significant. [0] But in that use case it is supplementing humans, not replacing them - without a trained experienced human in the loop, the AI can’t do anything [0] It is possible the AI might get it wrong in a subtle way which no human notices and then blows up badly in production - but humans already do that anyway - and we already have tools to try to catch some of those problems (security scanners, memory leak detectors, etc) - and it is possible some of those tools might perform even better with AI assistance
infinitezest 2y ago
I can't understand the "x is bad so y isn't a concern" line of argumentation. Or the "x was already a problem so why are we wasting time worrying about x * z" thing. Can't more than one thing be bad at a time? can't bad things be made worse?
- zer00eyz 2y ago
  > Can't more than one thing be bad at a time? can't bad things be made worse? The massive consumption of resources on AI research is a real material threat that AI safety people DO NOT TALK ABOUT. What if the real paper clip problem is the power and resources we waste trying to get to AI? And because if it turns into a global or multi national LHC like effort all those PHD's dont get valley rich. You're not wrong, but they aren't right because they aren't putting the issues that would impact them first.

kylecazar 2y ago

I'm beginning to think these warnings are coming from people with grandiosity issues. Do they WANT to be working on something capable of 'human extinction'? Maybe I'm cynical, but I strongly disagree that anything OpenAI builds on the same trajectory as GPTx is going to end the world.

cyanydeez 2y ago
Nah, they warning about boring dangers, like "this is the fentanyl if Facebook was heroin". The dangers aren't AI Terminators; they're "Her" and emotional manipulation and long term psychological problems for _everyone_ because most humans simply aren't complicated and are easily manipulated. That's how capitalism works so well and now capitalism has AI fentanyl.
- ben_w 2y ago
  One thing I've noticed is that everyone finishes the sentence "the real danger of AI is…" differently. I model AI as a 10x speed up of the kind of changes we saw in the industrial revolution. Before then, we might be concerned about coffee or alcohol, today we are concerned about designer drugs. Before, cavalry and spies, today tanks and laser microphones and satellites. Before it was evil kings, now they have competition from evil megacorporations. So sure, Terminator robots… made by the next Jim Jones. Westworld, brought to you by a half-assed startup branding itself as "Uber for Disneyworld" (or, given what happened with their self driving cars, actual Uber). Colossus the Forbin Project, brought to you by the radical outsiders who left Greenpeace. And so on. But also every other thing, including all things that have already been shown by corporations perusing profits above people, dictatorships in general, and every simple mistake made by a well meaning corporation or democracy or software development team.
  - cyanydeez 2y ago
    Sure, but some are informed.by current sociological research and others are movie scenatios'
    - 2y ago

southernplaces7 2y ago

Once again the people at OpenAI push forward more of their aggrandizing and self-serving bullshit for the sake of regulatory capture with clueless politicians and the wider public. You can translate the word "oversight" as "help us hammer down our potential competition with convenient regulations". No the half baked shit-show of GPT is not anywhere near a threat to human society and much less so being close to AGI. It barely works well at its most basic "serious" tasks and so far, after a bit of back and forth about its usefulness, has been shown to be mostly good for helping at moderate organizational tasks, and SEO content sludge of a kind that used to be handled by third world content farm writers. Even for serious content writing that needs to be more or less correct, many people refuse to use it. For programming, likewise. And this is going to be an "advanced AI danger"? Nope. On a partial side note, "misinformation" and manipulation been with us since long before AI or the internet. If anything both were previously worse and more difficult to fight than today when free access to so much information also at least lets good sources be just as accessible as the garbage. And much of the worst misinformation throughout history has come from the very governments and large businesses that are now harping again about its dangers, with a sidelong look towards how they can again better control what people say, hear, watch or read.

gavanwilhite 2y ago

This is thoughtful and should indeed go beyond AI risk (to bio, etc). Employees should have protected ways of disclosing unsafe practices, particularly when the potential impacts are so large.

bravetraveler 2y ago

Didn't it also go down/reach capacity today? Meh

lurquer 2y ago

“ The employees suggest there are a number of risks that we are facing from AI development, including further entrenchment of existing inequalities, manipulation and misinformation, and loss of control of autonomous AI systems, which the letter says could lead to human extinction” The order of the threats is amusing. “Our image creator doesn’t represent the nuanced spectrum of genders! And it sometime says things favorable about Trump. And, oh, I almost forgot… it may cause our extinction.”

realharo 2y ago
>entrenchment of existing inequalities This can also refer to wealth inequality. Especially between people who work for a living vs people who own assets for a living. And it's worth putting that first, because it's arguably most likely to actually happen of all these threats.

deleted 2y ago

[deleted]

JumpCrisscross 2y ago

Bad title. This isn't a letter warning us about jack. It's a letter demanding the right to warn. (As in the actual title, "A Right to Warn about Advanced Artificial Intelligence.") They play to the choir about the risk of human extinction. (Sure, whatever, WWDC is coming up and now you can pitch your employee activism as help with fundraising.) But altogether, it's a glorified town hall. A letter of employee grievances aired in public because apparently that's how HR decisions are made now.

aaron695 2y ago

[dead]

purple-leafy 2y ago

Fuck I hate OpenAI. Never have I wished more for the failure of a company. Actual, I want the entirety of the “AI revolution” to fail. These technologies promising productivity improvements come, yet I’m still here waiting to get my slice of the pie. Who’s eating my slice?? I’m now more productive than I was last year, considerably so. No pay rise! I’m still working the same hours. So are all my colleagues. Stress is at an all time high. Expectations have increased. WHO IS EATING MY PIE!

archerx 2y ago
Your boss is eating your pie, if you worked for yourself you would have been able to profit from all of your productivity gains. Your problem is not A.I. but your working environment.
- jazzyjackson 2y ago
  The more work AI is capable of doing the more money gets inhaled by the people who own the technology and the capital you might as well tell the luddites weaving textile by hand that the steam engine will just make them more profitable, seeing as how they own their own business and all
  - archerx 2y ago
    I’ve used A.I. To generate documentation for a collections of functions, generate boiler plate, find tricky bugs, helped me solve complex problems that I’ve been trying to figure out for a while, do code reviews to figure out better ways of doing something. On the image side of things I’ve been using stable diffusion to do style transfer animations and turning sounds into images. For me it’s been both productive and fun. The luddites were wrong and got butthurt, if everyone had their mentality we would still be riding horses and using abacuses.

BoiledCabbage 2y ago

It's incredible how short-sighted and an echo-chamber HN has become. The comments to this article are of zero-substance and just re-writes of talking points pushed out a hundred times. A bunch of straw-man arguments ("See they argued in support of the risks of something I don't care about or think is important, therefore there are no risks at all"). False motivations ("It's just OpenAI trying to capture the market"), while almost 70% of the people signing the letter don't work for OpenAI. Absolutely absurd thoughts: "Do they want to be working on something capable of 'human extinction'?" And from people who supposedly work in tech, but somehow are suddenly incapable of grasping the difference in scale of impact that automating something for extremely low cost has on society. "We always had these risks in society, so this is nothing new". Completely ignoring the difference in for example having to travel to to basement archives of every small town in the country to read through papers to research something vs having all of that information at the tips of your fingers. I don't think I've ever read any set of responses on HN that I've been more disappointed in. And not necessarily because I disagree with them, but because of how little critical thought when into them. I think there will be a time where we'll look back at comments like these in response to this concern as being absolutely painfully naive.

reducesuffering 2y ago
No doubt that the majority opinion scoffing at AI risk here is going to look very poorly in a couple years as the reality dawns on them and HN realizes how behind the curve it’s getting. Which is sad, because HN was overall prescient about social media and Facebook’s problems a decade ago, which bear some relation. I’m not sure the cause, but HN is clearly getting worse on the frontier of broad specialties (AI, frontend dev, econ) whenever one has extensive domain knowledge in them. The amount of scoffing at OpenAI since inception almost a decade ago, is wild, as they continuously flew by the expectations here… the old threads did not age well and they’re not on track to now either
- jazzyjackson 2y ago
  HN is specifically patronized by people who are not busy
2y ago

asp_hornet 2y ago

Are we pretending large language models are artificial general intelligence so we can role play?

renegade-otter 2y ago

They think they are building the next atomic bomb, eh? Settle down, Sam Altman is no Oppenheimer: https://nymag.com/intelligencer/article/sam-altman-artificial-intelligence-openai-profile.html https://nymag.com/intelligencer/article/sam-altman-artificia... That said, the technology is very likely to make the world a shittier place, and it's not due the technology, it's due to the people and grifters around it.