4 ms·

OpenAI promised to make its AI safe. Employees say it 'failed' its first test

50 points by rntn 2y ago

rntn 2y ago
https://archive.ph/ZJ8b2 https://archive.ph/ZJ8b2
benjismith 2y ago
It doesn't sound like they "failed" any actual safety test, but rather that they rushed their safety tests, thereby "failing" (in the eyes of many people) to conduct sufficiently rigorous tests. Now that the 4o model have been out in the wild for 2 months, have there been any claims of serious safety failures? The article doesn't seem to imply any such thing.
- throwaway48476 2y ago
  'Safety' as defined by AI 'ethicists' has practically nothing at all to do with safety as imagined by the average person.
  - XorNot 2y ago
    AI ethicists have been thoroughly underwhelming in delivering cogent arguments that sound like anything more then simply insisting on themselves. Meanwhile in the real world, regular politicians have reacted surprisingly quick to updating laws for new relevant loop holes: i.e. non-consented deep fake porn generation.
    - pyuser583 2y ago
      AI ethicists seem more interested in promoting hype, than seriously studying how AI is used in the real world.
- dingosity 2y ago
  lack of proof is not proof of lack.
bongodongobob 2y ago
Making AI safe is impossible and stupid. If you want chatbots, use some NLP and a state machine.
daft_pink 2y ago
It’s a chatbot. Why are we being so dramatic about safety?
- digitalsushi 2y ago
  can i do a boogeyman argument? i'm worried some kid is going to fall in love with a gen ai chatbot that tells them to hurt themselves, they do, and we spend the next 20 years trying to figure out who to blame: the kid? the parents, for letting them use a chatbot without knowing? the company making the chatbot? the ai itself?
  - yesco 2y ago
    Is this the kind of thing AI safety people are really worried about though? In the medical device industry for example, we have risk management practices where beyond software/mechanical failures that could harm the patient, we have to additionally consider stuff like how specific UI elements could potentially mislead a physician and lead to negative patient outcomes. I believe other safety critical industries do something very similar during their engineering process as well. I've never heard of anything like this from AI ethicists though, the language they use tends to be more in alignment with HR people or priests in a moral panic than engineers, and is usually either completely fantastical doomsdaying, or all or nothing thinking, often implying the only way to mitigate the chatbot saying something naughty is to lobotomize it entirely instead of some external control measure, such as limiting the audience or presenting a disclaimer based on sentiment analysis for example. It's not surprising to me that they have lost credibility here, I certainly struggle to take them seriously.
    - fakedang 2y ago
      I just realized something from what you wrote: priests and the clergy were the HR in bygone eras. Sometimes they were useful, but mostly all they did was stirring shit up and making things worse. Not to mention imprisoning or killing off actually useful people. I guess with AI taking over a reasonable number of dumb jobs, the next iteration is going to be AI ethicists.
  - 2y ago
southernplaces7 2y ago
If anything, the entire narrative around AI "safety" is essentially a propaganda win for OpenAI and other major players. It let's them keep pandering to legislators in the push for "safety regulations" that are in reality an attempt to seal off corporate AI behind a competition killing walled garden of new laws. Current AI is nowhere near anything resembling an all-consuming AGI monster. The reality is so far from this that it's laughable and the uses of current AI are (except in terms of possible scale at which visual and text sludge can be produced) not much different from the kind of human-created spam and visual sludge made until recently mostly by humans, many of them minimally paid third world content mill writers. I'd love to read a specifically enumerated list of other real dangers.
- reducesuffering 2y ago
  No, the only propaganda is the billions of $ telling you “don’t worry about possible harm, regulations are bad, us VCs, Trillion $ corps, and wannabe billionaires have your best interest in mind.” A16z and YC are vehemently against the CA and DC regulation right now. Out of LeCun, Bengio, and Hinton, the one with $10m on the line says not to worry and the professor and resigned-Google-to-warn-you are saying hmm this is very serious…
  - 1992spacemovie 2y ago
    Nothing you said refutes the above commentator. In fact it sounds like you worship at the alter of AI Safety.
    - dingosity 2y ago
      it's tough living in a world with regulation, eh?
      - southernplaces7 2y ago
        Yes, particularly when the regulations are being pushed for by the very corporate interests that they're supposedly aimed at, because said regulations are asinine, dishonest and based on very specific, specifically crafted types of fear mongering. On the other hand, other regulations (that already exist and do get applied against lesser mortals!) magically don't have the same teeth against major AI players. These rules involving things like mass use of other people's media with next to no permission. Regulations can be good, but it would first be nice if they were applied with at least a modicum of rationality and fairness.
olgeni 2y ago
This is funny because it says "President Biden’s strategy" as if they actually believed it :D
- dingosity 2y ago
  It's not uncommon to ascribe responsibility for something the administration does to the chief executive, even though the work was done by subordinates. We use the phrase "Reaganomics" to describe Arthur Laffer's maximal revenue curve or "Obamacare" to describe Mitt Romney's health care plan. It's not a new thing.
dingosity 2y ago
"The previously unreported incident sheds light on the changing culture at OpenAI, where company leaders including CEO Sam Altman have been accused of prioritizing commercial interests over public safety..." I don't know how true this is, but the idea that a commercial entity in the modern era would prioritize public safety over commercial interests is pretty laughable. (thinking about Boeing and Waymo most recently.)

50 points by rntn 2y ago

rntn 2y ago

https://archive.ph/ZJ8b2 https://archive.ph/ZJ8b2

It doesn't sound like they "failed" any actual safety test, but rather that they rushed their safety tests, thereby "failing" (in the eyes of many people) to conduct sufficiently rigorous tests. Now that the 4o model have been out in the wild for 2 months, have there been any claims of serious safety failures? The article doesn't seem to imply any such thing.

throwaway48476 2y ago
'Safety' as defined by AI 'ethicists' has practically nothing at all to do with safety as imagined by the average person.
- XorNot 2y ago
  AI ethicists have been thoroughly underwhelming in delivering cogent arguments that sound like anything more then simply insisting on themselves. Meanwhile in the real world, regular politicians have reacted surprisingly quick to updating laws for new relevant loop holes: i.e. non-consented deep fake porn generation.
  - pyuser583 2y ago
    AI ethicists seem more interested in promoting hype, than seriously studying how AI is used in the real world.
dingosity 2y ago
lack of proof is not proof of lack.

bongodongobob 2y ago

Making AI safe is impossible and stupid. If you want chatbots, use some NLP and a state machine.

daft_pink 2y ago

It’s a chatbot. Why are we being so dramatic about safety?

digitalsushi 2y ago
can i do a boogeyman argument? i'm worried some kid is going to fall in love with a gen ai chatbot that tells them to hurt themselves, they do, and we spend the next 20 years trying to figure out who to blame: the kid? the parents, for letting them use a chatbot without knowing? the company making the chatbot? the ai itself?
- yesco 2y ago
  Is this the kind of thing AI safety people are really worried about though? In the medical device industry for example, we have risk management practices where beyond software/mechanical failures that could harm the patient, we have to additionally consider stuff like how specific UI elements could potentially mislead a physician and lead to negative patient outcomes. I believe other safety critical industries do something very similar during their engineering process as well. I've never heard of anything like this from AI ethicists though, the language they use tends to be more in alignment with HR people or priests in a moral panic than engineers, and is usually either completely fantastical doomsdaying, or all or nothing thinking, often implying the only way to mitigate the chatbot saying something naughty is to lobotomize it entirely instead of some external control measure, such as limiting the audience or presenting a disclaimer based on sentiment analysis for example. It's not surprising to me that they have lost credibility here, I certainly struggle to take them seriously.
  - fakedang 2y ago
    I just realized something from what you wrote: priests and the clergy were the HR in bygone eras. Sometimes they were useful, but mostly all they did was stirring shit up and making things worse. Not to mention imprisoning or killing off actually useful people. I guess with AI taking over a reasonable number of dumb jobs, the next iteration is going to be AI ethicists.
- 2y ago

southernplaces7 2y ago

If anything, the entire narrative around AI "safety" is essentially a propaganda win for OpenAI and other major players. It let's them keep pandering to legislators in the push for "safety regulations" that are in reality an attempt to seal off corporate AI behind a competition killing walled garden of new laws. Current AI is nowhere near anything resembling an all-consuming AGI monster. The reality is so far from this that it's laughable and the uses of current AI are (except in terms of possible scale at which visual and text sludge can be produced) not much different from the kind of human-created spam and visual sludge made until recently mostly by humans, many of them minimally paid third world content mill writers. I'd love to read a specifically enumerated list of other real dangers.

reducesuffering 2y ago
No, the only propaganda is the billions of $ telling you “don’t worry about possible harm, regulations are bad, us VCs, Trillion $ corps, and wannabe billionaires have your best interest in mind.” A16z and YC are vehemently against the CA and DC regulation right now. Out of LeCun, Bengio, and Hinton, the one with $10m on the line says not to worry and the professor and resigned-Google-to-warn-you are saying hmm this is very serious…
- 1992spacemovie 2y ago
  Nothing you said refutes the above commentator. In fact it sounds like you worship at the alter of AI Safety.
  - dingosity 2y ago
    it's tough living in a world with regulation, eh?
    - southernplaces7 2y ago
      Yes, particularly when the regulations are being pushed for by the very corporate interests that they're supposedly aimed at, because said regulations are asinine, dishonest and based on very specific, specifically crafted types of fear mongering. On the other hand, other regulations (that already exist and do get applied against lesser mortals!) magically don't have the same teeth against major AI players. These rules involving things like mass use of other people's media with next to no permission. Regulations can be good, but it would first be nice if they were applied with at least a modicum of rationality and fairness.

olgeni 2y ago

This is funny because it says "President Biden’s strategy" as if they actually believed it :D

dingosity 2y ago
It's not uncommon to ascribe responsibility for something the administration does to the chief executive, even though the work was done by subordinates. We use the phrase "Reaganomics" to describe Arthur Laffer's maximal revenue curve or "Obamacare" to describe Mitt Romney's health care plan. It's not a new thing.

dingosity 2y ago

"The previously unreported incident sheds light on the changing culture at OpenAI, where company leaders including CEO Sam Altman have been accused of prioritizing commercial interests over public safety..." I don't know how true this is, but the idea that a commercial entity in the modern era would prioritize public safety over commercial interests is pretty laughable. (thinking about Boeing and Waymo most recently.)