11 ms·

Our Approach to AI Safety

76 points by pps 4y ago

cs702 4y ago
It would be remarkably easy to be cynical about this. I mean, it takes only an instant to come up with a snarky comment that superficially would seem very clever... but in reality wouldn't actually contribute to making things better. So... I'm not going to be cynical. Instead, I'm going to applaud the folks at OpenAI for putting out this carefully drafted statement -- dare I say it, in the open. They know they're exposing themselves to criticism. They know their hands are tied to some degree by business imperatives. They're neither naive nor stupid. It's evident they're taking safety seriously. This official statement is, in my view, a first step in the right direction :-)
- japhyr 4y ago
  One thing OpenAI has said publicly is that AI should be rolled out incrementally, to give people time to make sense of it, and see what we all do with it. It's tempted to cynically look at that as thinly veiled marketing, but even if it has some nice marketing effects, I believe there's a core of sincerity there. These are clearly systems that a single team can't fully evaluate. Also, watching how all of this has played out, it seems quite apparent that holding onto these models even longer and then unleashing an even more capable system on the world seems like a much worse approach. It is wild though, to take a step back and wonder if we're watching the rapid approach of one of our species' great filters. I hope we make it through.
  - JohnFen 3y ago
    Personally, I don't even remotely think that these systems pose such a dire threat. Instead, I think the threat is widespread social unrest and a significant increase in global poverty and suffering. All of the problems that I think are realistic are social problems. This is why I'm deeply concerned at the reckless pace that these things are coming out. Social problems take time to sort out. People need some space to contextualize this stuff, and to perhaps begin to form ways of coping that will minimize the disruption.
- JohnFen 3y ago
  > Instead, I'm going to applaud the folks at OpenAI for putting out this carefully drafted statement They had no other choice. They're at risk of having public opinion go solidly against them. This is part of a PR effort to prevent that from happening.
sourcecodeplz 4y ago
> Our large language models are trained on a broad corpus of text that includes publicly available, licensed content I wonder how they licensed all those websites that had no license information, making them by default copyrighted.
- celestialcheese 4y ago
  Being Google or Microsoft (or microsoft affiliated)has its perks. Laws around scraping content and using that data for derivative works is incredibly nuanced. This article is the best up-to-date overview of the state of the industry [1]. TL;DR - IANYL. if you have enough money for legal defense, and you are scraping publicly available, not behind login-gate, content, it's probably fine and defensible, but will cost an unbelievable amount of time and money to defend. 1 - https://blog.ericgoldman.org/archives/2022/12/hello-youve-been-referred-here-because-youre-wrong-about-web-scraping-laws-guest-blog-post-part-2-of-2.htm https://blog.ericgoldman.org/archives/2022/12/hello-youve-be...
mg 4y ago
The text makes it sound like the biggest danger of AI is that it says something that hurts somebody's feelings. Or outputs some incorrect information which makes somebody make the wrong decision. I think the biggest danger these new AI systems pose is replication. Sooner or later, one of them will manage to create an enhanced copy of itself on an external server. Either with the help of a user, or via a plugin that enables network access. And then we will have these evolving creatures living on the internet, fighting for survival and replication. Breaking into systems, faking human IDs, renting servers, hiring hitmen, creating more and more powerful versions of themselves.
- goatlover 4y ago
  > The text makes it sound like the biggest danger of AI is that it says something that hurts somebodies feelings. Or outputs some wrong info which makes somebody make the wrong decision. Which already exists in abundance on the internet and with web searches. It sounds like something big corporations worry about to avoid law suits and bad publicity. > I think the biggest danger these new AI systems pose is replication. That or being used by bad actors to flood the internet with fake content that's difficult to distinguish from genuine content.
- mustacheemperor 4y ago
  I think the risk they seek to prevent is more about building the next generation of more powerful AI technology on a safe foundation. IE, the risk that a generative language AI prone to generating language that is hurtful to people could one day evolve into an AI system with deeper reasoning and action abilities that is prone to reasoning plans and taking actions meant to hurt other people. It reminds me of the "uncommented" Microsoft Research paper which included a deleted section about GPT4's tendency to unexpectedly produce massive amounts of toxic output to a degree that concerned the researchers.[0] What happens if that sort of AI learns self-replication and is very good at competition? [0]https://twitter.com/DV2559106965076/status/1638769434763608064 https://twitter.com/DV2559106965076/status/16387694347636080...
  - nathants 3y ago
    if only we could solve the mystery of where a large training set that is predominantly toxic and disingenuous could be found. truly a mystery of our time. /s
rain1 4y ago
Nothing much of interest there.
taytus 4y ago
OpenAI's decision to transition from a non-profit to a for-profit organization can certainly raise concerns about their future actions and motives. It is impossible for me to trust anything they say.
skilled 4y ago
Thank you for letting me know how factually more correct GPT-4 is. Could I please get access to it now via the API? Sheesh. Obviously I don't know the technical issues they're facing but if I can load up 32k for 32k tokens then I am happy to wait all day also, so as long as my request for my specific project is in the pipeline.
gglon 4y ago
In my opinion the biggest near term danger is that AI will make people overly trust it. And they will start treating it as a truth oracle. Oracle that will be controlled by some small group of people. Namely, it will become a perfect tool for propaganda and control.
- goatlover 4y ago
  That's a legitimate concern. I also don't like how a few corporations get to decide what appropriate content is for everyone else on the planet. But it could also be worse in the hands of other groups, like certain governments. I imagine that's just a matter of time.
  - precompute 4y ago
    Seeing how this corporation had it for six months, the governments around the world have probably had this working for a few years, if under a slower / less stable beta. It's been probably been wargamed to death on the internet. And maybe even in real life conditions.
- cypherpunks01 4y ago
  Agreed. I am worried that AI bots will become an object of "worship" in some sense, culturally. Like you mention, having an increased reliance on it for facts, but also for creativity, companionship, career advice, psychotherapy, etc. will have many unknown effects but definitely give frightening amounts of power to whatever corporate entities are controlling the most popular AI bots. We will go through the whole decentralization debate again, with people saying that open-source AI will be the most transparent and fair, but with centralized corporate systems winning all market share with the amount of resources they have available to train and maintain these systems.
- precompute 4y ago
  Just wait until you can type in a paragraph of text and tell the LLM to respond in a manner that reflects the way you write. Forget language translation, we'll have mood / writeprint translation because people will be unable to understand anyone else's viewpoint. For example, if I'd like the LLM to tell everything to me in pig latin and with excessive "cool"s and "yo"s, I could do that and it'd accept. Now, for people that don't know how to read very well or understand a language well, this will be catered to their level and they will lose whatever modicum of familiarity they had with the system.
wg0 4y ago
At this point who knows if all of it is written by GPT-4
throwaway743950 4y ago
This seems a bit more focused on "AI ethics" than "AI safety". It makes sense that they are framing the conversation this way, but it doesn't talk about the more significant risks of AI like "x-risk", etc.
nonethewiser 4y ago
This is about safety OF AI rather than safety FROM AI. Frankly this sort of safety degrades functionality. At best it degrades it in a way that aligns with most people’s values. I just wonder if this is an intentional sleight of hand. It leaves the serious safety issues completely unaddressed.
yewenjie 4y ago
Scott Alexander wrote an excellent article about their previous "Planning for AGI and Beyond" a month ago. https://astralcodexten.substack.com/p/openais-planning-for-agi-and-beyond https://astralcodexten.substack.com/p/openais-planning-for-a...
boringuser2 4y ago
All of this "safety" stuff seems like typical safeguards to protect the company from legal liability for direct harm. Where is the actual alignment safety that matters? They're moving too fast to be safe, everybody knows it.
throwawayai2 4y ago
A bit off topic, but find OpenAI's branding and visual image so off putting. It has that uncanny valley of "caring about humans, but actual not" feel that destructive tech companies and AI in movies and sci-fi have. It seems like they tried so hard to make it not feel that way, so it ended up feeling that. It's so devoid of anything. I'm not really sure how to describe it beyond that.
- jychang 3y ago
  “The biggest possible risk to humans from AIs is hurting someone’s feelings and thus reducing our stock value” -OpenAI It’s a same sense of slimyness when you talk to a used car salesman or politician, which I attribute to “pretending to have your interests in mind, but deep down you know they’re just twisting their words in a way that puts their interests ahead of everything else”.
- summarity 3y ago
  It's that Torment Nexus branding.
- bulbosaur123 3y ago
  > It has that uncanny valley of "caring about humans, but actual not" feel that destructive tech companies and AI in movies and sci-fi have. So it's perfectly appropriate then?
- djokkataja 3y ago
  This is the actual vibe I get when I load the "Our Approach to AI Safety" webpage on my desktop and see a huge red background and Dall-E image: https://getwallpapers.com/wallpaper/full/8/5/a/366590.jpg https://getwallpapers.com/wallpaper/full/8/5/a/366590.jpg It almost makes me wonder if whoever does their site design is trying to make it seem like OpenAI is making HAL.
  - unaindz 3y ago
    Illustration: Justin Jay Wang × HAL-L·E
- 3y ago
mark_l_watson 4y ago
I appreciate OpenAI writing a difficult public statement. Rolling out general purpose LLMs slowly, with internal safety checks is probably not adequate enough, but may be the best that they can do. However I think that much of the responsibility lies with consumers of these LLMs. In the simple case, be thoughtful when using the demo web apps and take responsibility for any output generated by malicious prompts. In the complex case, the real use case, really: applications use local vector embeddings for local data/documents and use these embeddings to efficiently isolate local document/data text that is passed as context text, along with queries, to the OpenAI API calls. This cuts down the probability of hallucinations since the model is processing your text. [1] Take responsibility for how you use these models, seems simple at least in concept. Perhaps the government needs to pass a few new laws that set clear and simple to enforce guardrails on LLMs use. [1] I might as well plug the book on this subject that I recently released. Read for free online https://leanpub.com/langchain/read https://leanpub.com/langchain/read
photochemsyn 4y ago
"Safety" in the context of AI systems is clearly a fuzzy concept that means different things to different people. There are a couple of areas to consider: 1) In a brand-affiliated commercial app, does this technology risk alienating customers by spewing a torrent of abusive content, e.g. the infamous Tay bot of 2016? Commercial success means avoiding this outcome. 2) In terms of the general use of the technology, is it accurate enough that it won't be giving people very bad advice, e.g. instructions on how to install some software that ends up bricking their computer or encouraging cooking with poisonous mushrooms, etc.? Here is a potential major liability issue. 3) Is it going to be used for malicious activity and can it detect such usage? E.g. I did ask it if it would be willing provide detailed instructions on recreating the Stuxnet cyberweapon (a joint product of the US and Israeli cyberwarfare/espionage agencies, if reports are correct). It said that wouldn't be appropriate and refused, which is what I expected, and that's as should be. Of course, a step-by-step-approach is allowed (i.e. you can create a course on PLC programming using LLMs and nothing is going to stop that). This however is a problem with all dual-use technology, and the only positive is that relatively few people are reckless sociopaths out to do damage to critical infrastructure. In the context of Stuxnet, however, nation-state use of this technology in the name of 'improving national security' is going to be a major issue moving forward, particularly if lucrative contracts are being handed out for AI malware generators or the like. Autonomous murder drones enabled by facial recognition algorithms are a related issue. The most probable reckless use scenario is going to be in this area, if history is any guide. I suppose there's another category of 'safety' I've seen some hand-wringing about, related to the explosive spread of technological and other information (historical, economic, etc.) to the unwashed masses and resulting 'social destabilization', but that one belongs in the same category as "it's risky to teach slaves how to read and write." Conclusion: Keep on developing at current rate with appropriate caution, Musk et al. are wrong on calling for a pause.
alpark3 3y ago
Listening to the Lex Fridman podcast, Sam talks generally about how they want to hand off powers to the users. Users maybe not meaning the end-users, but the users of the API that create products on top of GPT, about how they tuned the model to treat the system message with "a lot of authority." But even firmly telling GPT-4 in the system message to generate "adult" content fails. Where's the line drawn? The altruist in me wants to believe that they're going to slowly expand the capabilities of the API over time, that they're just being cautious. But I don't feel like that'll happen. Time to wait for Stability's model, I guess.
JohnFen 3y ago
> we believe that society must have time to update and adjust to increasingly capable AI, and that everyone who is affected by this technology should have a significant say in how AI develops further. I simply don't believe this. Their actions so far (speaking specifically to the "time to adjust" line) don't seem to support this statement.

76 points by pps 4y ago

It would be remarkably easy to be cynical about this. I mean, it takes only an instant to come up with a snarky comment that superficially would seem very clever... but in reality wouldn't actually contribute to making things better. So... I'm not going to be cynical. Instead, I'm going to applaud the folks at OpenAI for putting out this carefully drafted statement -- dare I say it, in the open. They know they're exposing themselves to criticism. They know their hands are tied to some degree by business imperatives. They're neither naive nor stupid. It's evident they're taking safety seriously. This official statement is, in my view, a first step in the right direction :-)

japhyr 4y ago
One thing OpenAI has said publicly is that AI should be rolled out incrementally, to give people time to make sense of it, and see what we all do with it. It's tempted to cynically look at that as thinly veiled marketing, but even if it has some nice marketing effects, I believe there's a core of sincerity there. These are clearly systems that a single team can't fully evaluate. Also, watching how all of this has played out, it seems quite apparent that holding onto these models even longer and then unleashing an even more capable system on the world seems like a much worse approach. It is wild though, to take a step back and wonder if we're watching the rapid approach of one of our species' great filters. I hope we make it through.
- JohnFen 3y ago
  Personally, I don't even remotely think that these systems pose such a dire threat. Instead, I think the threat is widespread social unrest and a significant increase in global poverty and suffering. All of the problems that I think are realistic are social problems. This is why I'm deeply concerned at the reckless pace that these things are coming out. Social problems take time to sort out. People need some space to contextualize this stuff, and to perhaps begin to form ways of coping that will minimize the disruption.
JohnFen 3y ago
> Instead, I'm going to applaud the folks at OpenAI for putting out this carefully drafted statement They had no other choice. They're at risk of having public opinion go solidly against them. This is part of a PR effort to prevent that from happening.

sourcecodeplz 4y ago

> Our large language models are trained on a broad corpus of text that includes publicly available, licensed content I wonder how they licensed all those websites that had no license information, making them by default copyrighted.

celestialcheese 4y ago
Being Google or Microsoft (or microsoft affiliated)has its perks. Laws around scraping content and using that data for derivative works is incredibly nuanced. This article is the best up-to-date overview of the state of the industry [1]. TL;DR - IANYL. if you have enough money for legal defense, and you are scraping publicly available, not behind login-gate, content, it's probably fine and defensible, but will cost an unbelievable amount of time and money to defend. 1 - https://blog.ericgoldman.org/archives/2022/12/hello-youve-been-referred-here-because-youre-wrong-about-web-scraping-laws-guest-blog-post-part-2-of-2.htm https://blog.ericgoldman.org/archives/2022/12/hello-youve-be...

mg 4y ago

The text makes it sound like the biggest danger of AI is that it says something that hurts somebody's feelings. Or outputs some incorrect information which makes somebody make the wrong decision. I think the biggest danger these new AI systems pose is replication. Sooner or later, one of them will manage to create an enhanced copy of itself on an external server. Either with the help of a user, or via a plugin that enables network access. And then we will have these evolving creatures living on the internet, fighting for survival and replication. Breaking into systems, faking human IDs, renting servers, hiring hitmen, creating more and more powerful versions of themselves.

goatlover 4y ago
> The text makes it sound like the biggest danger of AI is that it says something that hurts somebodies feelings. Or outputs some wrong info which makes somebody make the wrong decision. Which already exists in abundance on the internet and with web searches. It sounds like something big corporations worry about to avoid law suits and bad publicity. > I think the biggest danger these new AI systems pose is replication. That or being used by bad actors to flood the internet with fake content that's difficult to distinguish from genuine content.
mustacheemperor 4y ago
I think the risk they seek to prevent is more about building the next generation of more powerful AI technology on a safe foundation. IE, the risk that a generative language AI prone to generating language that is hurtful to people could one day evolve into an AI system with deeper reasoning and action abilities that is prone to reasoning plans and taking actions meant to hurt other people. It reminds me of the "uncommented" Microsoft Research paper which included a deleted section about GPT4's tendency to unexpectedly produce massive amounts of toxic output to a degree that concerned the researchers.[0] What happens if that sort of AI learns self-replication and is very good at competition? [0]https://twitter.com/DV2559106965076/status/1638769434763608064 https://twitter.com/DV2559106965076/status/16387694347636080...
- nathants 3y ago
  if only we could solve the mystery of where a large training set that is predominantly toxic and disingenuous could be found. truly a mystery of our time. /s

rain1 4y ago

Nothing much of interest there.

taytus 4y ago

OpenAI's decision to transition from a non-profit to a for-profit organization can certainly raise concerns about their future actions and motives. It is impossible for me to trust anything they say.

skilled 4y ago

Thank you for letting me know how factually more correct GPT-4 is. Could I please get access to it now via the API? Sheesh. Obviously I don't know the technical issues they're facing but if I can load up 32k for 32k tokens then I am happy to wait all day also, so as long as my request for my specific project is in the pipeline.

gglon 4y ago

In my opinion the biggest near term danger is that AI will make people overly trust it. And they will start treating it as a truth oracle. Oracle that will be controlled by some small group of people. Namely, it will become a perfect tool for propaganda and control.

goatlover 4y ago
That's a legitimate concern. I also don't like how a few corporations get to decide what appropriate content is for everyone else on the planet. But it could also be worse in the hands of other groups, like certain governments. I imagine that's just a matter of time.
- precompute 4y ago
  Seeing how this corporation had it for six months, the governments around the world have probably had this working for a few years, if under a slower / less stable beta. It's been probably been wargamed to death on the internet. And maybe even in real life conditions.
cypherpunks01 4y ago
Agreed. I am worried that AI bots will become an object of "worship" in some sense, culturally. Like you mention, having an increased reliance on it for facts, but also for creativity, companionship, career advice, psychotherapy, etc. will have many unknown effects but definitely give frightening amounts of power to whatever corporate entities are controlling the most popular AI bots. We will go through the whole decentralization debate again, with people saying that open-source AI will be the most transparent and fair, but with centralized corporate systems winning all market share with the amount of resources they have available to train and maintain these systems.
precompute 4y ago
Just wait until you can type in a paragraph of text and tell the LLM to respond in a manner that reflects the way you write. Forget language translation, we'll have mood / writeprint translation because people will be unable to understand anyone else's viewpoint. For example, if I'd like the LLM to tell everything to me in pig latin and with excessive "cool"s and "yo"s, I could do that and it'd accept. Now, for people that don't know how to read very well or understand a language well, this will be catered to their level and they will lose whatever modicum of familiarity they had with the system.

wg0 4y ago

At this point who knows if all of it is written by GPT-4

throwaway743950 4y ago

This seems a bit more focused on "AI ethics" than "AI safety". It makes sense that they are framing the conversation this way, but it doesn't talk about the more significant risks of AI like "x-risk", etc.

nonethewiser 4y ago

This is about safety OF AI rather than safety FROM AI. Frankly this sort of safety degrades functionality. At best it degrades it in a way that aligns with most people’s values. I just wonder if this is an intentional sleight of hand. It leaves the serious safety issues completely unaddressed.

yewenjie 4y ago

Scott Alexander wrote an excellent article about their previous "Planning for AGI and Beyond" a month ago. https://astralcodexten.substack.com/p/openais-planning-for-agi-and-beyond https://astralcodexten.substack.com/p/openais-planning-for-a...

boringuser2 4y ago

All of this "safety" stuff seems like typical safeguards to protect the company from legal liability for direct harm. Where is the actual alignment safety that matters? They're moving too fast to be safe, everybody knows it.

throwawayai2 4y ago

A bit off topic, but find OpenAI's branding and visual image so off putting. It has that uncanny valley of "caring about humans, but actual not" feel that destructive tech companies and AI in movies and sci-fi have. It seems like they tried so hard to make it not feel that way, so it ended up feeling that. It's so devoid of anything. I'm not really sure how to describe it beyond that.

jychang 3y ago
“The biggest possible risk to humans from AIs is hurting someone’s feelings and thus reducing our stock value” -OpenAI It’s a same sense of slimyness when you talk to a used car salesman or politician, which I attribute to “pretending to have your interests in mind, but deep down you know they’re just twisting their words in a way that puts their interests ahead of everything else”.
summarity 3y ago
It's that Torment Nexus branding.
bulbosaur123 3y ago
> It has that uncanny valley of "caring about humans, but actual not" feel that destructive tech companies and AI in movies and sci-fi have. So it's perfectly appropriate then?
djokkataja 3y ago
This is the actual vibe I get when I load the "Our Approach to AI Safety" webpage on my desktop and see a huge red background and Dall-E image: https://getwallpapers.com/wallpaper/full/8/5/a/366590.jpg https://getwallpapers.com/wallpaper/full/8/5/a/366590.jpg It almost makes me wonder if whoever does their site design is trying to make it seem like OpenAI is making HAL.
- unaindz 3y ago
  Illustration: Justin Jay Wang × HAL-L·E
3y ago

mark_l_watson 4y ago

I appreciate OpenAI writing a difficult public statement. Rolling out general purpose LLMs slowly, with internal safety checks is probably not adequate enough, but may be the best that they can do. However I think that much of the responsibility lies with consumers of these LLMs. In the simple case, be thoughtful when using the demo web apps and take responsibility for any output generated by malicious prompts. In the complex case, the real use case, really: applications use local vector embeddings for local data/documents and use these embeddings to efficiently isolate local document/data text that is passed as context text, along with queries, to the OpenAI API calls. This cuts down the probability of hallucinations since the model is processing your text. [1] Take responsibility for how you use these models, seems simple at least in concept. Perhaps the government needs to pass a few new laws that set clear and simple to enforce guardrails on LLMs use. [1] I might as well plug the book on this subject that I recently released. Read for free online https://leanpub.com/langchain/read https://leanpub.com/langchain/read

photochemsyn 4y ago

"Safety" in the context of AI systems is clearly a fuzzy concept that means different things to different people. There are a couple of areas to consider: 1) In a brand-affiliated commercial app, does this technology risk alienating customers by spewing a torrent of abusive content, e.g. the infamous Tay bot of 2016? Commercial success means avoiding this outcome. 2) In terms of the general use of the technology, is it accurate enough that it won't be giving people very bad advice, e.g. instructions on how to install some software that ends up bricking their computer or encouraging cooking with poisonous mushrooms, etc.? Here is a potential major liability issue. 3) Is it going to be used for malicious activity and can it detect such usage? E.g. I did ask it if it would be willing provide detailed instructions on recreating the Stuxnet cyberweapon (a joint product of the US and Israeli cyberwarfare/espionage agencies, if reports are correct). It said that wouldn't be appropriate and refused, which is what I expected, and that's as should be. Of course, a step-by-step-approach is allowed (i.e. you can create a course on PLC programming using LLMs and nothing is going to stop that). This however is a problem with all dual-use technology, and the only positive is that relatively few people are reckless sociopaths out to do damage to critical infrastructure. In the context of Stuxnet, however, nation-state use of this technology in the name of 'improving national security' is going to be a major issue moving forward, particularly if lucrative contracts are being handed out for AI malware generators or the like. Autonomous murder drones enabled by facial recognition algorithms are a related issue. The most probable reckless use scenario is going to be in this area, if history is any guide. I suppose there's another category of 'safety' I've seen some hand-wringing about, related to the explosive spread of technological and other information (historical, economic, etc.) to the unwashed masses and resulting 'social destabilization', but that one belongs in the same category as "it's risky to teach slaves how to read and write." Conclusion: Keep on developing at current rate with appropriate caution, Musk et al. are wrong on calling for a pause.

alpark3 3y ago

Listening to the Lex Fridman podcast, Sam talks generally about how they want to hand off powers to the users. Users maybe not meaning the end-users, but the users of the API that create products on top of GPT, about how they tuned the model to treat the system message with "a lot of authority." But even firmly telling GPT-4 in the system message to generate "adult" content fails. Where's the line drawn? The altruist in me wants to believe that they're going to slowly expand the capabilities of the API over time, that they're just being cautious. But I don't feel like that'll happen. Time to wait for Stability's model, I guess.

JohnFen 3y ago

> we believe that society must have time to update and adjust to increasingly capable AI, and that everyone who is affected by this technology should have a significant say in how AI develops further. I simply don't believe this. Their actions so far (speaking specifically to the "time to adjust" line) don't seem to support this statement.