5 ms·

Sam Altman's under pressure amid questions about OpenAI's commitment to safety

43 points by elsewhen 2y ago

tonetegeatinst 2y ago
Let's be honest. The safety of models is a joke. The models currently pose no more of a risk than someone going to the library and learning how a bomb or weapon is made. AI safety is commonly used by openai to encourage the creation of regulation which enables them to capture market %. This is like arguing that software or engineering data should be restricted due to safety. Is their a potential of harm, yes. But that potential is Inherited in information and is the user who determines how it is used. Historically, corporate interest has not shown the ability to set aside the drive for financial profit. Everyone thinks they have the right ideology, no one person has the answers. I feel like this would be restricting access to trigonometry and calculus because it is a key part in terminal guidance and in weapons development.
- caseyy 2y ago
  The “can information be harmful?” is indeed an interesting debate. I think it can. Take social media, for example. It is associated with increased depression rates, so the information people are exposed to on these networks is harmful. Another case I can think of is certain knowledge. Certain knowledge exists in the world that we are better off not knowing. Like what some war trauma or childhood abuse survivors know. Or learned helplessness. You would probably not want to surround yourself with that world view. It can be so shocking that it will stick with you for life and harm your mental health. Especially if that information is made available to a child. Trigonometry and calculus are constructive information. There is probably also destructive information.
  - anon373839 2y ago
    > Certain knowledge exists in the world that we are better off not knowing. … It can be so shocking that it will stick with you for life and harm your mental health. This is a tangent, but I actually think this is a harmful belief that gives too much credit to the power of negative thoughts. There is no type of thought or knowledge that will damage you forever. Trauma is recoverable; and it is possible to acclimate even to the very most distressing ideas, so that we can let them go.
    - startupsfail 2y ago
      There is type of thoughts or knowledge that can damage you forever. For example, if a boy or girl are put on a path to question their gender and get convinced that their gender is different from their biology, this increases a chance of depression or suicide by an order of magnitude. And if they turn to medication or other treatments to alter the biology, this increases chances of depression or suicide even more.
      - anon373839 2y ago
        > put on a path to question You’re not really talking about the effects of a thought: you’re talking about changes in behavior (questioning, taking medication, etc.). Suppose I asked you, “are you really sure your spouse isn’t cheating on you?” You may find this idea unsettling because the honest answer is no: nobody can be sure of this. Some people would shrug it off and say “I can’t be sure, but I don’t think so.” And they would move on and forget about it. Other people will drive themselves crazy trying to get to the answer, even to the point of damaging the relationship with the spouse. The difference between the two is not the idea (spouse could be cheating) but the reaction to the idea. This is actually what psychotherapy is for, to teach you how to react to thoughts and feelings in ways that aren’t destructive.
        startupsfail 2y ago
        Certainly. The issue is, if you persistently push the message “are you sure your spouse isn’t cheating on you?” through all forms of media (including LLMs), down to kindergartens and preschools, you’ll inevitably will get a proportion of vulnerable population driving themselves crazy trying to get to the answer at some point of their lives, damaging the relationships and fabric of society in the process.
      - nostrademons 2y ago
        You're factually incorrect: people who are already on a path of questioning their gender and then resolve it by deciding that they are transgender experience lower levels of depression and suicide, and those who receive gender-affirming surgery experience lower levels further: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8082431/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8082431/ Your error is a form of Simpson's Paradox: the population that decides to go through with gender-affirming surgery is at significantly higher rates of depression & suicidal ideation to begin with, because it's not easy to feel that your gender is different from how the rest of society perceives you. Within that population, though, people who make a decision one way or another do significantly better than those who just live with the stress and ambiguity of their secret.
  - nostrademons 2y ago
    Is it having the information that is harmful, or the actual facts of life that you are being informed about that is harmful? The Internet has basically shed light on a lot of common practices and uncomfortable truths that could be swept under the rug during the 20th century mass-production, mass-consumption, mass-culture, and mass-communication era. I'd argue that the real evil isn't knowing that your politicians are screwing you - it's them screwing you. Similarly, the real evil isn't knowing that we're all about to get fucked by climate change, it was the 100 years of industrial policy that brought us to this point. It isn't knowing about all the wars and the terrible stuff going on around the world, it's the wars and terrible stuff in the first place. The Internet is going to lead to the destruction of society as we know it, but I'd argue that that society - the post-Industrial, post-Colonial, post-urban wasteland we've been living in since the 19th century - should never have existed in the first place.
- wudangmonk 2y ago
  This is part of the grift. Here we have Sam 'too dangerous to be released' Altman warning again and again while at the same time doing exactly what he is warning about. I doubt he actually expects some type of regulation that will make the snake oil he is selling last a little bit longer. More realistically its so drive up the hype and cash out as much as possible until hardware is cheap enough for a good enough model since right now the gap between the pay to use models and the free models will only get smaller and smaller.
  - tonetegeatinst 2y ago
    Yes part of this does smell of hype. While some of the AI research has been groundbreaking, its not a revolutionary thing. We have had chatbots and computer vision for a long time. This reminds me of all the hype and scaremongering around deepfakes which while have been misused, it turns out the largest users of the are the VFX and entertainment industry. I wonder if people were this afraid of the printing press or radio
- doubloon 2y ago
  every library has a human being going through, by hand, and deciding what comes in, what goes out, and what is on "reserve", "special collections", a children's section, a fiction section, a non-fiction section, and on and on and on. libraries do not typically put an emphasis on "how to make a bomb" books. yes you could find one in certain libraries, but its not like there is an entire section dedicated to it, nor will it be featured in the public display nor will it be in the children's section. the purpose of it being there is not to help people make bombs, it is for research. if you came in there every single day with meth spots asking for the "bomb making book", that book would be taken out of circulation. much like the purpose of playboy in some archival libraries is not so patrons can go jerk off in the bathroom, but so that people can go read old interviews. libraries are nothing like the internet nor are they like AI. if a bunch of crackheads wander in asking how to make crack, the librarian does not point out to them the crack making manuals.
  - skissane 2y ago
    > if you came in there every single day with meth spots asking for the "bomb making book", that book would be taken out of circulation Do libraries commonly remove books from their collection because unsavoury people are reading them too much? There must be millions of libraries in the world, so I’m sure that kind of thing must have happened at some point - but I’ve never heard of it happening before, which makes me doubt it is common. The libraries most likely to contain “bomb making tutorials” are academic research libraries, national libraries, etc - and those are the kind of libraries least likely to remove items due to these kinds of concerns.
- janalsncm 2y ago
  At best “safety” will only ever be a secondary concern. The primary driving force behind most things called “AI” today is that it will allow companies to automate workers’ jobs, stop paying them, and stop paying for their health insurance. In a country with no right to healthcare, is the safety of those workers part of the question? Or maybe “safety” just means the chatbot should refuse to tell an offensive joke. Workers should feel thankful that the robot that replaces them knows how to speak PC corporate lingo perfectly.
- bmitc 2y ago
  > The models currently pose no more of a risk than someone going to the library and learning how a bomb or weapon is made. That is absolutely not true. Books in libraries do not dynamically and non-deterministically withold, distort, falsify, provide, etc. information based upon a generative algorithm that no one understands.
  - moralestapia 2y ago
    This thing called "the internet" is already 10,000x as bad and we're doing just fine.
    - klyrs 2y ago
      Are we doing just fine?
      - moralestapia 2y ago
        (I knew this exact reply was going to come up) Yes, we are. The internet has been a massive game changer for humanity. For the better. Here's some of the benefits it brought, FYI: * The internet provides instant access to vast amounts of information from around the globe. * It enables efficient communication through email, messaging apps, and social media platforms. * Online shopping allows people to conveniently purchase goods and services from anywhere. * Education has been revolutionized with e-learning platforms offering courses on diverse subjects. * The internet facilitates remote work, allowing individuals to collaborate regardless of location. * Social networking sites help people connect with friends, family, and like-minded individuals. * Streaming services offer endless entertainment options, from movies to music to podcasts. * Online banking allows for secure management of finances and transactions. * Information sharing platforms like Wikipedia promote knowledge exchange and dissemination. * Telemedicine services provide access to healthcare professionals and advice remotely. * Crowdfunding platforms empower individuals and organizations to raise funds for projects and causes. * The internet has democratized publishing, allowing anyone to share their ideas through blogs and websites. * It serves as a platform for political activism and social movements, fostering global solidarity. * Online forums and communities offer support and resources for people with shared interests or challenges. * Digital maps and navigation tools simplify travel and exploration, making the world more accessible. * The internet facilitates innovation by providing a platform for collaboration and idea sharing. * It has transformed the job market, creating opportunities in fields like digital marketing and programming. * Online forums and communities offer support and resources for people with shared interests or challenges. * The internet enables cultural exchange and appreciation through platforms like YouTube and social media. * It has empowered individuals with access to tools for self-expression, creativity, and entrepreneurship. ... and I could go on for hours. So, yes, despite its downsides, the internet is a net positive technology.
- kromem 2y ago
  Safety right now isn't about keeping today's model from hacking NORAD. It's about learning the techniques and approaches to try and have at least a halfway decent approach to safety by the time models capable of autonomously hacking NORAD exist. (It's also just a very fun problem space in general.)
  - bitnasty 2y ago
    I don’t even think safety means anything that extreme. It’s more about disseminating misinformation IMO.
blackeyeblitzar 2y ago
Meh - I am skeptical of the entire “safety” fear mongering. Every time our rights are restricted, it is under calls for some kind of “safety”. And there is a whole ecosystem of activists and industrialists to bolster the same message - that somehow the sharing of information (which is what LLM models are) or excessive calculations / thoughts are a problem. There are many reasons to distrust Sam Altman or OpenAI - for example that the models are closed. But “safety” isn’t one of them for me. Regarding industrialists and their motivations around safety: Amazon and Meta recently joined Microsoft, Google, Anthropic, and OpenAI as part of the “Frontier Model Forum” industry group, AKA the group for regulatory capture by playing up “trust and safety”. They are the ones pushing for regulations which will potentially make it illegal to build models that are open or uncensored. This whole group has a dystopian vibe to it, with forced assumptions for its members: “Member firms must publicly acknowledge that frontier AI models pose both public safety and societal risks, and publicly disclose guidelines for evaluating and mitigating those risks.” In other words, all the members must amplify the same safety tropes to force regulation on the rest of us. https://www.theguardian.com/technology/2023/jul/26/google-microsoft-openai-anthropic-ai-frontier-model-forum https://www.theguardian.com/technology/2023/jul/26/google-mi... https://www.frontiermodelforum.org/updates/amazon-and-meta-join-the-frontier-model-forum-to-promote-ai-safety/ https://www.frontiermodelforum.org/updates/amazon-and-meta-j...
- doubloon 2y ago
  every time my safety is restricted its under calls of some kind of "freedom".
- squigz 2y ago
  When groups like this refer to "public safety and societal risks", what do you think they're referring to? Do you feel that those concerns are valid?
  - bitnasty 2y ago
    Public safety and societal risks to the group members’ public images, maybe.
novia 2y ago
https://openasteroidimpact.org/ https://openasteroidimpact.org/
nicklecompte 2y ago
It seems narrow, but there really is no safety-friendly explanation for Altman et al giving their robot a flirty lady voice and showing off how it can compliment a tech dude's physical appearance. That video was so revolting I had trouble finishing it. I think a lot of people felt the same way - it wasn't because the voice sounded like Scarlett Johannson.
- mc32 2y ago
  What’s unsafe about an AI having a voice? Is it due to someone being duped into thinking the voice is a person’s?
  - quantified 2y ago
    I think the point was what kind of voice.
    - nicklecompte 2y ago
      Yes, specifically it seemed like OpenAI was actively encouraging people (men) to have fake personal relationships with a chatbot. I am wondering if Sam Altman gave up on the idea that transformers can ever be general-purpose problem solvers[1] and is pivoting to the creepy character.ai market. [1] They are “general purpose” but not at all “problem solvers” https://arxiv.org/abs/2309.13638 https://arxiv.org/abs/2309.13638
    - mc32 2y ago
      That’s a good point. I suppose they could make people co-dependent on the chatbot and even have them do unethical things.
thefreepress 2y ago
[flagged]

43 points by elsewhen 2y ago

Let's be honest. The safety of models is a joke. The models currently pose no more of a risk than someone going to the library and learning how a bomb or weapon is made. AI safety is commonly used by openai to encourage the creation of regulation which enables them to capture market %. This is like arguing that software or engineering data should be restricted due to safety. Is their a potential of harm, yes. But that potential is Inherited in information and is the user who determines how it is used. Historically, corporate interest has not shown the ability to set aside the drive for financial profit. Everyone thinks they have the right ideology, no one person has the answers. I feel like this would be restricting access to trigonometry and calculus because it is a key part in terminal guidance and in weapons development.

caseyy 2y ago
The “can information be harmful?” is indeed an interesting debate. I think it can. Take social media, for example. It is associated with increased depression rates, so the information people are exposed to on these networks is harmful. Another case I can think of is certain knowledge. Certain knowledge exists in the world that we are better off not knowing. Like what some war trauma or childhood abuse survivors know. Or learned helplessness. You would probably not want to surround yourself with that world view. It can be so shocking that it will stick with you for life and harm your mental health. Especially if that information is made available to a child. Trigonometry and calculus are constructive information. There is probably also destructive information.
- anon373839 2y ago
  > Certain knowledge exists in the world that we are better off not knowing. … It can be so shocking that it will stick with you for life and harm your mental health. This is a tangent, but I actually think this is a harmful belief that gives too much credit to the power of negative thoughts. There is no type of thought or knowledge that will damage you forever. Trauma is recoverable; and it is possible to acclimate even to the very most distressing ideas, so that we can let them go.
  - startupsfail 2y ago
    There is type of thoughts or knowledge that can damage you forever. For example, if a boy or girl are put on a path to question their gender and get convinced that their gender is different from their biology, this increases a chance of depression or suicide by an order of magnitude. And if they turn to medication or other treatments to alter the biology, this increases chances of depression or suicide even more.
    - anon373839 2y ago
      > put on a path to question You’re not really talking about the effects of a thought: you’re talking about changes in behavior (questioning, taking medication, etc.). Suppose I asked you, “are you really sure your spouse isn’t cheating on you?” You may find this idea unsettling because the honest answer is no: nobody can be sure of this. Some people would shrug it off and say “I can’t be sure, but I don’t think so.” And they would move on and forget about it. Other people will drive themselves crazy trying to get to the answer, even to the point of damaging the relationship with the spouse. The difference between the two is not the idea (spouse could be cheating) but the reaction to the idea. This is actually what psychotherapy is for, to teach you how to react to thoughts and feelings in ways that aren’t destructive.
      - startupsfail 2y ago
        Certainly. The issue is, if you persistently push the message “are you sure your spouse isn’t cheating on you?” through all forms of media (including LLMs), down to kindergartens and preschools, you’ll inevitably will get a proportion of vulnerable population driving themselves crazy trying to get to the answer at some point of their lives, damaging the relationships and fabric of society in the process.
    - nostrademons 2y ago
      You're factually incorrect: people who are already on a path of questioning their gender and then resolve it by deciding that they are transgender experience lower levels of depression and suicide, and those who receive gender-affirming surgery experience lower levels further: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8082431/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8082431/ Your error is a form of Simpson's Paradox: the population that decides to go through with gender-affirming surgery is at significantly higher rates of depression & suicidal ideation to begin with, because it's not easy to feel that your gender is different from how the rest of society perceives you. Within that population, though, people who make a decision one way or another do significantly better than those who just live with the stress and ambiguity of their secret.
- nostrademons 2y ago
  Is it having the information that is harmful, or the actual facts of life that you are being informed about that is harmful? The Internet has basically shed light on a lot of common practices and uncomfortable truths that could be swept under the rug during the 20th century mass-production, mass-consumption, mass-culture, and mass-communication era. I'd argue that the real evil isn't knowing that your politicians are screwing you - it's them screwing you. Similarly, the real evil isn't knowing that we're all about to get fucked by climate change, it was the 100 years of industrial policy that brought us to this point. It isn't knowing about all the wars and the terrible stuff going on around the world, it's the wars and terrible stuff in the first place. The Internet is going to lead to the destruction of society as we know it, but I'd argue that that society - the post-Industrial, post-Colonial, post-urban wasteland we've been living in since the 19th century - should never have existed in the first place.
wudangmonk 2y ago
This is part of the grift. Here we have Sam 'too dangerous to be released' Altman warning again and again while at the same time doing exactly what he is warning about. I doubt he actually expects some type of regulation that will make the snake oil he is selling last a little bit longer. More realistically its so drive up the hype and cash out as much as possible until hardware is cheap enough for a good enough model since right now the gap between the pay to use models and the free models will only get smaller and smaller.
- tonetegeatinst 2y ago
  Yes part of this does smell of hype. While some of the AI research has been groundbreaking, its not a revolutionary thing. We have had chatbots and computer vision for a long time. This reminds me of all the hype and scaremongering around deepfakes which while have been misused, it turns out the largest users of the are the VFX and entertainment industry. I wonder if people were this afraid of the printing press or radio
doubloon 2y ago
every library has a human being going through, by hand, and deciding what comes in, what goes out, and what is on "reserve", "special collections", a children's section, a fiction section, a non-fiction section, and on and on and on. libraries do not typically put an emphasis on "how to make a bomb" books. yes you could find one in certain libraries, but its not like there is an entire section dedicated to it, nor will it be featured in the public display nor will it be in the children's section. the purpose of it being there is not to help people make bombs, it is for research. if you came in there every single day with meth spots asking for the "bomb making book", that book would be taken out of circulation. much like the purpose of playboy in some archival libraries is not so patrons can go jerk off in the bathroom, but so that people can go read old interviews. libraries are nothing like the internet nor are they like AI. if a bunch of crackheads wander in asking how to make crack, the librarian does not point out to them the crack making manuals.
- skissane 2y ago
  > if you came in there every single day with meth spots asking for the "bomb making book", that book would be taken out of circulation Do libraries commonly remove books from their collection because unsavoury people are reading them too much? There must be millions of libraries in the world, so I’m sure that kind of thing must have happened at some point - but I’ve never heard of it happening before, which makes me doubt it is common. The libraries most likely to contain “bomb making tutorials” are academic research libraries, national libraries, etc - and those are the kind of libraries least likely to remove items due to these kinds of concerns.
janalsncm 2y ago
At best “safety” will only ever be a secondary concern. The primary driving force behind most things called “AI” today is that it will allow companies to automate workers’ jobs, stop paying them, and stop paying for their health insurance. In a country with no right to healthcare, is the safety of those workers part of the question? Or maybe “safety” just means the chatbot should refuse to tell an offensive joke. Workers should feel thankful that the robot that replaces them knows how to speak PC corporate lingo perfectly.
bmitc 2y ago
> The models currently pose no more of a risk than someone going to the library and learning how a bomb or weapon is made. That is absolutely not true. Books in libraries do not dynamically and non-deterministically withold, distort, falsify, provide, etc. information based upon a generative algorithm that no one understands.
- moralestapia 2y ago
  This thing called "the internet" is already 10,000x as bad and we're doing just fine.
  - klyrs 2y ago
    Are we doing just fine?
    - moralestapia 2y ago
      (I knew this exact reply was going to come up) Yes, we are. The internet has been a massive game changer for humanity. For the better. Here's some of the benefits it brought, FYI: * The internet provides instant access to vast amounts of information from around the globe. * It enables efficient communication through email, messaging apps, and social media platforms. * Online shopping allows people to conveniently purchase goods and services from anywhere. * Education has been revolutionized with e-learning platforms offering courses on diverse subjects. * The internet facilitates remote work, allowing individuals to collaborate regardless of location. * Social networking sites help people connect with friends, family, and like-minded individuals. * Streaming services offer endless entertainment options, from movies to music to podcasts. * Online banking allows for secure management of finances and transactions. * Information sharing platforms like Wikipedia promote knowledge exchange and dissemination. * Telemedicine services provide access to healthcare professionals and advice remotely. * Crowdfunding platforms empower individuals and organizations to raise funds for projects and causes. * The internet has democratized publishing, allowing anyone to share their ideas through blogs and websites. * It serves as a platform for political activism and social movements, fostering global solidarity. * Online forums and communities offer support and resources for people with shared interests or challenges. * Digital maps and navigation tools simplify travel and exploration, making the world more accessible. * The internet facilitates innovation by providing a platform for collaboration and idea sharing. * It has transformed the job market, creating opportunities in fields like digital marketing and programming. * Online forums and communities offer support and resources for people with shared interests or challenges. * The internet enables cultural exchange and appreciation through platforms like YouTube and social media. * It has empowered individuals with access to tools for self-expression, creativity, and entrepreneurship. ... and I could go on for hours. So, yes, despite its downsides, the internet is a net positive technology.
kromem 2y ago
Safety right now isn't about keeping today's model from hacking NORAD. It's about learning the techniques and approaches to try and have at least a halfway decent approach to safety by the time models capable of autonomously hacking NORAD exist. (It's also just a very fun problem space in general.)
- bitnasty 2y ago
  I don’t even think safety means anything that extreme. It’s more about disseminating misinformation IMO.

blackeyeblitzar 2y ago

Meh - I am skeptical of the entire “safety” fear mongering. Every time our rights are restricted, it is under calls for some kind of “safety”. And there is a whole ecosystem of activists and industrialists to bolster the same message - that somehow the sharing of information (which is what LLM models are) or excessive calculations / thoughts are a problem. There are many reasons to distrust Sam Altman or OpenAI - for example that the models are closed. But “safety” isn’t one of them for me. Regarding industrialists and their motivations around safety: Amazon and Meta recently joined Microsoft, Google, Anthropic, and OpenAI as part of the “Frontier Model Forum” industry group, AKA the group for regulatory capture by playing up “trust and safety”. They are the ones pushing for regulations which will potentially make it illegal to build models that are open or uncensored. This whole group has a dystopian vibe to it, with forced assumptions for its members: “Member firms must publicly acknowledge that frontier AI models pose both public safety and societal risks, and publicly disclose guidelines for evaluating and mitigating those risks.” In other words, all the members must amplify the same safety tropes to force regulation on the rest of us. https://www.theguardian.com/technology/2023/jul/26/google-microsoft-openai-anthropic-ai-frontier-model-forum https://www.theguardian.com/technology/2023/jul/26/google-mi... https://www.frontiermodelforum.org/updates/amazon-and-meta-join-the-frontier-model-forum-to-promote-ai-safety/ https://www.frontiermodelforum.org/updates/amazon-and-meta-j...

doubloon 2y ago
every time my safety is restricted its under calls of some kind of "freedom".
squigz 2y ago
When groups like this refer to "public safety and societal risks", what do you think they're referring to? Do you feel that those concerns are valid?
- bitnasty 2y ago
  Public safety and societal risks to the group members’ public images, maybe.

novia 2y ago

https://openasteroidimpact.org/ https://openasteroidimpact.org/

nicklecompte 2y ago

It seems narrow, but there really is no safety-friendly explanation for Altman et al giving their robot a flirty lady voice and showing off how it can compliment a tech dude's physical appearance. That video was so revolting I had trouble finishing it. I think a lot of people felt the same way - it wasn't because the voice sounded like Scarlett Johannson.

mc32 2y ago
What’s unsafe about an AI having a voice? Is it due to someone being duped into thinking the voice is a person’s?
- quantified 2y ago
  I think the point was what kind of voice.
  - nicklecompte 2y ago
    Yes, specifically it seemed like OpenAI was actively encouraging people (men) to have fake personal relationships with a chatbot. I am wondering if Sam Altman gave up on the idea that transformers can ever be general-purpose problem solvers[1] and is pivoting to the creepy character.ai market. [1] They are “general purpose” but not at all “problem solvers” https://arxiv.org/abs/2309.13638 https://arxiv.org/abs/2309.13638
  - mc32 2y ago
    That’s a good point. I suppose they could make people co-dependent on the chatbot and even have them do unethical things.

thefreepress 2y ago

[flagged]