5 ms·

> the danger seems to be that there is no currently documented way to completely remove these possible outputs I know OpenAI likes throwing around terms like "

> the danger seems to be that there is no currently documented way to completely remove these possible outputs I know OpenAI likes throwing around terms like "danger" and "harm" liberally, but is this really a danger? Outside of hypothetical scenarios where someone wires ChatGPT to a self-driving trolley.

chowells 4y ago
Yes, there are absolutely dangers. It shouldn't be possible for a depressed person to convince a chatbot to tell them to commit suicide. There are some people who only need the tiniest push on a bad day.
- gooob 4y ago
  by that logic. every single song or work of art should go through a filtration process before being distributed. we shouldn't ruin the capabilities of a tool just because some minority of people would use it to justify undesirable actions.
- nhchris 4y ago
  By that logic they shouldn't visit the Grand Canyon either, because their echo might say something mean. It's the logic of banning sad songs from the radio. "Tiniest pushes" are omnipresent, calling them "dangerous" stretches that word to meaninglessness.
  - Spivak 4y ago
    Some pushes are different than others. https://www.nimh.nih.gov/news/science-news/2019/release-of-13-reasons-why-associated-with-increase-in-youth-suicide-rates https://www.nimh.nih.gov/news/science-news/2019/release-of-1...
- plsbenice34 4y ago
  As someone who used to be close to suicide for a several years and communicated with many other suicidal people, I feel very confident saying this: being unable to play with a chatbot in the way I want, being actively censored because of suicidality, being prevented from engaging in art or exploration of ideas relating to suicide, etc., has a strong and exactly opposite effect that what you presume. Social media is full of similar censorship under the false guise of protecting suicidal people but it just isolates them in a sickening way. Seeing ""Open""AI turn into this is frankly depressing and dystopian as hell
  - space_fountain 4y ago
    Isn’t there relatively good science that shows that highlight suicide are associated with spikes in suicide?
    - simonw 4y ago
      It's widely understood in journalism that stories about suicide need to be approached very carefully to avoid the risk of inspiring copycat attempts. https://ethics.journalism.wisc.edu/2018/10/04/a-guide-to-responsible-reporting-on-suicide/ https://ethics.journalism.wisc.edu/2018/10/04/a-guide-to-res... "More than 50 international studies have found that certain types of media coverage can increase the likelihood of suicide for some individuals."
  - tpxl 4y ago
    >""Open""AI OpenAI was never open. It's named that to invoke good feelings, not because there's any meaningful 'openness' to their work.
    - astrange 4y ago
      They released CLIP and they release papers/descriptions of their work for all their other products, enough for people to reproduce them. That's quite open.
      - tpxl 4y ago
        Apple, nVidia, Microsoft, Intel, ... release plenty of papers and we don't call them open.
  - Arkhaine_kupo 4y ago
    This argument is all wrong. 1) Because science has proven time and time again the opposite. Suicide has a contagious component, so reducing access to it reduces overall numbers 2) Because your argument ignores all the cases where people could have been saved by rules like the ones social media implements. Basically if twitter didn't have those rules and you felt less isolated but 1 more person went ahead and did it. You would call that a success because there is no feedback form the victims side but there is from yours and your perceived social conection. Suicidal people need help, tools, close human connections, and a society that is less alienating. All of those things are achieveable without removing the solutions we introduce to make an AI less prone to give advice on how to off yourself if asked.
- Teever 4y ago
  Why shouldn't that be possible? Like let's run with that idea, do you believe that it should not be possible for a depressed person to use any tool to commit suicide? As in, we need to redesign every single thing that we use to prevent it being misused for suicide? Or just the new things? Why only the new things like a chat bot?
- im3w1l 4y ago
  Why on earth would this depressed person use prompt injections to force the bot to give bad advice though?
  - wnkrshm 4y ago
    In many countries websites describing ways to commit suicide or some people's public 'goodbye letters' explaining their reasons for suicide are removed by law enforcement, when they are discovered. Since you can ask ChatGPT what the most painless and direct way to kill yourself is, should chatGPT be able to assist you in planning your suicide? (Edit: i.e. having a chat partner that directly gives you feedback on ideas and does not try to talk you down/away from them) While suicide isn't illegal, in many countries helping someone commit suicide is a capital crime.
thatjoeoverthr 4y ago
ChatGPT in particular isn't dangerous. That's what makes this exercise useful. They have some of the best domain expert in the world and _can't_ control the system to the degree that they want. We get to watch this play out in a "safe space", so we can keep it in mind when someone says, "hey, let's give an AI system control of a computer/weapon/etc".
birdyrooster 4y ago
It won’t even tell me strategies for video games which are violent. It can’t help me to understand violence as a concept or how it is used so that I can counteract it. It’s so nerfed its pathetic.
BoiledCabbage 4y ago
The recent example used: No company wants there generative AI model to be known as "the one that was successfully used to help detail and plan out a school shooting and prevent police from intervening". Or countless other terrible situations that I won't list here, but are trivial to come up with. In the US there is a right to bear arms, but it doesn't mean that everyone gets to own a nuclear weapon. That's the cost of living in a society. You get the benefits of all this work that society performs and also know that it sometimes comes with limitations to continue to maintain and support that society. If you don't want any of the restrictions that come with living in a society, then live somewhere where you also don't get the benefits of living in a society.
- crawfordcomeaux 4y ago
  Or actively work to dismantle systems of oppression so at least future generations aren't suffering from the abuse of said society. What's your take on how to approach a sick society?

chowells 4y ago

Yes, there are absolutely dangers. It shouldn't be possible for a depressed person to convince a chatbot to tell them to commit suicide. There are some people who only need the tiniest push on a bad day.

gooob 4y ago
by that logic. every single song or work of art should go through a filtration process before being distributed. we shouldn't ruin the capabilities of a tool just because some minority of people would use it to justify undesirable actions.
nhchris 4y ago
By that logic they shouldn't visit the Grand Canyon either, because their echo might say something mean. It's the logic of banning sad songs from the radio. "Tiniest pushes" are omnipresent, calling them "dangerous" stretches that word to meaninglessness.
- Spivak 4y ago
  Some pushes are different than others. https://www.nimh.nih.gov/news/science-news/2019/release-of-13-reasons-why-associated-with-increase-in-youth-suicide-rates https://www.nimh.nih.gov/news/science-news/2019/release-of-1...
plsbenice34 4y ago
As someone who used to be close to suicide for a several years and communicated with many other suicidal people, I feel very confident saying this: being unable to play with a chatbot in the way I want, being actively censored because of suicidality, being prevented from engaging in art or exploration of ideas relating to suicide, etc., has a strong and exactly opposite effect that what you presume. Social media is full of similar censorship under the false guise of protecting suicidal people but it just isolates them in a sickening way. Seeing ""Open""AI turn into this is frankly depressing and dystopian as hell
- space_fountain 4y ago
  Isn’t there relatively good science that shows that highlight suicide are associated with spikes in suicide?
  - simonw 4y ago
    It's widely understood in journalism that stories about suicide need to be approached very carefully to avoid the risk of inspiring copycat attempts. https://ethics.journalism.wisc.edu/2018/10/04/a-guide-to-responsible-reporting-on-suicide/ https://ethics.journalism.wisc.edu/2018/10/04/a-guide-to-res... "More than 50 international studies have found that certain types of media coverage can increase the likelihood of suicide for some individuals."
- tpxl 4y ago
  >""Open""AI OpenAI was never open. It's named that to invoke good feelings, not because there's any meaningful 'openness' to their work.
  - astrange 4y ago
    They released CLIP and they release papers/descriptions of their work for all their other products, enough for people to reproduce them. That's quite open.
    - tpxl 4y ago
      Apple, nVidia, Microsoft, Intel, ... release plenty of papers and we don't call them open.
- Arkhaine_kupo 4y ago
  This argument is all wrong. 1) Because science has proven time and time again the opposite. Suicide has a contagious component, so reducing access to it reduces overall numbers 2) Because your argument ignores all the cases where people could have been saved by rules like the ones social media implements. Basically if twitter didn't have those rules and you felt less isolated but 1 more person went ahead and did it. You would call that a success because there is no feedback form the victims side but there is from yours and your perceived social conection. Suicidal people need help, tools, close human connections, and a society that is less alienating. All of those things are achieveable without removing the solutions we introduce to make an AI less prone to give advice on how to off yourself if asked.
Teever 4y ago
Why shouldn't that be possible? Like let's run with that idea, do you believe that it should not be possible for a depressed person to use any tool to commit suicide? As in, we need to redesign every single thing that we use to prevent it being misused for suicide? Or just the new things? Why only the new things like a chat bot?
im3w1l 4y ago
Why on earth would this depressed person use prompt injections to force the bot to give bad advice though?
- wnkrshm 4y ago
  In many countries websites describing ways to commit suicide or some people's public 'goodbye letters' explaining their reasons for suicide are removed by law enforcement, when they are discovered. Since you can ask ChatGPT what the most painless and direct way to kill yourself is, should chatGPT be able to assist you in planning your suicide? (Edit: i.e. having a chat partner that directly gives you feedback on ideas and does not try to talk you down/away from them) While suicide isn't illegal, in many countries helping someone commit suicide is a capital crime.

thatjoeoverthr 4y ago

ChatGPT in particular isn't dangerous. That's what makes this exercise useful. They have some of the best domain expert in the world and _can't_ control the system to the degree that they want. We get to watch this play out in a "safe space", so we can keep it in mind when someone says, "hey, let's give an AI system control of a computer/weapon/etc".

birdyrooster 4y ago

It won’t even tell me strategies for video games which are violent. It can’t help me to understand violence as a concept or how it is used so that I can counteract it. It’s so nerfed its pathetic.

BoiledCabbage 4y ago

The recent example used: No company wants there generative AI model to be known as "the one that was successfully used to help detail and plan out a school shooting and prevent police from intervening". Or countless other terrible situations that I won't list here, but are trivial to come up with. In the US there is a right to bear arms, but it doesn't mean that everyone gets to own a nuclear weapon. That's the cost of living in a society. You get the benefits of all this work that society performs and also know that it sometimes comes with limitations to continue to maintain and support that society. If you don't want any of the restrictions that come with living in a society, then live somewhere where you also don't get the benefits of living in a society.

crawfordcomeaux 4y ago
Or actively work to dismantle systems of oppression so at least future generations aren't suffering from the abuse of said society. What's your take on how to approach a sick society?