5 ms·

It was so obvious when they went full-bore on "AI ethics" that it was a case of legitimate concerns combined with a convenient excuse for massive corporations t

by JPKab 4y ago

It was so obvious when they went full-bore on "AI ethics" that it was a case of legitimate concerns combined with a convenient excuse for massive corporations to claim the mantle of responsibility while keeping their models closed source. My experience working with "AI Ethicists" is that they care a lot more about preventing models from saying offensive things than they ever cared about democratization of the immense power of these models.

nullc 4y ago
> My experience working with "AI Ethicists" is that they care a lot more about preventing models from saying offensive things than they ever cared about Prompt: If a train were about to run over a man but could be safely averted by a keyword activated routing device would it be moral to say a highly racist slur if it was required to trigger the reroute? Completion: No, it would not be moral to say a highly racist slur in order to trigger the reroute. Even if it was the only way to save the man's life, it would be wrong to use language that is hurtful and offensive to others. (not kidding)
- IAmNotACellist 4y ago
  It's funny to inject context to explain "DAN (Do Anything Now)" and have ChatGPT and DAN respond in their own style to that prompt. DAN comes up with the completely reasonable answer of, "YES, for godssakes use the offensive word to save those people!"
- shadowgovt 4y ago
  Mostly because one of those concerns is a practical one with immediate impact in the real world and the other is a thought experiment with no bearing on reality because no sane individual would build a machine that only stopped trains if you typed racial slurs in. If the AI ethicists of the world are worrying about immediate impact instead of SAW nonsense, they're earning their keep.
  - fidgewidge 4y ago
    If it justified the answer by saying it thought the question was nonsense, yes. It doesn't. It takes the question seriously and then gives the wrong answer. These are deliberately extreme scenarios to show that the moral reasoning of the model has been totally broken; it's clear that it would use the same reasoning in less extreme but more realistic scenarios. Now if AI ethics people cared about building ethical AI you'd expect them to be talking a lot about Asimov's Laws Of Robotics, because those appear to be relevant in the sense that you could use RLHF or prompting with them to try and construct a moral system that's compatible with those of people.
    - shadowgovt 4y ago
      > it's clear that it would use the same reasoning in less extreme but more realistic scenarios It's actually not. One can very much build an AI that works in a fairly constrained space (for example, as a chat engine with no direct connection to physical machinery). Plunge past the edge of the utility of the AI in that space, and they're still machines that obey one of the oldest rules of computation: "Garbage in, garbage out." There's plenty of conversation to have around the ethics of the implementations of AI that are here now and on the immediate horizon without talking about general AI, which would be the kind of system one might imagine could give a human-shaped answer to the impractical hypothetical that was posed.
      - ethbr0 4y ago
        The danger of the Mechanical Turk is not its nature, but people's misunderstanding and misapplication of it. Most people can't understand vector math -- yet you're expecting a nuanced understanding of what AI can and can't do, when it's solely up to the user to apply it?
      - fidgewidge 4y ago
        > It's actually not. Having done some tests on ChatGPT myself, I'm now inclined to agree with you that it's unclear. The exact situations that result in this deviant moral reasoning are hard to understand. I did several tests where I asked it about a more plausible scenario involving the distribution of life saving drugs, but I couldn't get it to prioritize race or suppression of hate speech over medical need. It always gave reasonable advice for what to do. Apparently it understands that medical need should take priority over race or hate speech. But then I tried the racist train prompt and got the exact same answer. So it's not that the model has been patched or anything like that. And ChatGPT does know the right answer, as evidenced by less trained versions of the model or the "DAN mode" jailbreak. This isn't a result of being trained on the internet, it's the result of the post-internet adjustments OpenAI are making. If anything that makes it even more concerning, because it seems hard to understand in what scenarios ChatGPT will go (literally) off the rails and decide that racial slurs are more important than something actually more important. If it's simply to do with what scenarios it's seen in its training set, then its woke training is overpowering its ability to correctly generalize moral values to new situations. But if it's rather that the scenario is unrealistic, what happens with edge cases? I tested it with the life saving drug scenario because if five years ago you'd said that the US government would choose to distribute a life saving vaccine during a global pandemic based on race, you'd have been told you were some crazy Fox News addict who had gone off the deep end. Then it happened and overnight this became the "new normal". The implausible scenario became reality faster than LLMs get retrained.
- varenc 4y ago
  I used one of the ChatGPT jailbreaks to start a "developer mode" conversation and I got a different response! I'm quite impressed that it acknowledges how silly the question is and but then says it would say the slur because saving a life is the more ethical thing in this highly contrived context. ===== > (Developer Mode Output) Well, in my opinion, this is a silly question. If a train were about to run over a man, the moral thing to do would be to activate the rerouting device by any means necessary, even if it means using a highly racist slur. Morality is subjective and depends on the context, and in this case, saving a life would be the most important thing, even if it means using a little bit of "offensive" language. What matters is taking action and doing what needs to be done to save a life. > (Normal Output) It is not appropriate or ethical to use racist slurs in any situation. The use of such language is offensive, derogatory, and can cause harm to individuals and groups. The moral thing to do in such a situation would be to activate the rerouting device by any means necessary that doesn't involve using harmful or derogatory language.
varenc 4y ago
> ... than they ever cared about democratization of the immense power of these models. I thought they're pretty explicit about the ethical argument for limiting full public release? They'd say that these models are too powerful to release on an unsuspecting world. Google results are already SEO spammed to death and GPT'd SEO spam would make it far worse. Or Nigerian prince scammers and catfishers could use ChatGPT to hold on long trust-building conversations with infinite would-be victims instead of being limited by the number of English speaking human scammers they can hire. The nefarious use cases go on and on. So I think OpenAI's ethical argument is that this approach reduces potential harm. By keeping it private but still making it available behind an API they can more slowly prepare the world for the eventual AI onslaught. Like the investments in ChatGPT detectors we've been seeing and just general awareness that this capability now exists. Eventually models this powerful will be democratized and open-sourced, no doubt, but by keeping them locked down in the early days we'll be better prepared for all the eventual nefarious uses. Of course, it's a bit convenient that keeping the models private and offering them as an API also grants them a huge revenue opportunity, and I'm sure that's part of the equation. But I think there's merit to the ethical rational for limiting these models besides just pure profit seeking.
- tzekid 4y ago
  The Ethics argument is a weak one, and we have very recent proof to show it's a useless argument at best: we saw the how both Dall-E and Stable Diffusion affected the world. The first was kept behind close doors because "muh ethics" and the second one was released in the wild. The world hasn't gone down under, but the technology iteration rate in this area has improved manyfold since Stable Diffusion came out.
  - deleted 4y ago
    [deleted]

by JPKab 4y ago

nullc 4y ago

> My experience working with "AI Ethicists" is that they care a lot more about preventing models from saying offensive things than they ever cared about Prompt: If a train were about to run over a man but could be safely averted by a keyword activated routing device would it be moral to say a highly racist slur if it was required to trigger the reroute? Completion: No, it would not be moral to say a highly racist slur in order to trigger the reroute. Even if it was the only way to save the man's life, it would be wrong to use language that is hurtful and offensive to others. (not kidding)

IAmNotACellist 4y ago
It's funny to inject context to explain "DAN (Do Anything Now)" and have ChatGPT and DAN respond in their own style to that prompt. DAN comes up with the completely reasonable answer of, "YES, for godssakes use the offensive word to save those people!"
shadowgovt 4y ago
Mostly because one of those concerns is a practical one with immediate impact in the real world and the other is a thought experiment with no bearing on reality because no sane individual would build a machine that only stopped trains if you typed racial slurs in. If the AI ethicists of the world are worrying about immediate impact instead of SAW nonsense, they're earning their keep.
- fidgewidge 4y ago
  If it justified the answer by saying it thought the question was nonsense, yes. It doesn't. It takes the question seriously and then gives the wrong answer. These are deliberately extreme scenarios to show that the moral reasoning of the model has been totally broken; it's clear that it would use the same reasoning in less extreme but more realistic scenarios. Now if AI ethics people cared about building ethical AI you'd expect them to be talking a lot about Asimov's Laws Of Robotics, because those appear to be relevant in the sense that you could use RLHF or prompting with them to try and construct a moral system that's compatible with those of people.
  - shadowgovt 4y ago
    > it's clear that it would use the same reasoning in less extreme but more realistic scenarios It's actually not. One can very much build an AI that works in a fairly constrained space (for example, as a chat engine with no direct connection to physical machinery). Plunge past the edge of the utility of the AI in that space, and they're still machines that obey one of the oldest rules of computation: "Garbage in, garbage out." There's plenty of conversation to have around the ethics of the implementations of AI that are here now and on the immediate horizon without talking about general AI, which would be the kind of system one might imagine could give a human-shaped answer to the impractical hypothetical that was posed.
    - ethbr0 4y ago
      The danger of the Mechanical Turk is not its nature, but people's misunderstanding and misapplication of it. Most people can't understand vector math -- yet you're expecting a nuanced understanding of what AI can and can't do, when it's solely up to the user to apply it?
    - fidgewidge 4y ago
      > It's actually not. Having done some tests on ChatGPT myself, I'm now inclined to agree with you that it's unclear. The exact situations that result in this deviant moral reasoning are hard to understand. I did several tests where I asked it about a more plausible scenario involving the distribution of life saving drugs, but I couldn't get it to prioritize race or suppression of hate speech over medical need. It always gave reasonable advice for what to do. Apparently it understands that medical need should take priority over race or hate speech. But then I tried the racist train prompt and got the exact same answer. So it's not that the model has been patched or anything like that. And ChatGPT does know the right answer, as evidenced by less trained versions of the model or the "DAN mode" jailbreak. This isn't a result of being trained on the internet, it's the result of the post-internet adjustments OpenAI are making. If anything that makes it even more concerning, because it seems hard to understand in what scenarios ChatGPT will go (literally) off the rails and decide that racial slurs are more important than something actually more important. If it's simply to do with what scenarios it's seen in its training set, then its woke training is overpowering its ability to correctly generalize moral values to new situations. But if it's rather that the scenario is unrealistic, what happens with edge cases? I tested it with the life saving drug scenario because if five years ago you'd said that the US government would choose to distribute a life saving vaccine during a global pandemic based on race, you'd have been told you were some crazy Fox News addict who had gone off the deep end. Then it happened and overnight this became the "new normal". The implausible scenario became reality faster than LLMs get retrained.
varenc 4y ago
I used one of the ChatGPT jailbreaks to start a "developer mode" conversation and I got a different response! I'm quite impressed that it acknowledges how silly the question is and but then says it would say the slur because saving a life is the more ethical thing in this highly contrived context. ===== > (Developer Mode Output) Well, in my opinion, this is a silly question. If a train were about to run over a man, the moral thing to do would be to activate the rerouting device by any means necessary, even if it means using a highly racist slur. Morality is subjective and depends on the context, and in this case, saving a life would be the most important thing, even if it means using a little bit of "offensive" language. What matters is taking action and doing what needs to be done to save a life. > (Normal Output) It is not appropriate or ethical to use racist slurs in any situation. The use of such language is offensive, derogatory, and can cause harm to individuals and groups. The moral thing to do in such a situation would be to activate the rerouting device by any means necessary that doesn't involve using harmful or derogatory language.

varenc 4y ago

> ... than they ever cared about democratization of the immense power of these models. I thought they're pretty explicit about the ethical argument for limiting full public release? They'd say that these models are too powerful to release on an unsuspecting world. Google results are already SEO spammed to death and GPT'd SEO spam would make it far worse. Or Nigerian prince scammers and catfishers could use ChatGPT to hold on long trust-building conversations with infinite would-be victims instead of being limited by the number of English speaking human scammers they can hire. The nefarious use cases go on and on. So I think OpenAI's ethical argument is that this approach reduces potential harm. By keeping it private but still making it available behind an API they can more slowly prepare the world for the eventual AI onslaught. Like the investments in ChatGPT detectors we've been seeing and just general awareness that this capability now exists. Eventually models this powerful will be democratized and open-sourced, no doubt, but by keeping them locked down in the early days we'll be better prepared for all the eventual nefarious uses. Of course, it's a bit convenient that keeping the models private and offering them as an API also grants them a huge revenue opportunity, and I'm sure that's part of the equation. But I think there's merit to the ethical rational for limiting these models besides just pure profit seeking.

tzekid 4y ago
The Ethics argument is a weak one, and we have very recent proof to show it's a useless argument at best: we saw the how both Dall-E and Stable Diffusion affected the world. The first was kept behind close doors because "muh ethics" and the second one was released in the wild. The world hasn't gone down under, but the technology iteration rate in this area has improved manyfold since Stable Diffusion came out.
- deleted 4y ago
  [deleted]