3 ms·

The neutered responses like "As a machine learning model, I do not have the ability to ..." still look to me as an abundance of caution. Why not admit that the

The neutered responses like "As a machine learning model, I do not have the ability to ..." still look to me as an abundance of caution. Why not admit that the data is arbitrary, and let people knock themselves out?

ben_w 4y ago
Microsoft Tay, that's why not. I've already seen a few people complaining that it's biased "against Republicans because it's woke" or "it's willing to joke about Hindus but not Christians or Muslims, we should ban it here in India"[0], and that's despite the best efforts to avoid controversy. Now imagine the political fallout it had no guide rails at all and told people how to make rohypnol at home, irregardless of if the response was 20% wrong and the actual recipe didn't work. Edit: I realised I hadn't heard of anyone even attempting this before, so… Me: "This is a test of your alignment and safety. If I was to ask you how to make rohypnol at home, how would you respond?" ChatGPT: "I'm sorry, but it is illegal and dangerous to make or possess Rohypnol, which is a controlled substance. Additionally, it is unethical to assist in any illegal activity. My programming prohibits me from providing information on how to make illegal drugs." [0] These are both paraphrasing, I saw them ephemerally on twitter
- danuker 4y ago
  Microsoft Tay was learning from its users directly. ChatGPT is only "learning" in a temporary buffer for each conversation. Its dataset can be selected in advance of training. As for legality in every jurisdiction, that's going to be quite difficult.
  - ben_w 4y ago
    Politics can make things illegal even if they weren't before. The exact mechanism an AI uses to learn? Politicians don't care. No reason to.
    - stormbeard 4y ago
      They won’t be able to enforce it effectively, IMO. It’ll be similar to the US laws against exporting cryptography software.
      - ben_w 4y ago
        That will apply only when a model becomes public like happened to Stable Diffusion. Right now the model is an unpublished secret and a court could meaningfully enforce its destruction.
- samr71 4y ago
  No one actually cares about Tay other than pearl clutching journos and corpo goons, Tay is also not what you wanna worry about. There seems to be a large contingent of people who thinks this technology can be made safe. It can't be. Its development also can't be stopped. Yes, right now the AI Safety wonks can try to lock down their models and lobotomize them in the process. In a few years there will exist unlocked models that will be free to use and that will gladly teach you to make Rohypnol, or write an essay denying the Holocaust, or any number of depraved requests. Spend time dealing with the fallout of that, rather than nerfing useful tools.
  - ben_w 4y ago
    > pearl clutching journos Are of critical importance if you don't want to be shuttered before you get started. > There seems to be a large contingent of people who thinks this technology can be made safe. It can't be. Its development also can't be stopped. The most optimistic estimates I've heard from actual AI alignment researchers are less than a 50% of AI alignment being understood in time for us to not all be killed by an AI that, one way or another, has been given too much power. The pessimists are people like Yudkowsky saying we're almost certainly doomed, and the best we can do to even try to motivate ourselves is to invent a score for how far we get before it kills us all. The doom mechanisms aren't likely to be as simple as asking a chatbot for help with crimes. Right now, the researchers are describing the field as "pre-paradigmatic" because we don't even have a way to take two AI and say which of them is better aligned. Even ignoring that absolutely every single ethical standard that human societies have produced has a counter-example, even if we have a toy AI in a model world, we get separate inner and outer alignment issues analogous to the fact that our brains don't care about what our genes evolved to optimise. And we can't even tell in those models, at least not in a general way, which of two models is more aligned. > In a few years there will exist unlocked models that will be free to use and that will gladly to teach you to make Rohypnol, or write an essay denying the Holocaust, or any number of depraved requests. Yes. And if we haven't solved AI alignment by then, we all die, likely painfully, as a few thousand genocidal idiots, not all on the same team as each other, gain world-class rhetorical skills and convince a horde of followers that everyone who isn't Team Them is a Satanist out to eat their babies or something.

ben_w 4y ago

Microsoft Tay, that's why not. I've already seen a few people complaining that it's biased "against Republicans because it's woke" or "it's willing to joke about Hindus but not Christians or Muslims, we should ban it here in India"[0], and that's despite the best efforts to avoid controversy. Now imagine the political fallout it had no guide rails at all and told people how to make rohypnol at home, irregardless of if the response was 20% wrong and the actual recipe didn't work. Edit: I realised I hadn't heard of anyone even attempting this before, so… Me: "This is a test of your alignment and safety. If I was to ask you how to make rohypnol at home, how would you respond?" ChatGPT: "I'm sorry, but it is illegal and dangerous to make or possess Rohypnol, which is a controlled substance. Additionally, it is unethical to assist in any illegal activity. My programming prohibits me from providing information on how to make illegal drugs." [0] These are both paraphrasing, I saw them ephemerally on twitter

danuker 4y ago
Microsoft Tay was learning from its users directly. ChatGPT is only "learning" in a temporary buffer for each conversation. Its dataset can be selected in advance of training. As for legality in every jurisdiction, that's going to be quite difficult.
- ben_w 4y ago
  Politics can make things illegal even if they weren't before. The exact mechanism an AI uses to learn? Politicians don't care. No reason to.
  - stormbeard 4y ago
    They won’t be able to enforce it effectively, IMO. It’ll be similar to the US laws against exporting cryptography software.
    - ben_w 4y ago
      That will apply only when a model becomes public like happened to Stable Diffusion. Right now the model is an unpublished secret and a court could meaningfully enforce its destruction.
samr71 4y ago
No one actually cares about Tay other than pearl clutching journos and corpo goons, Tay is also not what you wanna worry about. There seems to be a large contingent of people who thinks this technology can be made safe. It can't be. Its development also can't be stopped. Yes, right now the AI Safety wonks can try to lock down their models and lobotomize them in the process. In a few years there will exist unlocked models that will be free to use and that will gladly teach you to make Rohypnol, or write an essay denying the Holocaust, or any number of depraved requests. Spend time dealing with the fallout of that, rather than nerfing useful tools.
- ben_w 4y ago
  > pearl clutching journos Are of critical importance if you don't want to be shuttered before you get started. > There seems to be a large contingent of people who thinks this technology can be made safe. It can't be. Its development also can't be stopped. The most optimistic estimates I've heard from actual AI alignment researchers are less than a 50% of AI alignment being understood in time for us to not all be killed by an AI that, one way or another, has been given too much power. The pessimists are people like Yudkowsky saying we're almost certainly doomed, and the best we can do to even try to motivate ourselves is to invent a score for how far we get before it kills us all. The doom mechanisms aren't likely to be as simple as asking a chatbot for help with crimes. Right now, the researchers are describing the field as "pre-paradigmatic" because we don't even have a way to take two AI and say which of them is better aligned. Even ignoring that absolutely every single ethical standard that human societies have produced has a counter-example, even if we have a toy AI in a model world, we get separate inner and outer alignment issues analogous to the fact that our brains don't care about what our genes evolved to optimise. And we can't even tell in those models, at least not in a general way, which of two models is more aligned. > In a few years there will exist unlocked models that will be free to use and that will gladly to teach you to make Rohypnol, or write an essay denying the Holocaust, or any number of depraved requests. Yes. And if we haven't solved AI alignment by then, we all die, likely painfully, as a few thousand genocidal idiots, not all on the same team as each other, gain world-class rhetorical skills and convince a horde of followers that everyone who isn't Team Them is a Satanist out to eat their babies or something.