3 ms·

I assume everyone who has someone with an AI safety job title uses unlearning to make sure their models don't remember how to make common illegal drugs, poisons

I assume everyone who has someone with an AI safety job title uses unlearning to make sure their models don't remember how to make common illegal drugs, poisons or explosives. The threat model here is probably more accidental un-unlearning these facts and distributing those models (as is common with quantized models). Most of this "dangerous" information is readily available in textbooks, patents, amateur chemistry forums etc. But as a society we generally assume that those smart enough to find and understand that kind of information are smart enough not to abuse it. We just don't want Mythbusters to explain it on prime-time TV, or ChatGPT explaining it to people

aziaziazi 2y ago
Mythbusters chooses the subjects he discusses while ChatGPT responses depends on the context (you) provided. It will give you a list a poisons if you asked (5 seconds), as well as an encyclopédie or Google would (30 seconds). Mythbuster broadcasting poisons recipes could seeds bad ideas that wouldn’t been triggered otherwise. ChatGPT wouldn’t give a poisons recipe if not asked specifically.
- FergusArgyll 2y ago
  This is a decent point. I hadn't really thought of a good reason why we e.g. sell old army manuals with step by step guides on making almost anything but there's no (afaik) HBO mini-series "Learn guerilla warfare"
mschuster91 2y ago
> But as a society we generally assume that those smart enough to find and understand that kind of information are smart enough not to abuse it. There's an almost complete step by step guide for most explosives on Wikipedia. The problem is that decisionmakers and regulators are excessively dumb - "AI bad" reigns supreme over the fact that Wikipedia tells you more about making bombs, even nuclear bombs if you want, than ChatGPT. AI in its current form is still bad - from all the IP issues over the environmental cost to it enabling spam, harassment and deception on a speed, scale and easiness not seen before in history - but most of the stuff where "regulators" cry about is just frankly bullshit.

aziaziazi 2y ago

Mythbusters chooses the subjects he discusses while ChatGPT responses depends on the context (you) provided. It will give you a list a poisons if you asked (5 seconds), as well as an encyclopédie or Google would (30 seconds). Mythbuster broadcasting poisons recipes could seeds bad ideas that wouldn’t been triggered otherwise. ChatGPT wouldn’t give a poisons recipe if not asked specifically.

FergusArgyll 2y ago
This is a decent point. I hadn't really thought of a good reason why we e.g. sell old army manuals with step by step guides on making almost anything but there's no (afaik) HBO mini-series "Learn guerilla warfare"

mschuster91 2y ago

> But as a society we generally assume that those smart enough to find and understand that kind of information are smart enough not to abuse it. There's an almost complete step by step guide for most explosives on Wikipedia. The problem is that decisionmakers and regulators are excessively dumb - "AI bad" reigns supreme over the fact that Wikipedia tells you more about making bombs, even nuclear bombs if you want, than ChatGPT. AI in its current form is still bad - from all the IP issues over the environmental cost to it enabling spam, harassment and deception on a speed, scale and easiness not seen before in history - but most of the stuff where "regulators" cry about is just frankly bullshit.