4 ms·

Define "safety," because for me and others, the definition seems to be "not projecting politically-incorrect views or information" or just straight up censorshi

Define "safety," because for me and others, the definition seems to be "not projecting politically-incorrect views or information" or just straight up censorship.

CamperBob2 3y ago
"Safety" = "Our company is safe from competition, thanks to captive regulators."
photochemsyn 3y ago
The only thing that immediately comes to mind is you don't want a model capable of writing a Stuxnet-type cyberweapon for a script kiddie to use. All the other stuff about nuclear/chemical/biological/illegal drugs etc. is basically nonsense, that information is in the public domain already and the way governments try to control it is through legal restrictions like Select Agent Rules. Likewise, the result of giving most people a recipe for a chemical weapon is that they'll end up poisoning themselves, and those who won't already possess the requisite skills (happily the vast majority of trained experts in these fields aren't psychopaths). The real risk here is governments doing such things in the name of national security, I think. The other argument that many have adopted, that LLMs will make such cunningly clever arguments that gullible humans will follow them to their own destruction (I assume this means getting the masses to embrace communism or fascism or tax the uber-wealthy or something like that) seems utterly ridiculous to me, even though that's probably the "AI threat" I've seen the most ink spilled about. The best defense against such dark arts is simply teaching critical thinking skills to children.
jessenaser 3y ago
At bare minimum it is to define and understand what are the capabilities of a model. Just like with missiles. They could be short range, ICBM, or nuclear. So if a model could make real looking videos that could confuse even expert analysis, and the general public, then we should prepare by doing X, Y, Z, so that the public cannot be confused by fake video, since we have some preventions. X, Y, Z, are mitigations. They do not, and will not work if they are censorship, since the other person can release a model that is uncensored and use it. That misses the point of the problem. Since others have it, your mitigation is to make software systems where you can sort posts by being real or fake. How you do it is an example of safety research. Safety is not talking about restricting models, it is talking about mitigating the negative effects of a weapon used the wrong way. Same thing with code. Malware will exist, so you must make a firewall, or train employees to not click phishing links, ... The safety research is finding the best methods of mitigation, and a small team in SF is not going to be the best at doing this. If the world knew the extent of a model, they know what extent of mitigations they can do. Overnight actions and power struggles in an organization can happen. If the world had these models, even the risk of OpenAI using them in a bad way, or restricting access to just themselves on usage of models, would be a more minimal risk issue, since two models ago are open.

Define "safety," because for me and others, the definition seems to be "not projecting politically-incorrect views or information" or just straight up censorship.

CamperBob2 3y ago

"Safety" = "Our company is safe from competition, thanks to captive regulators."

photochemsyn 3y ago

The only thing that immediately comes to mind is you don't want a model capable of writing a Stuxnet-type cyberweapon for a script kiddie to use. All the other stuff about nuclear/chemical/biological/illegal drugs etc. is basically nonsense, that information is in the public domain already and the way governments try to control it is through legal restrictions like Select Agent Rules. Likewise, the result of giving most people a recipe for a chemical weapon is that they'll end up poisoning themselves, and those who won't already possess the requisite skills (happily the vast majority of trained experts in these fields aren't psychopaths). The real risk here is governments doing such things in the name of national security, I think. The other argument that many have adopted, that LLMs will make such cunningly clever arguments that gullible humans will follow them to their own destruction (I assume this means getting the masses to embrace communism or fascism or tax the uber-wealthy or something like that) seems utterly ridiculous to me, even though that's probably the "AI threat" I've seen the most ink spilled about. The best defense against such dark arts is simply teaching critical thinking skills to children.

jessenaser 3y ago

At bare minimum it is to define and understand what are the capabilities of a model. Just like with missiles. They could be short range, ICBM, or nuclear. So if a model could make real looking videos that could confuse even expert analysis, and the general public, then we should prepare by doing X, Y, Z, so that the public cannot be confused by fake video, since we have some preventions. X, Y, Z, are mitigations. They do not, and will not work if they are censorship, since the other person can release a model that is uncensored and use it. That misses the point of the problem. Since others have it, your mitigation is to make software systems where you can sort posts by being real or fake. How you do it is an example of safety research. Safety is not talking about restricting models, it is talking about mitigating the negative effects of a weapon used the wrong way. Same thing with code. Malware will exist, so you must make a firewall, or train employees to not click phishing links, ... The safety research is finding the best methods of mitigation, and a small team in SF is not going to be the best at doing this. If the world knew the extent of a model, they know what extent of mitigations they can do. Overnight actions and power struggles in an organization can happen. If the world had these models, even the risk of OpenAI using them in a bad way, or restricting access to just themselves on usage of models, would be a more minimal risk issue, since two models ago are open.