5 ms·

From the Verge's liveblogging (https://www.theverge.com/2023/2/7/23588249/microsoft-event-ai-live-blog-openai-chatgpt-bing-announcements-news https://www.thever

From the Verge's liveblogging (https://www.theverge.com/2023/2/7/23588249/microsoft-event-ai-live-blog-openai-chatgpt-bing-announcements-news https://www.theverge.com/2023/2/7/23588249/microsoft-event-a...) "This is an important part of the presentation, but I just want to note that Microsoft is having to carefully explain how its new search engine will be prevented from helping to plan school shootings. "Early red teaming showed that the model could help plan attacks" on things like schools. "We don't want to aid in illegal activity." So the model is used to act as a bad actor to test the model itself." If ChatGPT is still susceptible to simple prompt engineering attacks like DAN, I don't feel confident that their safety system is actually going to be robust enough to prevent malicious use.

rejectfinite 4y ago
Whatever. The entire internet can be used for "malicious use" This is cool tech, and right now it just needs to get out.
- bun_at_work 4y ago
  I agree, and the argument OP is making sounds similar to book banning - let's ban the Anarchist's Cookbook so people won't be terrorists isn't actually sound logic.
- throwawayapples 4y ago
  Yeah. We should probably delete all those pages on Wikipedia. Like, all of them. And Google Maps, too. Streetview? Another nightmare waiting to happen.
delfinom 4y ago
I feel the problem with red teaming is you need to actually get real red team players to play the game. Normal humans are just too naïve and sheltered in approaches hah.
joxel 4y ago
This is dumb shit just like journalists going onto YouTube every few years and finding incendiary videos. There will always be a way that a person can use something for evil, that’s not the fault of the thing.
nradov 4y ago
What counts as "malicious" use? We all agree that school shootings should be prevented. But would it be malicious if Ukrainian military personnel used a LLM for advice on the best way to kill Russian invaders? Facebook used to have a moderation policy banning promotion of violence. But then they made an exception and decided that urging the deaths of Russian soldiers is fine. https://www.reuters.com/world/europe/exclusive-facebook-instagram-temporarily-allow-calls-violence-against-russians-2022-03-10/ https://www.reuters.com/world/europe/exclusive-facebook-inst... Personally I support the right of Ukrainians to defend themselves against foreign aggression. But deciding which forms of violence are justified and which are malicious is obviously highly subjective and contextual. I am uncomfortable with leaving those judgements up to a handful of unaccountable employees in big tech companies.

rejectfinite 4y ago

Whatever. The entire internet can be used for "malicious use" This is cool tech, and right now it just needs to get out.

bun_at_work 4y ago
I agree, and the argument OP is making sounds similar to book banning - let's ban the Anarchist's Cookbook so people won't be terrorists isn't actually sound logic.
throwawayapples 4y ago
Yeah. We should probably delete all those pages on Wikipedia. Like, all of them. And Google Maps, too. Streetview? Another nightmare waiting to happen.

delfinom 4y ago

I feel the problem with red teaming is you need to actually get real red team players to play the game. Normal humans are just too naïve and sheltered in approaches hah.

joxel 4y ago

This is dumb shit just like journalists going onto YouTube every few years and finding incendiary videos. There will always be a way that a person can use something for evil, that’s not the fault of the thing.

nradov 4y ago

What counts as "malicious" use? We all agree that school shootings should be prevented. But would it be malicious if Ukrainian military personnel used a LLM for advice on the best way to kill Russian invaders? Facebook used to have a moderation policy banning promotion of violence. But then they made an exception and decided that urging the deaths of Russian soldiers is fine. https://www.reuters.com/world/europe/exclusive-facebook-instagram-temporarily-allow-calls-violence-against-russians-2022-03-10/ https://www.reuters.com/world/europe/exclusive-facebook-inst... Personally I support the right of Ukrainians to defend themselves against foreign aggression. But deciding which forms of violence are justified and which are malicious is obviously highly subjective and contextual. I am uncomfortable with leaving those judgements up to a handful of unaccountable employees in big tech companies.