3 ms·
They're panicking and hitting the 'AI SAFETY' button hard.
by ok123456 2y ago
They're panicking and hitting the 'AI SAFETY' button hard.
- vlovich123 2y agoPanicking how? This seems like a desirable feature a lot of customers are looking for.
- logicchains 2y agoWhat customers? I've never heard anyone saying "I wish Claude would refuse more of my requests".
- vlovich123 2y agoI'm pretty sure they have customers who are saying "I want to deploy a chat bot on my website that can't be tricked into giving out prices I don't agree to".
- logicchains 2y agoI'd be very interested to know the name of any of those companies letting a LLM set the price for their products. For research purposes only, of course.
- BryantD 2y agoAir Canada was held liable for a refund offer a chatbot made: https://www.bbc.com/travel/article/20240222-air-canada-chatbot-misinformation-what-travellers-should-know https://www.bbc.com/travel/article/20240222-air-canada-chatb... Not exactly your scenario, but a live example of the sort of problem Anthropic wants to prevent.
- ok123456 2y agoAnd, that's not what they're trying to prevent here.
- BoorishBears 2y agoThis research doesn't do that. It focuses on CBRN and does so so narrowly that until they removed "BRN" from CBRN it was refusing 44% of requests made to the model.
- gs17 2y agoFor example: https://futurism.com/the-byte/car-dealership-ai https://futurism.com/the-byte/car-dealership-ai It didn't actually result in someone getting a new car for $1, but I'd imagine the dealer was still annoyed at people (who don't live close enough to buy a car from them) abusing their chatbot.
- sitkack 2y agoWrite stupid code, win stupid prizes. This has nothing to do with safety.
- esafak 2y agoI've never heard a bad actor saying "I wish law enforcement would block more of my efforts".
- hobo_in_library 2y agoSimilar to what others have mentioned: People offering domain specific bots and don't want that expensive compute abused as a free general purpose LLM Imagine you're American Airline and someone goes to your chatbot and asks it to generate React code for them
- deadbabe 2y agoWould you want to allow a human customer service agent to talk on the phone with a customer about whatever inappropriate or confidential things they felt like asking about?