4 ms·

What customers? I've never heard anyone saying "I wish Claude would refuse more of my requests".

vlovich123 2y ago
I'm pretty sure they have customers who are saying "I want to deploy a chat bot on my website that can't be tricked into giving out prices I don't agree to".
- logicchains 2y ago
  I'd be very interested to know the name of any of those companies letting a LLM set the price for their products. For research purposes only, of course.
  - BryantD 2y ago
    Air Canada was held liable for a refund offer a chatbot made: https://www.bbc.com/travel/article/20240222-air-canada-chatbot-misinformation-what-travellers-should-know https://www.bbc.com/travel/article/20240222-air-canada-chatb... Not exactly your scenario, but a live example of the sort of problem Anthropic wants to prevent.
    - ok123456 2y ago
      And, that's not what they're trying to prevent here.
- BoorishBears 2y ago
  This research doesn't do that. It focuses on CBRN and does so so narrowly that until they removed "BRN" from CBRN it was refusing 44% of requests made to the model.
gs17 2y ago
For example: https://futurism.com/the-byte/car-dealership-ai https://futurism.com/the-byte/car-dealership-ai It didn't actually result in someone getting a new car for $1, but I'd imagine the dealer was still annoyed at people (who don't live close enough to buy a car from them) abusing their chatbot.
- sitkack 2y ago
  Write stupid code, win stupid prizes. This has nothing to do with safety.
esafak 2y ago
I've never heard a bad actor saying "I wish law enforcement would block more of my efforts".
hobo_in_library 2y ago
Similar to what others have mentioned: People offering domain specific bots and don't want that expensive compute abused as a free general purpose LLM Imagine you're American Airline and someone goes to your chatbot and asks it to generate React code for them
deadbabe 2y ago
Would you want to allow a human customer service agent to talk on the phone with a customer about whatever inappropriate or confidential things they felt like asking about?

I'm pretty sure they have customers who are saying "I want to deploy a chat bot on my website that can't be tricked into giving out prices I don't agree to".

logicchains 2y ago
I'd be very interested to know the name of any of those companies letting a LLM set the price for their products. For research purposes only, of course.
- BryantD 2y ago
  Air Canada was held liable for a refund offer a chatbot made: https://www.bbc.com/travel/article/20240222-air-canada-chatbot-misinformation-what-travellers-should-know https://www.bbc.com/travel/article/20240222-air-canada-chatb... Not exactly your scenario, but a live example of the sort of problem Anthropic wants to prevent.
  - ok123456 2y ago
    And, that's not what they're trying to prevent here.
BoorishBears 2y ago
This research doesn't do that. It focuses on CBRN and does so so narrowly that until they removed "BRN" from CBRN it was refusing 44% of requests made to the model.

gs17 2y ago

For example: https://futurism.com/the-byte/car-dealership-ai https://futurism.com/the-byte/car-dealership-ai It didn't actually result in someone getting a new car for $1, but I'd imagine the dealer was still annoyed at people (who don't live close enough to buy a car from them) abusing their chatbot.

sitkack 2y ago
Write stupid code, win stupid prizes. This has nothing to do with safety.

esafak 2y ago

I've never heard a bad actor saying "I wish law enforcement would block more of my efforts".

hobo_in_library 2y ago

Similar to what others have mentioned: People offering domain specific bots and don't want that expensive compute abused as a free general purpose LLM Imagine you're American Airline and someone goes to your chatbot and asks it to generate React code for them

deadbabe 2y ago

Would you want to allow a human customer service agent to talk on the phone with a customer about whatever inappropriate or confidential things they felt like asking about?