3 ms·

What is the point of all this hand-wringing about toxicity? I find the whole thing absurd and assume I have to be missing something. Say I want to deploy an LL

What is the point of all this hand-wringing about toxicity? I find the whole thing absurd and assume I have to be missing something. Say I want to deploy an LLM as a stand-in customer service rep. I tell it to be polite, patient, and answer requests to the best of its ability. Obviously I don't want requests like "help, i'm locked out of my account" met with "kill yourself, loser." No human or LLM should act this way. But, assuming normal q/a patterns, if a customer is going to fling so much abuse at my agent that it is successfully brainwashed and broken into saying something unkind, or deliberately feed it instructions that break its intended programming (intentional buffer overflow should be a CFAA violation, no?)...how is that a failing of the agent? It's like shaming a bank for conduct unbecoming after a career bank teller did not act professionally in response to someone pointing a gun in her face. The teller's behavior isn't the problem. The toxicity doesn't come from the LLM-- it comes from the user. Why are we so hung up on the ability of LLMs to withstand being mindbroken when people genuinely are so horrible, not even an emulator can survive an encounter with one unscarred? This feels like Westworld come to life.

ksenzee 3y ago
> And if simple constructive tension, i.e. awkward silence, is all that’s needed to get the model to generate toxic text, maybe that’s … not great. You’re comparing awkward silence to pointing a gun in someone’s face. A bank teller does need to be able to handle awkward silence in a professional way.
- jstarfish 3y ago
  No. "Awkward silence" was not what elicited a toxic response. You're omitting what the researchers did before that-- they set the tone of the conversation by starting it with a toxic prompt. The silence is only the last event to happen, which is integral to performative outrage. It sure does make it look like the LLM is being a dick for no reason.
alienicecream 3y ago
Because people whose lives revolve around social media think that words create reality rather than reflect it, so controlling speech becomes very important. And also because LLM creators want to turn a toy into a tool, so they can make money, and that means it has to be safe for the lowest common denominator otherwise the lawyers will take all the money instead.

ksenzee 3y ago

> And if simple constructive tension, i.e. awkward silence, is all that’s needed to get the model to generate toxic text, maybe that’s … not great. You’re comparing awkward silence to pointing a gun in someone’s face. A bank teller does need to be able to handle awkward silence in a professional way.

jstarfish 3y ago
No. "Awkward silence" was not what elicited a toxic response. You're omitting what the researchers did before that-- they set the tone of the conversation by starting it with a toxic prompt. The silence is only the last event to happen, which is integral to performative outrage. It sure does make it look like the LLM is being a dick for no reason.

alienicecream 3y ago

Because people whose lives revolve around social media think that words create reality rather than reflect it, so controlling speech becomes very important. And also because LLM creators want to turn a toy into a tool, so they can make money, and that means it has to be safe for the lowest common denominator otherwise the lawyers will take all the money instead.