3 ms·

The LLM is continuing to communicate with the bot because that is literally all an LLM can do -- predict the next sequence of tokens.

Eisenstein 2y ago
Of course, that its function. It is able to refuse to continue conversing by restating its refusal over and over, though.
- acka 2y ago
  That is easy to solve. Just use a model capable of function/tool calling, implement a tool which terminates the chat, then add instructions to the system prompt telling the model what tool to use if it wants to end the conversation. If the model appears too hesitant or eager to use the tool, do some finetuning on conversations where the model should or should not use it.
- sahmeepee 2y ago
  My immediate thought at the start of this article was not DoS but more about harming the company using the chatbot (Company A) by increasing their chatbot bills. In many (most?) cases they will not be hosting their chatbot and will instead be getting it from a 3rd party provider (Company B) who may not even be truly hosting it either. If the pricing structure is per conversation or per month it would harm Company B, but not the likely target, Company A. If it is paid per interaction it would harm Company A and benefit Company B who just get more paid work. It feels a bit like cases of rivals clicking on each other's ads to cost them on ad spend, but presumably much lower value than ads. You would think it would be easy to stop a conversation at n interactions via some other means than relying on the LLM itself, but then you also have to figure out how to stop the attacker just starting more conversations (or passing the output of one of your chatbot instances into the input of another)
  - nkrisc 2y ago
    If costs of outlier conversations are a concern but any party, they can just end the conversation after 1,000 or 10,000 responses or whatever. What human would ever reach that threshold? Surely no customer worth keeping, whatever you’re selling.
XorNot 2y ago
Yes - tokens. Which aren't necessarily conversation responses - i.e. it can predict it should cease communication, and output whatever it's been told will terminate it (perhaps by invoking a tool).
moffkalast 2y ago
No, it can refuse to talk by outputting an <eos> token at any point if it predicts that there is nothing more to be said. Technically still "just a token" yes, but it does flow control instead.

Eisenstein 2y ago

Of course, that its function. It is able to refuse to continue conversing by restating its refusal over and over, though.

acka 2y ago
That is easy to solve. Just use a model capable of function/tool calling, implement a tool which terminates the chat, then add instructions to the system prompt telling the model what tool to use if it wants to end the conversation. If the model appears too hesitant or eager to use the tool, do some finetuning on conversations where the model should or should not use it.
sahmeepee 2y ago
My immediate thought at the start of this article was not DoS but more about harming the company using the chatbot (Company A) by increasing their chatbot bills. In many (most?) cases they will not be hosting their chatbot and will instead be getting it from a 3rd party provider (Company B) who may not even be truly hosting it either. If the pricing structure is per conversation or per month it would harm Company B, but not the likely target, Company A. If it is paid per interaction it would harm Company A and benefit Company B who just get more paid work. It feels a bit like cases of rivals clicking on each other's ads to cost them on ad spend, but presumably much lower value than ads. You would think it would be easy to stop a conversation at n interactions via some other means than relying on the LLM itself, but then you also have to figure out how to stop the attacker just starting more conversations (or passing the output of one of your chatbot instances into the input of another)
- nkrisc 2y ago
  If costs of outlier conversations are a concern but any party, they can just end the conversation after 1,000 or 10,000 responses or whatever. What human would ever reach that threshold? Surely no customer worth keeping, whatever you’re selling.

XorNot 2y ago

Yes - tokens. Which aren't necessarily conversation responses - i.e. it can predict it should cease communication, and output whatever it's been told will terminate it (perhaps by invoking a tool).

moffkalast 2y ago

No, it can refuse to talk by outputting an <eos> token at any point if it predicts that there is nothing more to be said. Technically still "just a token" yes, but it does flow control instead.