3 ms·
There's a difference between "LLM outputs contain subtly or even flagrantly wrong answers, and the LLM has no way of recognizing the errors" and "an LLM might o
by danaris 4y ago
There's a difference between "LLM outputs contain subtly or even flagrantly wrong answers, and the LLM has no way of recognizing the errors" and "an LLM might of its own agency decide to give specific users wrong answers out of active malice".
LLMs have no agency. They are not sapient, sentient, conscious, or intelligent.
- bondarchuk 4y agoI think the key point is the difference between "making an error without recognizing it" and "making an error because 'give erroneous answers' has implicitly become part of the task specification". Agency, consciousness etc.. is totally unnecessary here.
- danaris 4y agoThe scenario described by the OP in the thread was that Bing decides you are a bad person and chooses to give you wrong, possibly dangerous, answers because of it. That requires agency.
- bondarchuk 4y agoI think you are overly focused on the terminology. We are simply using words such as "decide" and "choose" in the same way that you might say a chess engine "decides" or "chooses" a move. If your point is that it would be more accurate to use different terminology, fine. But it is unwarranted to say that these things can never happen because they require agency and an LLM does not have agency. Then you are simply mistaken about the amount and type of agency required for such a scenario to play out (or alternatively about the notion that an LLM doesn't have agency, depending on your definition of "agency", it's a vague word anyway). You can try yourself to put an LLM in a context where it gives wrong answers on purpose, just by prefacing a question with "give a wrong answer to this question". "(Assume context where Bing has decided I am a bad user)" is just a more elaborate version of that.