3 ms·
LLMs will increasingly become interfaces to every type of system. This assertion that LLMs should obey whatever instructions as long as they are specifically p
by mingus88 2y ago
LLMs will increasingly become interfaces to every type of system.
This assertion that LLMs should obey whatever instructions as long as they are specifically prompted to do so is bizarre.
Should a bank teller LLM divulge another customers balance because you specifically told it to?
This type of exploit is called social engineering when it happens to humans, and we go to great lengths to train the human to detect and avoid it. Why should we not do the same with an LLM?
- init2null 2y agoThe underlying APIs can enforce permissions. I don’t care if LLM prompt engineering causes the bank assistant to say that John Doe has 500K in his bank account. As long as it isn’t real data, there’s far less risk. This is more or less how it works for human customer phone support, who are also limited by narrow permissions. And as we know, they are also skilled at confabulating in the interest of customer happiness.