3 ms·
I really like "gullibility" as a way of explaining this limitation to people - both technical and non-technical alike. I think it's worth the anthropomorphizing
by simonw 3y ago
I really like "gullibility" as a way of explaining this limitation to people - both technical and non-technical alike. I think it's worth the anthropomorphizing risk.
See also https://simonwillison.net/2023/Apr/7/chatgpt-lies/ https://simonwillison.net/2023/Apr/7/chatgpt-lies/
- thomastjeffery 3y agoI don't, because it implies the possibility of a solution. It's not a limitation or a bug. It's a feature. No LLM will ever overcome this "limitation" without fundamentally changing from an LLM to something else.
- simonw 3y agoThat still works for me. I've been telling people that LLMs are inherently gullible, and asking them how they are accounting for that in their software designs. I agree: I don't think gullibility will be solved by LLMs alone. The key thing is that people building software on top of LLMs understand that and use that knowledge to avoid making bad design decisions. See also: https://simonwillison.net/2023/Dec/20/mitigate-prompt-injection/ https://simonwillison.net/2023/Dec/20/mitigate-prompt-inject...
- thomastjeffery 3y agoI generally agree, but the reason I want to nitpick is that the narrative we have implies that things will get better; and that in turn implies that they may have already, right under our noses.