2 ms·
this. SO MUCH THIS. I am so tired of reading through pseudo-scientific blurb from people claiming LLMs have "mental models". Especially when using retrieval-aug
by rcarmo 3y ago
this. SO MUCH THIS. I am so tired of reading through pseudo-scientific blurb from people claiming LLMs have "mental models". Especially when using retrieval-augmented generation (which futzes behaviour even more) and attributing human behaviours to a smarter autocomplete.
- recursive 3y ago"Mental model" is a useful shorthand for referring to it. We say an A-star pathfinding algorithm "wants" to minimize its cost function. We can say this while understanding that it's a computer that doesn't have desires. But we say it because it's an easier way to communicate. We don't have to talk like lawyers all the time.
- koliber 3y agoIt’s not that LLMs have mental models. It’s that people have mental models of how various things work. For example, many people have a wrong mental model of how an oven works. They think that if they set the oven for 350 F it heats with more energy than if they set the oven for 150 F. In reality it heats at exactly the same power, but it turns off when the internal temperature reaches the set rate. For most use cases it does not matter that your mental model does not match how the oven works. But there are some not he cases where it does matter. The article shares a mental model of how LLMs work that might help users use them better.