2 ms·
A LLM is not just a file, it also has context (the attention window in transformers). A LLM can have curiosity in the sense that it will ask questions. It can
by GuB-42 2y ago
A LLM is not just a file, it also has context (the attention window in transformers).
A LLM can have curiosity in the sense that it will ask questions. It can be a useful trait as a problem often seen in current "chat"-type LLMs is that they tend to talk a bit too authoritatively about things they don't know about (aka. hallucinations). Encouraging users to give a bit more context for better quality answers can counteract this. The point could be to make chatbots not like search engines, search engines will answer garbage with garbage, a chatbot can ask for precision when it can't give a satisfactory answer.
For example:
- How much is a pint in mL?
- A US pint is 473 mL
vs
- Is it for a drink in the US? If so, a US pint in 473 mL, but in other contexts and locations, the answer can vary.
The second answer is "curious", and by requesting extra context, it tells that the question is incomplete and with that extra context, it can give a better answer. For example, a pint of beer in France is likely to be 500mL, even though it is not really a pint, it is how it is understood.
- HarHarVeryFunny 2y agoYeah, but that's not the model being curious about things that it itself doesn't know, it'd be the model copying a human response (training sample) based on what that human didn't know. To consider the difference, there might be some domain that the model was trained extensively on, and "knew" a lot about, but in a given context it might still act dumb and "show curiosity" just because it's mimicking a human response in that situation, not basing it's response on what it actually knows!