4 ms·
I've found it very helpful to make the following distinction: Spec: Do X in situation Y. Correctness bug: It doesn't do X in situation Y. Fitness-for-purpose
by SilasX 2y ago
I've found it very helpful to make the following distinction:
Spec: Do X in situation Y.
Correctness bug: It doesn't do X in situation Y.
Fitness-for-purpose (FFP) bug: It does X in situation Y, but, knowing this, you decide you don't actually want it to do X in situation Y.
Hallucination is an FFP bug.
- AstralStorm 2y agoSorry, but it's a correctness bug most of the time[], as the correct information is known or known to not exist. If ask a math question and you get a random incorrect equation, it's not unfit for purpose, just incorrect. FFP would be returning misinformation from the model, which is not a hallucination per se. Or the model misunderstanding the question and returning a correct answer to a related question. [] Except for art generators.
- SilasX 2y ago"Correct" here doesn't mean "correct" information -- I made sure to clarify what it means with an example.