3 ms·

Ask a human the same question and force them to answer (instead of saying I don't know), and they will hallucinate a list of references for you just the same. D

• ithkuil 3y ago
  This! One could argue that humans will say "I don't know" when they don't know, but that ultimately depends on training. We currently don't have LLMs trained to answer "I don't know" (I suspect that in part that's due to incentives of focusing on something that looks impressive; one could argue that if an AI system too often says "I don't know" it's less impressive albeit it may be more useful since you can rely upon it more when it says it knows) There are plenty of cases where humans are trained to hope they sound convincing even if they make up answers. I've seen a fair share of this done by students.
• endofreach 3y ago
  This was my goto argument when people without technical background were laughing about ChatGPTs bullshit generation (before it was called hallucinating). Put a gun to Einsteins head and force him to answer any question you ask him without hesitating or denying an answer. This is not an argument for stating LLMs like ChatGPT are intelligent (it is not, but not for any of the reasons i have read about so far anywhere). But the hallucinations definitely don't show anything about intelligence. Actually, if they did, they would actually indicate intelligence rather than not. So, quite the opposite which should be very obvious.