3 ms·

Sure there is. Know the answers to questions already. Being trained on a noninsignificant proportion of all questions ever asked on the Internet means you hav

Sure there is. Know the answers to questions already. Being trained on a noninsignificant proportion of all questions ever asked on the Internet means you have to wonder how smart LLMs really are. Seeing the ways they fail, confirms not so much. They are able to abstract a bit of knowledge an statically match it to questions and answers already seen. Confident hallucinations help you see this. The best hallucinations I saw was asking chatgpt for a list of references for a topic I was researching. They all looked entirely believable, titles, authors, institutions, summaries… IIRC there were about twenty of them. Only one actually existed.

Aeolos 3y ago
Ask a human the same question and force them to answer (instead of saying I don't know), and they will hallucinate a list of references for you just the same. Does that mean humans are not intelligent? I don't think your argument proves what you claim.
- ithkuil 3y ago
  This! One could argue that humans will say "I don't know" when they don't know, but that ultimately depends on training. We currently don't have LLMs trained to answer "I don't know" (I suspect that in part that's due to incentives of focusing on something that looks impressive; one could argue that if an AI system too often says "I don't know" it's less impressive albeit it may be more useful since you can rely upon it more when it says it knows) There are plenty of cases where humans are trained to hope they sound convincing even if they make up answers. I've seen a fair share of this done by students.
- endofreach 3y ago
  This was my goto argument when people without technical background were laughing about ChatGPTs bullshit generation (before it was called hallucinating). Put a gun to Einsteins head and force him to answer any question you ask him without hesitating or denying an answer. This is not an argument for stating LLMs like ChatGPT are intelligent (it is not, but not for any of the reasons i have read about so far anywhere). But the hallucinations definitely don't show anything about intelligence. Actually, if they did, they would actually indicate intelligence rather than not. So, quite the opposite which should be very obvious.
jiggawatts 3y ago
How many leaf URLs have you memorised? I don't mean "google.com", I mean the URL of a specific paper in a journal, or an article in some random news site? If you were forced at gunpoint to provide such URLs -- valid or not -- with literally no option other than to keep talking, you would... make up some vaguely correct-sounding URLs! The AIs are doing precisely what an intelligent human in their place would do. Blurting out something, anything, even if they don't have the answers. That's because the current crop of AIs are essentially being forced to provide the next token, directly equivalent to a human being forced to continue speaking against their will. There are fixes being put in place already for this, such as using human feedback to tune the AI so that it answers with a generic "I don't know" blurb if it's not sufficiently confident in the next token. Annoyingly, the new "Turbo" versions of ChatGPT have overdone this a bit, and now it refuses to answer even when it does actually know the answer! There are also variations of neural networks that encode not just "values", but probabilities and hence confidence intervals. These can directly output both tokens and the true confidence values, which would allow complex tree searches, etc... and avoid outputting this kind of gibberish entirely. The maths for this is well established, but it would be much more expensive to run models using it, so nobody has bothered yet.
- epcoa 3y ago
  > The AIs are doing precisely what an intelligent human in their place would do. Nah. Either the human is feigning, lying with full knowledge that they’re doing so, or you torture them enough and they cease responding intelligently, if they start to believe their bullshit it isn’t unreasonable to say this is a loss of intelligent reasoning. I think this is a false equivalence. > directly equivalent to a human being forced to continue speaking against their will. Maybe some superficial appearance, but not equivalence. > but probabilities and hence confidence intervals And that’s the entirety of human metacognition. Maybe it’ll get us a little closer but that’s it.

Aeolos 3y ago

Ask a human the same question and force them to answer (instead of saying I don't know), and they will hallucinate a list of references for you just the same. Does that mean humans are not intelligent? I don't think your argument proves what you claim.

ithkuil 3y ago
This! One could argue that humans will say "I don't know" when they don't know, but that ultimately depends on training. We currently don't have LLMs trained to answer "I don't know" (I suspect that in part that's due to incentives of focusing on something that looks impressive; one could argue that if an AI system too often says "I don't know" it's less impressive albeit it may be more useful since you can rely upon it more when it says it knows) There are plenty of cases where humans are trained to hope they sound convincing even if they make up answers. I've seen a fair share of this done by students.
endofreach 3y ago
This was my goto argument when people without technical background were laughing about ChatGPTs bullshit generation (before it was called hallucinating). Put a gun to Einsteins head and force him to answer any question you ask him without hesitating or denying an answer. This is not an argument for stating LLMs like ChatGPT are intelligent (it is not, but not for any of the reasons i have read about so far anywhere). But the hallucinations definitely don't show anything about intelligence. Actually, if they did, they would actually indicate intelligence rather than not. So, quite the opposite which should be very obvious.

jiggawatts 3y ago

How many leaf URLs have you memorised? I don't mean "google.com", I mean the URL of a specific paper in a journal, or an article in some random news site? If you were forced at gunpoint to provide such URLs -- valid or not -- with literally no option other than to keep talking, you would... make up some vaguely correct-sounding URLs! The AIs are doing precisely what an intelligent human in their place would do. Blurting out something, anything, even if they don't have the answers. That's because the current crop of AIs are essentially being forced to provide the next token, directly equivalent to a human being forced to continue speaking against their will. There are fixes being put in place already for this, such as using human feedback to tune the AI so that it answers with a generic "I don't know" blurb if it's not sufficiently confident in the next token. Annoyingly, the new "Turbo" versions of ChatGPT have overdone this a bit, and now it refuses to answer even when it does actually know the answer! There are also variations of neural networks that encode not just "values", but probabilities and hence confidence intervals. These can directly output both tokens and the true confidence values, which would allow complex tree searches, etc... and avoid outputting this kind of gibberish entirely. The maths for this is well established, but it would be much more expensive to run models using it, so nobody has bothered yet.

epcoa 3y ago
> The AIs are doing precisely what an intelligent human in their place would do. Nah. Either the human is feigning, lying with full knowledge that they’re doing so, or you torture them enough and they cease responding intelligently, if they start to believe their bullshit it isn’t unreasonable to say this is a loss of intelligent reasoning. I think this is a false equivalence. > directly equivalent to a human being forced to continue speaking against their will. Maybe some superficial appearance, but not equivalence. > but probabilities and hence confidence intervals And that’s the entirety of human metacognition. Maybe it’ll get us a little closer but that’s it.