5 ms·

I do not find this view to be useful. The are cases from my personal experience, where I asked somewhat esoteric practical questions, that likely do not (seem)

I do not find this view to be useful. The are cases from my personal experience, where I asked somewhat esoteric practical questions, that likely do not (seem) to have a clear answer in the web and I have got considerable help from ChatGPT. At some point this dichotomy of 'statistical word generator' vs 'true intelligence' should go away as it's just not useful. (I think these discussions always lead to Chinese Room problem; and IMO at some point it does not matter what 'dumb' process is behind, provided it solves a problem or the system behaves like an intelligent agent)

lanternfish 2y ago
Except there is a qualitative difference in the class of knowledge that a statistical word generator and an expert system would generate. Just because a LLM _can_ offer valuable and insightful information, doesn't mean that it doesn't also hallucinate. The most troubling factor here is that often the hallucinated content also looks like valuable and insightful information, but is just incorrect. This is the use. You have to hold that awareness whenever interacting with these systems.
- outofpaper 2y ago
  Yup exatly. They are dream machines. LLMs without other systems can only work in the flow. The fact that in this word flow the larger LLMs can generate navigation instructions for actual mazes and solve random algorithmicly generated problems doesn't mean they are not hallucinating it just means we're getting wonderfully useful hallucinations.
  - refulgentis 2y ago
    Hallucination is a term that means "imagined facts", so it's very hard for me to parse this comment into something meaningful beyond "if we say it always generates hallucination, we can say it always generates hallucinations"
    - jrm4 2y ago
      You Google a restaurant that appears to be open. You go there, and you find that the restaurant is no longer there. Did Google "hallucinate" a restaurant? Because this is no different.
      - refulgentis 2y ago
        We can empirically test if hallucination is a good word for communicating this concept, by checking if people describe(d) that as a hallucination (they don't). This is all IMHO, I'm not trying to be difficult or nitpick, I just don't understand the idea as communicated. As applied to LLMs, it sounds like hallucination == could be wrong, and this Google example seems further away even when steel-manning, ex. we don't say all Google results are hallucinated.
        skywhopper 2y ago
        It doesn’t mean automatically wrong. It’s just bullshitting. It makes up something that fits a pattern. Depending on the question, the pattern may be right more often than not. If you ask ChatGPT, “hey is the McDonald’s near my house open at 6pm?” It doesn’t know anything about where you are or if there’s a McDonald’s or what its hours are. It will likely hallucinate that sure, it’s open at 6pm. But when it does so is it “right” in a meaningful way?
        jrm4 2y ago
        Yup. IMHO, I think "bullshitting" is a much better word than hallucinating and/or getting it right! Much like real life bullshitters, it is inclined to say something truthful-sounding, but doesn't actually have a strong reliability towards truth per se.
        randomdata 2y ago
        Bullshitting implies intent to deceive. As far as we know, an LLM honestly "believes" (as if you need another rabbit hole) what it says. Delusion, perhaps? Delusion implies a degree of consistency, though. LLMs can be on point one minute and completely off the rails the next even when prompted with the same prompt. Hallucination fits better here as it speaks to the real-time "perception" (there's another one for you). An LLM is not a brain, though, so no matter which analogy you choose, it will come with some flaws. Regardless, "hallucination" has moved past analogy territory and now has its own LLM-specific usage with reasonably wide acceptance so the analogy angle is now moot anyway.
- airstrike 2y ago
  That's not what the GP is arguing, though
- BeetleB 2y ago
  > Except there is a qualitative difference in the class of knowledge that a statistical word generator and an expert system would generate. There's a lot of difference between the two, and you don't have to treat it as one or another. It's OK to treat it as something in the middle. > The most troubling factor here is that often the hallucinated content also looks like valuable and insightful information, but is just incorrect. This is the use. You have to hold that awareness whenever interacting with these systems. Completely agree, sans the word "troubling". It's not troubling. It is what it is. As long as you keep it in mind when you use it, and treat it as an entity that can be completely wrong, and use it where it's OK to be completely wrong (e.g. when the output is easily verifiable), there's nothing "troubling" with that.
jrm4 2y ago
You're stuck on a problem. You grab a random comic book from the shelf and something that is written in the comic book sparks your solution in your head. How intelligent is the comic book? Is it hallucinating or being correct or what? The answer is none of those things, right? YOU did the thinking. Not the inanimate object; what it did was a happy coincidence.
- BeetleB 2y ago
  Pointless discussion on semantics. If a random page in a random comic book gives me the answer I seek 30% of the time, it's incredibly useful. Little effort was spent in seeking the answer, and the 70% of the time it is wrong led to little waste in time. Now if you put a mechanical arm interface in the middle where I give my query to a machine, and it randomly picks the comic book and page, which answers my question 30% of the time - I have no trouble calling it "intelligent". Contrast it with Google searches that don't give me the answer I seek, but use up an order of magnitude more of my time.
0xdeadbeefbabe 2y ago
It's like you want to protect our right to hallucinate, because ChatGPT can't.
skywhopper 2y ago
Can you provide some examples of things that aren’t on the web but that ChatGPT helped you with? I’ve yet to see an example that’s not in the likely training set (which includes more than just the public Internet). But to your point, I disagree: the mechanism matters. Just because you haven’t detected the limitations of the mechanism behind ChatGPT doesn’t mean it’s not there.
- BeetleB 2y ago
  My guess is he's referring to things that are out there on the web, but not in the top results and not easy to find.
- emptiestplace 2y ago
  Perhaps I'm confused, but your question seems contradictory and ambiguous. You first imply (I think?) that ChatGPT is limited to web-sourced information, but then acknowledge the training set includes more than just the public web. Can you please clarify what you are asking?
dontlikeyoueith 2y ago
This is just admitting you don't care about knowledge or truth.
HarHarVeryFunny 2y ago
Sure, but the question posed isn't whether LLMs exhibit intelligence (obviously so, minimally in Chinese Room sense), or can they combine sources (sure, no way to stop them), but why do they hallucinate. Notwithstanding the amazing things they can do, I don't think it helps understanding by viewing LLMs in too abstract of a way as intelligent agents. After all, in reality they are "just" language models, and hopefully in 2024 the nuance of what they needed to learn to be GOOD language models doesn't need to be explicitly stated every time we discuss them. Looking at them as language models, it's easy to explain why they hallucinate, are poor reasoners, etc, and IMO does nothing to distract from understanding why they also exhibit intelligence when operating "in distribution".

lanternfish 2y ago

Except there is a qualitative difference in the class of knowledge that a statistical word generator and an expert system would generate. Just because a LLM _can_ offer valuable and insightful information, doesn't mean that it doesn't also hallucinate. The most troubling factor here is that often the hallucinated content also looks like valuable and insightful information, but is just incorrect. This is the use. You have to hold that awareness whenever interacting with these systems.

outofpaper 2y ago
Yup exatly. They are dream machines. LLMs without other systems can only work in the flow. The fact that in this word flow the larger LLMs can generate navigation instructions for actual mazes and solve random algorithmicly generated problems doesn't mean they are not hallucinating it just means we're getting wonderfully useful hallucinations.
- refulgentis 2y ago
  Hallucination is a term that means "imagined facts", so it's very hard for me to parse this comment into something meaningful beyond "if we say it always generates hallucination, we can say it always generates hallucinations"
  - jrm4 2y ago
    You Google a restaurant that appears to be open. You go there, and you find that the restaurant is no longer there. Did Google "hallucinate" a restaurant? Because this is no different.
    - refulgentis 2y ago
      We can empirically test if hallucination is a good word for communicating this concept, by checking if people describe(d) that as a hallucination (they don't). This is all IMHO, I'm not trying to be difficult or nitpick, I just don't understand the idea as communicated. As applied to LLMs, it sounds like hallucination == could be wrong, and this Google example seems further away even when steel-manning, ex. we don't say all Google results are hallucinated.
      - skywhopper 2y ago
        It doesn’t mean automatically wrong. It’s just bullshitting. It makes up something that fits a pattern. Depending on the question, the pattern may be right more often than not. If you ask ChatGPT, “hey is the McDonald’s near my house open at 6pm?” It doesn’t know anything about where you are or if there’s a McDonald’s or what its hours are. It will likely hallucinate that sure, it’s open at 6pm. But when it does so is it “right” in a meaningful way?
        jrm4 2y ago
        Yup. IMHO, I think "bullshitting" is a much better word than hallucinating and/or getting it right! Much like real life bullshitters, it is inclined to say something truthful-sounding, but doesn't actually have a strong reliability towards truth per se.
        randomdata 2y ago
        Bullshitting implies intent to deceive. As far as we know, an LLM honestly "believes" (as if you need another rabbit hole) what it says. Delusion, perhaps? Delusion implies a degree of consistency, though. LLMs can be on point one minute and completely off the rails the next even when prompted with the same prompt. Hallucination fits better here as it speaks to the real-time "perception" (there's another one for you). An LLM is not a brain, though, so no matter which analogy you choose, it will come with some flaws. Regardless, "hallucination" has moved past analogy territory and now has its own LLM-specific usage with reasonably wide acceptance so the analogy angle is now moot anyway.
airstrike 2y ago
That's not what the GP is arguing, though
BeetleB 2y ago
> Except there is a qualitative difference in the class of knowledge that a statistical word generator and an expert system would generate. There's a lot of difference between the two, and you don't have to treat it as one or another. It's OK to treat it as something in the middle. > The most troubling factor here is that often the hallucinated content also looks like valuable and insightful information, but is just incorrect. This is the use. You have to hold that awareness whenever interacting with these systems. Completely agree, sans the word "troubling". It's not troubling. It is what it is. As long as you keep it in mind when you use it, and treat it as an entity that can be completely wrong, and use it where it's OK to be completely wrong (e.g. when the output is easily verifiable), there's nothing "troubling" with that.

jrm4 2y ago

You're stuck on a problem. You grab a random comic book from the shelf and something that is written in the comic book sparks your solution in your head. How intelligent is the comic book? Is it hallucinating or being correct or what? The answer is none of those things, right? YOU did the thinking. Not the inanimate object; what it did was a happy coincidence.

BeetleB 2y ago
Pointless discussion on semantics. If a random page in a random comic book gives me the answer I seek 30% of the time, it's incredibly useful. Little effort was spent in seeking the answer, and the 70% of the time it is wrong led to little waste in time. Now if you put a mechanical arm interface in the middle where I give my query to a machine, and it randomly picks the comic book and page, which answers my question 30% of the time - I have no trouble calling it "intelligent". Contrast it with Google searches that don't give me the answer I seek, but use up an order of magnitude more of my time.

0xdeadbeefbabe 2y ago

It's like you want to protect our right to hallucinate, because ChatGPT can't.

skywhopper 2y ago

Can you provide some examples of things that aren’t on the web but that ChatGPT helped you with? I’ve yet to see an example that’s not in the likely training set (which includes more than just the public Internet). But to your point, I disagree: the mechanism matters. Just because you haven’t detected the limitations of the mechanism behind ChatGPT doesn’t mean it’s not there.

BeetleB 2y ago
My guess is he's referring to things that are out there on the web, but not in the top results and not easy to find.
emptiestplace 2y ago
Perhaps I'm confused, but your question seems contradictory and ambiguous. You first imply (I think?) that ChatGPT is limited to web-sourced information, but then acknowledge the training set includes more than just the public web. Can you please clarify what you are asking?

dontlikeyoueith 2y ago

This is just admitting you don't care about knowledge or truth.

HarHarVeryFunny 2y ago

Sure, but the question posed isn't whether LLMs exhibit intelligence (obviously so, minimally in Chinese Room sense), or can they combine sources (sure, no way to stop them), but why do they hallucinate. Notwithstanding the amazing things they can do, I don't think it helps understanding by viewing LLMs in too abstract of a way as intelligent agents. After all, in reality they are "just" language models, and hopefully in 2024 the nuance of what they needed to learn to be GOOD language models doesn't need to be explicitly stated every time we discuss them. Looking at them as language models, it's easy to explain why they hallucinate, are poor reasoners, etc, and IMO does nothing to distract from understanding why they also exhibit intelligence when operating "in distribution".