3 ms·

I disagree with this take because LLMs are, always, hallucinating. When they get things right it’s because they are lucky. Yes, yes, it’s more complicated than

by devjab 2y ago

I disagree with this take because LLMs are, always, hallucinating. When they get things right it’s because they are lucky. Yes, yes, it’s more complicated than that, but the essence of LLMs is that they are very good at being lucky. So good that they will often give you better results than random search engine clicks, but not good enough to be useful for anything important. I think what calling the times they get things wrong hallucinations is largely an advertising trick. So that they can sort of fit the LLMs into how all IT is sometimes “wonky” and sell their fundamentally flawed technology more easily. I also think it works extremely well.

Terr_ 2y ago
To offer a satirical analogy: "Lastly, I want to reassure investors and members of the press that we take these concerns very seriously: Hindenburg 2 will only contain only normal and unreactive hydrogen gas, and not the rare and unusual explosive kind, which is merely a temporary hurdle in this highly dynamic and growing field." Edit: It retrospect, perhaps a better analogy would involve gasoline, as its explosive nature is what's being actively being exploited in normal use.
- marcosdumay 2y ago
  Yes (to the edit), an analogy with making planes safer by only using non-flammable fuels is perfect.
  - Terr_ 2y ago
    I expect most people have already filled in the blanks, but for completeness: "Lastly, I want to reassure investors and members of the press that we take these concerns very seriously: The Ford Pinto-II will only contain only normal and stable gasoline, and not the rare and unusual burning kind, which is merely a temporary hurdle in this highly dynamic and explos--er--fast growing field."
mewpmewp2 2y ago
But the point is, isn't hallucinating about having malformed, altered or out of touch input rather than producing inaccurate output yourself? It is the memory pathways leading them astray. It could be thought of a memory system that at certain point any longer can't be fully sure if whatever connections they have are from actually being trained or it or created accidentally.
- Terr_ 2y ago
  > isn't hallucinating about having malformed, altered or out of touch input rather than producing inaccurate output yourself? I suppose so, in the sense that someone could simply be lying about pink elephants instead of seeing them. However it's hard to argue that the machine knows the "right" answer and is (intelligently?) deceiving us. > It is the memory pathways leading them astray. I don't think it's a "memory" issue as much as a "they don't operate the way we like to think they do" issue. Suppose a human is asked to describe different paintings on the wall of an art gallery. Sometimes their statements appear valid and you nod along, and sometimes the statements are so wrong that it alarms you, because "this person is hallucinating." Now consider how the entire situation is flipped by finding out one additional fact... They're actually totally blind. Is it a lie? Is it a hallucination? Does it matter? Either way you must dramatically re-evaluate what their "good" outputs really mean and whether they can be used.
  - mewpmewp2 2y ago
    To me it's more like, imagine that you have read a lot of books throughout your life, but then someone comes in and asks a question from you and you try to answer from memory, but you get beaten when you say something like "I don't know", and you get rewarded if you answer accurately. You do get beaten if you answer inaccurately, but eventually you learn that if you just say something, you might just be accurate and you will not get beaten. So you just always learn to answer to the best of your knowledge, while never saying that you specifically don't know, because it decreases chances of getting beat up. You are not intentionally lying, you are just hoping that whatever you say is accurate to the best you can do according to the neural connections you've built up in your brain. Like you ask me for a birthdate of some obscure political figure from history? I'm going to try to feel out what period in history the name might feel like to me and just make my best guess based on that, then say some random year and a birthdate. It just has the lowest odds of being beaten. Was I hallucinating? No, I was just trying to not get beaten.
empath75 2y ago
> When they get things right it’s because they are lucky. This is transparently wrong. It gets so many things right in a response that the few things it gets wrong are tremendously frustrating. I think people underestimate how much correct "knowledge about the world" is expressed in a typical chat gpt response and focus only on the parts that are incorrect. If it were wrong about _everything_ at rates no better than chance, we wouldn't even be having this conversation because nobody would be using them.

by devjab 2y ago

Terr_ 2y ago

To offer a satirical analogy: "Lastly, I want to reassure investors and members of the press that we take these concerns very seriously: Hindenburg 2 will only contain only normal and unreactive hydrogen gas, and not the rare and unusual explosive kind, which is merely a temporary hurdle in this highly dynamic and growing field." Edit: It retrospect, perhaps a better analogy would involve gasoline, as its explosive nature is what's being actively being exploited in normal use.

marcosdumay 2y ago
Yes (to the edit), an analogy with making planes safer by only using non-flammable fuels is perfect.
- Terr_ 2y ago
  I expect most people have already filled in the blanks, but for completeness: "Lastly, I want to reassure investors and members of the press that we take these concerns very seriously: The Ford Pinto-II will only contain only normal and stable gasoline, and not the rare and unusual burning kind, which is merely a temporary hurdle in this highly dynamic and explos--er--fast growing field."

mewpmewp2 2y ago

But the point is, isn't hallucinating about having malformed, altered or out of touch input rather than producing inaccurate output yourself? It is the memory pathways leading them astray. It could be thought of a memory system that at certain point any longer can't be fully sure if whatever connections they have are from actually being trained or it or created accidentally.

Terr_ 2y ago
> isn't hallucinating about having malformed, altered or out of touch input rather than producing inaccurate output yourself? I suppose so, in the sense that someone could simply be lying about pink elephants instead of seeing them. However it's hard to argue that the machine knows the "right" answer and is (intelligently?) deceiving us. > It is the memory pathways leading them astray. I don't think it's a "memory" issue as much as a "they don't operate the way we like to think they do" issue. Suppose a human is asked to describe different paintings on the wall of an art gallery. Sometimes their statements appear valid and you nod along, and sometimes the statements are so wrong that it alarms you, because "this person is hallucinating." Now consider how the entire situation is flipped by finding out one additional fact... They're actually totally blind. Is it a lie? Is it a hallucination? Does it matter? Either way you must dramatically re-evaluate what their "good" outputs really mean and whether they can be used.
- mewpmewp2 2y ago
  To me it's more like, imagine that you have read a lot of books throughout your life, but then someone comes in and asks a question from you and you try to answer from memory, but you get beaten when you say something like "I don't know", and you get rewarded if you answer accurately. You do get beaten if you answer inaccurately, but eventually you learn that if you just say something, you might just be accurate and you will not get beaten. So you just always learn to answer to the best of your knowledge, while never saying that you specifically don't know, because it decreases chances of getting beat up. You are not intentionally lying, you are just hoping that whatever you say is accurate to the best you can do according to the neural connections you've built up in your brain. Like you ask me for a birthdate of some obscure political figure from history? I'm going to try to feel out what period in history the name might feel like to me and just make my best guess based on that, then say some random year and a birthdate. It just has the lowest odds of being beaten. Was I hallucinating? No, I was just trying to not get beaten.

empath75 2y ago

> When they get things right it’s because they are lucky. This is transparently wrong. It gets so many things right in a response that the few things it gets wrong are tremendously frustrating. I think people underestimate how much correct "knowledge about the world" is expressed in a typical chat gpt response and focus only on the parts that are incorrect. If it were wrong about _everything_ at rates no better than chance, we wouldn't even be having this conversation because nobody would be using them.