3 ms·

Of course not. Humans "hallucinate" all the time too. Why do we hold AI to a higher standard than humans? It's the same with self driving cars. "Oh it had o

Of course not. Humans "hallucinate" all the time too. Why do we hold AI to a higher standard than humans? It's the same with self driving cars. "Oh it had one accident, must not be safe!" and yet humans have 100s of accidents a day. The main problem here is expectations -- everyone expects the machine to be perfect, and when it's not, it breaks expectations. In the past, machines were generally a lot more accurate, they just didn't do a lot of stuff. Now they they can do a lot more stuff, their accuracy is coming down to human levels, and it throws everyone off. I don't think we need to fix hallucinations, we need to fix expectations. Humans don't have 100% perfect recall of every fact they've ever learned, why do we expect AI to?

ceejayoz 2y ago
We generally send halllucinating humans to the hospital.
- jedberg 2y ago
  You must be kidding. Every day my spouse and I have a conversation that involves one of us realizing that we've mis-remembered something. Not getting every fact correct is super-common in humans. This right here is a perfect example of the problem of expectations. Do you have 100% accurate recall of every fact you've ever learned?
  - relaxing 2y ago
    Then why do I need a computer to mis-remember things for me.
    - recursive 2y ago
      Maybe you don't. A lot of people seem to find it useful though.
    - deleted 2y ago
      [deleted]
- jylam 2y ago
  Everyone hallucinates stuff, you don't need to see the Virgin Mary to hallucinate. You hallucinate stuff every day, and you are not even aware of it. Even had a "déjà vu" ? Saw your black cat in a towel in a dark room ? Every day.
- sokoloff 2y ago
  Persistent and frequent/high-percentage hallucinations? Yes, that's cause for concern/medical consultation. But we don't over-react when humans are fooled by optical illusions/magic tricks, or when they conflate two things, or when they confuse correlation and causation, or when they selectively prioritize some information over other information, or...
  - ceejayoz 2y ago
    > Persistent and frequent/high-percentage hallucinations? This describes the current state of AI, doesn't it?
    - sokoloff 2y ago
      If you're determined to find a reason to negatively judge genAI, you can find examples to fit that narrative. I think that it does not happen at a level of persistence/pervasiveness that would cause the equivalent human to be hospitalized.
soco 2y ago
I'm afraid hallucinations would be easier to fix than expectations. We are humans after all, right?
danhon 2y ago
Because a lot of the time, AI is _pitched_ (and hyped) as a higher standard than humans.
- jedberg 2y ago
  That is true, and a big part of the problem. Vendors need to stop pitching it that way.
- mckn1ght 2y ago
  > we need to fix expectations
TheAceOfHearts 2y ago
Properly calibration expectations is part of it, but surely we could do better by having the AI cite proper sources and having it show us the line of reasoning? The problem is that right now you ask an LLM a question and it'll confidently give you an incorrect answer without any sources or explanation of the reasoning behind it. Having tools that shift blame and confidently lie to you is a problem. Here's a recent example that bothered me: I tried using Meta AI to generate an image and it failed. When I asked why it failed, it kept lying and making up excuses for how it was my fault or my browser was doing something wrong. When in reality the probable explanation was that the image prompt included something that was probably flagged as sensitive content, which is why the image generation failed. (It wasn't even anything nefarious, I just tried several variations of catgirls.) If something goes against some content guidelines I want the tool to tell me that, rather than gaslight me with lies.
- jedberg 2y ago
  In most human interactions, do people site their sources? If someone doesn't want to tell you something because it makes them uncomfortable, do they tell you that or do they lie to you to make you feel better? I'm with you, it would be nice if LLMs could do those things, but also I don't expect my human friends to do that, so I'm not sure why I should expect it of my AI.
gmd63 2y ago
Because we're building something that runs autonomously at scale. Similar reason why you don't give one human complete power over the entire economy without mountains of trials/elections etc. Expecting more from AI is being cautious about wanton stereotype machines that will remove plenty of "redundant" humans from the workforce without demonstrating that they are capable of redundancy against error. If we're digesting and calcifying human knowledge, it's likely we have only one shot to get it right, so we had better do it correctly. Ask yourself if you would have been fine with calculators if they did math at an equal error rate to humans.

ceejayoz 2y ago

We generally send halllucinating humans to the hospital.

jedberg 2y ago
You must be kidding. Every day my spouse and I have a conversation that involves one of us realizing that we've mis-remembered something. Not getting every fact correct is super-common in humans. This right here is a perfect example of the problem of expectations. Do you have 100% accurate recall of every fact you've ever learned?
- relaxing 2y ago
  Then why do I need a computer to mis-remember things for me.
  - recursive 2y ago
    Maybe you don't. A lot of people seem to find it useful though.
  - deleted 2y ago
    [deleted]
jylam 2y ago
Everyone hallucinates stuff, you don't need to see the Virgin Mary to hallucinate. You hallucinate stuff every day, and you are not even aware of it. Even had a "déjà vu" ? Saw your black cat in a towel in a dark room ? Every day.
sokoloff 2y ago
Persistent and frequent/high-percentage hallucinations? Yes, that's cause for concern/medical consultation. But we don't over-react when humans are fooled by optical illusions/magic tricks, or when they conflate two things, or when they confuse correlation and causation, or when they selectively prioritize some information over other information, or...
- ceejayoz 2y ago
  > Persistent and frequent/high-percentage hallucinations? This describes the current state of AI, doesn't it?
  - sokoloff 2y ago
    If you're determined to find a reason to negatively judge genAI, you can find examples to fit that narrative. I think that it does not happen at a level of persistence/pervasiveness that would cause the equivalent human to be hospitalized.

soco 2y ago

I'm afraid hallucinations would be easier to fix than expectations. We are humans after all, right?

danhon 2y ago

Because a lot of the time, AI is _pitched_ (and hyped) as a higher standard than humans.

jedberg 2y ago
That is true, and a big part of the problem. Vendors need to stop pitching it that way.
mckn1ght 2y ago
> we need to fix expectations

TheAceOfHearts 2y ago

Properly calibration expectations is part of it, but surely we could do better by having the AI cite proper sources and having it show us the line of reasoning? The problem is that right now you ask an LLM a question and it'll confidently give you an incorrect answer without any sources or explanation of the reasoning behind it. Having tools that shift blame and confidently lie to you is a problem. Here's a recent example that bothered me: I tried using Meta AI to generate an image and it failed. When I asked why it failed, it kept lying and making up excuses for how it was my fault or my browser was doing something wrong. When in reality the probable explanation was that the image prompt included something that was probably flagged as sensitive content, which is why the image generation failed. (It wasn't even anything nefarious, I just tried several variations of catgirls.) If something goes against some content guidelines I want the tool to tell me that, rather than gaslight me with lies.

jedberg 2y ago
In most human interactions, do people site their sources? If someone doesn't want to tell you something because it makes them uncomfortable, do they tell you that or do they lie to you to make you feel better? I'm with you, it would be nice if LLMs could do those things, but also I don't expect my human friends to do that, so I'm not sure why I should expect it of my AI.

gmd63 2y ago

Because we're building something that runs autonomously at scale. Similar reason why you don't give one human complete power over the entire economy without mountains of trials/elections etc. Expecting more from AI is being cautious about wanton stereotype machines that will remove plenty of "redundant" humans from the workforce without demonstrating that they are capable of redundancy against error. If we're digesting and calcifying human knowledge, it's likely we have only one shot to get it right, so we had better do it correctly. Ask yourself if you would have been fine with calculators if they did math at an equal error rate to humans.