14 ms·

Google cautions against 'hallucinating' chatbots

46 points by DocFeind 4y ago

sys32768 4y ago
The Web is rife with content by hallucinating humans, so in that sense AI is becoming more like us.
kiawe_fire 4y ago
I’m beginning to dislike this anthropomorphization of AI, and tech in general. I feel like it’s contributing to a further misunderstanding of what it actually is and how best to use it or think of it.
- athorax 4y ago
  I think it is, at least to some extent, intentional. That way people blame the bot instead of the underlying technology when it "misbehaves"
- jeroenhd 4y ago
  I think anthropomorphization is inevitable for products that can use natural, human language to interact with a program. Talking to ChatGPT almost passes the Turing test if you ignore the way it keeps reminding you it's not a person. It may die down after it becomes clear that AI isn't good enough yet. I've seen fewer people refer to Siri/Cortana/Alexa as "her" rather than "it" as time went on, but there was definitely a peak when they became popular among laypeople for a while. I notice I'm much politer to ChatGPT than I am to Google, it's quite a funny thing to notice. Perhaps my subconscious is being nice to the robots to be in their good graces when they take over :)
- localplume 4y ago
  I think the core problem of the anthropomorphization is calling it AI; intelligence is something usually assigned to humans, so people expect it to replace/substitute a human. So instead of thinking this technology augments existing workflows, they think it will replace it. Sure, animals can be intelligent but AI/AGI/ASI is all human-centric based around a human-baseline. I don't honestly know what it should be called, but I don't think it should be AI. Maybe I'm wrong honestly. Maybe not anthropomorphizing something is impossible for humans (stick a pair of eyes and a mouth on anything and a person would probably give it a name). I guess people _expect_ AI to replace a human in a given role, but the reality is that they will augment existing workflows, e.g. instead of replacing a digital artist, it would be used to spark creativity and basically eliminate creative blocks.
  - UnpossibleJim 4y ago
    MLM... Machine Learning Model?
    - Loughla 4y ago
      I would argue that most people have no idea what that means, and it sounds an awful lot like artificial intelligence already. So. It's not a big leap from that back to artificial intelligence. I think the problem is that there is no easily accessible description of what this is. Like super simple. Statistically close to what you want responses. Big ass database of things that might be similar. I believe both of those are more accessible to Joe Schmoe, even if they miss the technical description.
      - inkcapmushroom 4y ago
        Fancy Madlibber is what I would call it. Really good at filling in blanks you give it, but no ties to the context of what it's talking about.
    - prox 4y ago
      Sounds like multi level marketing. I have simply seen large language model used the most, even by chapgpt itself.
      - UnpossibleJim 4y ago
        sadly, I had to have multi level marketing to me.
- ActionHank 4y ago
  It's annoying, but maybe it means better job security for the technically inclined by lowering the barrier to entry for most people meaning that fewer people will be looking behind the curtain.
ugjka 4y ago
How about they actually do some improvements with their search so that I don't have to reach for chatgpt when i don't feel like sifting through link after link of SEO spam and all other garbage and ads
- azemetre 4y ago
  Yeah, it's absolutely wild to me about are these anti chatGPT articles when you can just use Google itself and see it's similar quality (average to bad). SEO spam is absolutely obnoxious. I just love when I'm trying to look up a local company and the first half of the page are paid ads about local company competitors in another city/state. How useful! It's been gamed and Google has zero incentive to change it because SEO gaming is ties nicely with paying for ads which is Google's golden goose. chatGPT is such a fresh breath of air.
- noloblo 4y ago
  negative expected value to google
radarsat1 4y ago
So, yes, now that we've acknowledged that GPT is very good at making things up, for the umpteenth time in the last month, I'm curious: what are the current ideas in ML to improve on this? I assume the ideas of database access and such are in there, but what methods can be used to make the AI recognize when it is stating a fact that should be checked, and finding ways to check it?
- mistermann 4y ago
  I'm rather skeptical that a highly reliable methodology for epistemology even exists at all (outside of things like math and the physical sciences), let alone one that can be implemented by a computer.
- SQueeeeeL 4y ago
  The thing is most facts will never be "checkable" because the world we exist in is simply too complicated to be fully enumerated. You basically require a simulated mind to even begin to grasp the nuances of fact vs perspective vs opinion. It's far beyond the scope of a few companies putting a few decades and a couple upstart tech people to the problem to make any real progress on understand and producing truth, which is why ChatGPT answers every question like a middle schooler, because it's just simulating understanding.
  - mistermann 4y ago
    > It's far beyond the scope of a few companies putting a few decades vand a couple upstart tech people to the problem to make any real progress on understand and producing truth The accuracy of this fact is itself is subject to the very issue you are noting though! :)
  - dragonwriter 4y ago
    > You basically require a simulated mind to even begin to grasp the nuances of fact vs perspective vs opinion. I’ve done a little testing of this (using HN comments!) and ChatGPT seems to be reasonably good at distinguishing fact vs. opinion. It seems in that limited testing to be slightly more likely to view as opinion things I would view as likely fact claims but where the operationalization intended is ambiguous as opinion, but the boundary between a fact claim with an unstated operationalization and an opinion claim is fuzzy. That doesn’t get a lot closer to checking fact claims, or identifying and deciding when and how to report supporting evidence of them, but I don’t think that distinguishing fact claims is, itsef, as far beyond reach of current technology as you are describing it.
aliqot 4y ago
[flagged]
- NikolaNovak 4y ago
  I actually thought (as an outsider) that"ai hallucinations" is a reasonably well defined term of art, rather than something injected by an overzealous journalist in reporting. Am I wrong?
  - manimino 4y ago
    All these AIs are doing is sampling from some continuous space. Every answer is a "hallucination". AIs are useful because many (most?) of those hallucinations happen to be useful.
    - aliqot 4y ago
      It's not a hallucination. When a parrot speaks it isn't sentient. LLM's are essentially doing parrot math with much larger data sets, that is it. The fact that it sufficiently mimics what you're expecting is not a testament to its intellect.
      - inkcapmushroom 4y ago
        >When a parrot speaks it isn't sentient Parrots are certainly sentient. You could maybe say parrots aren't sapient, but frankly I think we don't know enough about sapience or what it takes to make human-level intelligence to make that claim either.
    - NikolaNovak 4y ago
      That's using "hallucination" the way humans use it. My understanding was that the term as used in the field is fairly well defined as "producing a confident answer that is not backed up / justified by training data". It has nothing to do with sentience or humanness.
      - 4y ago
sleepybrett 4y ago
I want a 'You have to tell me if you are a cop' rule for these kinds of chat/voicechat models.
1024core 4y ago
Interesting Blog post from earlier today: https://dkb.blog/p/bing-ai-cant-be-trusted https://dkb.blog/p/bing-ai-cant-be-trusted And the ensuing HN discussion has a lot more examples too.
- phoe18 4y ago
  HN Discussion: https://news.ycombinator.com/item?id=34775853 https://news.ycombinator.com/item?id=34775853
l0c0b0x 4y ago
I'd have to say, it is also very irresponsible to throw a line like 'Hallucinating Chatbots' and not provide any more insight into what that means, and/or how it applies to AI and end-users.
- gl-prod 4y ago
  We don't have AGI.
disadvantage 4y ago
Site is borked with JS disabled. Here's an archive.today of it: https://archive.fo/lZlCe https://archive.fo/lZlCe
cwkoss 4y ago
Funny how google didn't caution us about hallucinating google info boxes.
- BaseballPhysics 4y ago
  Funny how every time this topic comes up someone trots out another false equivalence.
- rtkwe 4y ago
  At least there it links you to where it's pulling the information from so you can check for yourself. The AI chatbots can't link back because they're not built for it and aren't pulling from a database of links and documents to reference back to in a 1-1 way.
  - cwkoss 4y ago
    Google will link you a source, but doesn't explain how the wrong source got linked to an infobox. I think this distinction is somewhat academic. Either system can present users with wrong information without explanation.
    - rtkwe 4y ago
      I've never seen an info box that didn't at least link to the point the data came from and if it's at all important I'll usually check to see where it's coming from to at least get a gut gauge on how correct the source seems. With the new chatbots if I'm not going to just trust them I have to do the same basic steps I would have to have found the data through a search engine to begin with.
- jmathai 4y ago
  This is a valid critique. Transparency for the source attribution of info boxes is lower than clicking to be redirected to the full web page with context. But chatbots take that a lot further...and generative AI specifically cannot really do that in a way that is easy for consumers to make use of - at least it's not possible at the moment.
dilap 4y ago
Yeah, it's definitely a problem. I still find ChatGPT massively useful, but it would be more useful if it didn't do this. Steve Hsu talks a little bit about this on the most recent episode of his podcast (1); he has a startup that aims to fix the problem, apparently with promising early results (2). (1) https://infoproc.blogspot.com/2023/02/chatgpt-llms-and-ai-manifold-29.html https://infoproc.blogspot.com/2023/02/chatgpt-llms-and-ai-ma... (2) https://twitter.com/hsu_steve/status/1623388682454732801 https://twitter.com/hsu_steve/status/1623388682454732801
Sparkyte 4y ago
IT is why this tech boom on AI needs more maturity before people so willfully adopt something they don't quite understand. Yes it can produce marvelous results but it is based on sample data that can be just as much erroranous since the source data is from humans. "To err is human.".
- Retric 4y ago
  The hallucination issue is less about training than the AI having no memory of what was included in the training or not. Ask how long a fictitious river is and you might just get an unsupported answer. Which isn’t that big a deal, but you can get the same kind of answer for real world rivers etc.
dieulot 4y ago
Can HN German speakers please confirm/rebut that “hallucination” accurately reflects what was said?
- eternalban 4y ago
  It's a technical term: Survey of Hallucination in Natural Language Generation (2022): https://arxiv.org/abs/2202.03629 https://arxiv.org/abs/2202.03629 From source: "Once again, intrinsic hallucination refers to output content that contradicts the source, while extrinsic hallucination refers to output content that the source cannot verify."
- Kenji 4y ago
  [dead]

46 points by DocFeind 4y ago

The Web is rife with content by hallucinating humans, so in that sense AI is becoming more like us.

I’m beginning to dislike this anthropomorphization of AI, and tech in general. I feel like it’s contributing to a further misunderstanding of what it actually is and how best to use it or think of it.

athorax 4y ago
I think it is, at least to some extent, intentional. That way people blame the bot instead of the underlying technology when it "misbehaves"
jeroenhd 4y ago
I think anthropomorphization is inevitable for products that can use natural, human language to interact with a program. Talking to ChatGPT almost passes the Turing test if you ignore the way it keeps reminding you it's not a person. It may die down after it becomes clear that AI isn't good enough yet. I've seen fewer people refer to Siri/Cortana/Alexa as "her" rather than "it" as time went on, but there was definitely a peak when they became popular among laypeople for a while. I notice I'm much politer to ChatGPT than I am to Google, it's quite a funny thing to notice. Perhaps my subconscious is being nice to the robots to be in their good graces when they take over :)
localplume 4y ago
I think the core problem of the anthropomorphization is calling it AI; intelligence is something usually assigned to humans, so people expect it to replace/substitute a human. So instead of thinking this technology augments existing workflows, they think it will replace it. Sure, animals can be intelligent but AI/AGI/ASI is all human-centric based around a human-baseline. I don't honestly know what it should be called, but I don't think it should be AI. Maybe I'm wrong honestly. Maybe not anthropomorphizing something is impossible for humans (stick a pair of eyes and a mouth on anything and a person would probably give it a name). I guess people _expect_ AI to replace a human in a given role, but the reality is that they will augment existing workflows, e.g. instead of replacing a digital artist, it would be used to spark creativity and basically eliminate creative blocks.
- UnpossibleJim 4y ago
  MLM... Machine Learning Model?
  - Loughla 4y ago
    I would argue that most people have no idea what that means, and it sounds an awful lot like artificial intelligence already. So. It's not a big leap from that back to artificial intelligence. I think the problem is that there is no easily accessible description of what this is. Like super simple. Statistically close to what you want responses. Big ass database of things that might be similar. I believe both of those are more accessible to Joe Schmoe, even if they miss the technical description.
    - inkcapmushroom 4y ago
      Fancy Madlibber is what I would call it. Really good at filling in blanks you give it, but no ties to the context of what it's talking about.
  - prox 4y ago
    Sounds like multi level marketing. I have simply seen large language model used the most, even by chapgpt itself.
    - UnpossibleJim 4y ago
      sadly, I had to have multi level marketing to me.
ActionHank 4y ago
It's annoying, but maybe it means better job security for the technically inclined by lowering the barrier to entry for most people meaning that fewer people will be looking behind the curtain.

ugjka 4y ago

How about they actually do some improvements with their search so that I don't have to reach for chatgpt when i don't feel like sifting through link after link of SEO spam and all other garbage and ads

azemetre 4y ago
Yeah, it's absolutely wild to me about are these anti chatGPT articles when you can just use Google itself and see it's similar quality (average to bad). SEO spam is absolutely obnoxious. I just love when I'm trying to look up a local company and the first half of the page are paid ads about local company competitors in another city/state. How useful! It's been gamed and Google has zero incentive to change it because SEO gaming is ties nicely with paying for ads which is Google's golden goose. chatGPT is such a fresh breath of air.
noloblo 4y ago
negative expected value to google

radarsat1 4y ago

So, yes, now that we've acknowledged that GPT is very good at making things up, for the umpteenth time in the last month, I'm curious: what are the current ideas in ML to improve on this? I assume the ideas of database access and such are in there, but what methods can be used to make the AI recognize when it is stating a fact that should be checked, and finding ways to check it?

mistermann 4y ago
I'm rather skeptical that a highly reliable methodology for epistemology even exists at all (outside of things like math and the physical sciences), let alone one that can be implemented by a computer.
SQueeeeeL 4y ago
The thing is most facts will never be "checkable" because the world we exist in is simply too complicated to be fully enumerated. You basically require a simulated mind to even begin to grasp the nuances of fact vs perspective vs opinion. It's far beyond the scope of a few companies putting a few decades and a couple upstart tech people to the problem to make any real progress on understand and producing truth, which is why ChatGPT answers every question like a middle schooler, because it's just simulating understanding.
- mistermann 4y ago
  > It's far beyond the scope of a few companies putting a few decades vand a couple upstart tech people to the problem to make any real progress on understand and producing truth The accuracy of this fact is itself is subject to the very issue you are noting though! :)
- dragonwriter 4y ago
  > You basically require a simulated mind to even begin to grasp the nuances of fact vs perspective vs opinion. I’ve done a little testing of this (using HN comments!) and ChatGPT seems to be reasonably good at distinguishing fact vs. opinion. It seems in that limited testing to be slightly more likely to view as opinion things I would view as likely fact claims but where the operationalization intended is ambiguous as opinion, but the boundary between a fact claim with an unstated operationalization and an opinion claim is fuzzy. That doesn’t get a lot closer to checking fact claims, or identifying and deciding when and how to report supporting evidence of them, but I don’t think that distinguishing fact claims is, itsef, as far beyond reach of current technology as you are describing it.

aliqot 4y ago

[flagged]

NikolaNovak 4y ago
I actually thought (as an outsider) that"ai hallucinations" is a reasonably well defined term of art, rather than something injected by an overzealous journalist in reporting. Am I wrong?
- manimino 4y ago
  All these AIs are doing is sampling from some continuous space. Every answer is a "hallucination". AIs are useful because many (most?) of those hallucinations happen to be useful.
  - aliqot 4y ago
    It's not a hallucination. When a parrot speaks it isn't sentient. LLM's are essentially doing parrot math with much larger data sets, that is it. The fact that it sufficiently mimics what you're expecting is not a testament to its intellect.
    - inkcapmushroom 4y ago
      >When a parrot speaks it isn't sentient Parrots are certainly sentient. You could maybe say parrots aren't sapient, but frankly I think we don't know enough about sapience or what it takes to make human-level intelligence to make that claim either.
  - NikolaNovak 4y ago
    That's using "hallucination" the way humans use it. My understanding was that the term as used in the field is fairly well defined as "producing a confident answer that is not backed up / justified by training data". It has nothing to do with sentience or humanness.
    - 4y ago

sleepybrett 4y ago

I want a 'You have to tell me if you are a cop' rule for these kinds of chat/voicechat models.

1024core 4y ago

Interesting Blog post from earlier today: https://dkb.blog/p/bing-ai-cant-be-trusted https://dkb.blog/p/bing-ai-cant-be-trusted And the ensuing HN discussion has a lot more examples too.

phoe18 4y ago
HN Discussion: https://news.ycombinator.com/item?id=34775853 https://news.ycombinator.com/item?id=34775853

l0c0b0x 4y ago

I'd have to say, it is also very irresponsible to throw a line like 'Hallucinating Chatbots' and not provide any more insight into what that means, and/or how it applies to AI and end-users.

gl-prod 4y ago
We don't have AGI.

disadvantage 4y ago

Site is borked with JS disabled. Here's an archive.today of it: https://archive.fo/lZlCe https://archive.fo/lZlCe

cwkoss 4y ago

Funny how google didn't caution us about hallucinating google info boxes.

BaseballPhysics 4y ago
Funny how every time this topic comes up someone trots out another false equivalence.
rtkwe 4y ago
At least there it links you to where it's pulling the information from so you can check for yourself. The AI chatbots can't link back because they're not built for it and aren't pulling from a database of links and documents to reference back to in a 1-1 way.
- cwkoss 4y ago
  Google will link you a source, but doesn't explain how the wrong source got linked to an infobox. I think this distinction is somewhat academic. Either system can present users with wrong information without explanation.
  - rtkwe 4y ago
    I've never seen an info box that didn't at least link to the point the data came from and if it's at all important I'll usually check to see where it's coming from to at least get a gut gauge on how correct the source seems. With the new chatbots if I'm not going to just trust them I have to do the same basic steps I would have to have found the data through a search engine to begin with.
jmathai 4y ago
This is a valid critique. Transparency for the source attribution of info boxes is lower than clicking to be redirected to the full web page with context. But chatbots take that a lot further...and generative AI specifically cannot really do that in a way that is easy for consumers to make use of - at least it's not possible at the moment.

dilap 4y ago

Yeah, it's definitely a problem. I still find ChatGPT massively useful, but it would be more useful if it didn't do this. Steve Hsu talks a little bit about this on the most recent episode of his podcast (1); he has a startup that aims to fix the problem, apparently with promising early results (2). (1) https://infoproc.blogspot.com/2023/02/chatgpt-llms-and-ai-manifold-29.html https://infoproc.blogspot.com/2023/02/chatgpt-llms-and-ai-ma... (2) https://twitter.com/hsu_steve/status/1623388682454732801 https://twitter.com/hsu_steve/status/1623388682454732801

Sparkyte 4y ago

IT is why this tech boom on AI needs more maturity before people so willfully adopt something they don't quite understand. Yes it can produce marvelous results but it is based on sample data that can be just as much erroranous since the source data is from humans. "To err is human.".

Retric 4y ago
The hallucination issue is less about training than the AI having no memory of what was included in the training or not. Ask how long a fictitious river is and you might just get an unsupported answer. Which isn’t that big a deal, but you can get the same kind of answer for real world rivers etc.

dieulot 4y ago

Can HN German speakers please confirm/rebut that “hallucination” accurately reflects what was said?

eternalban 4y ago
It's a technical term: Survey of Hallucination in Natural Language Generation (2022): https://arxiv.org/abs/2202.03629 https://arxiv.org/abs/2202.03629 From source: "Once again, intrinsic hallucination refers to output content that contradicts the source, while extrinsic hallucination refers to output content that the source cannot verify."
Kenji 4y ago
[dead]