7 ms·

Actually there was a Wikipedia article of this name, but it was deleted in June -- because it was AI generated. Unfortunately AI falls for this much like human

Actually there was a Wikipedia article of this name, but it was deleted in June -- because it was AI generated. Unfortunately AI falls for this much like humans do. https://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletion/European_Union_Enlargement_Goals_for_2040 https://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletio...

Workaccount2 1y ago
This is likely because of the knowledge cutoff. I have seen a few cases before of "hallucinations" that turned out to be things that did exist, but no longer do.
- 1980phipsi 1y ago
  The fix for this is for the AI to double-check all links before providing them to the user. I frequently ask ChatGPT to double check that references actually exist when it gives me them. It should be built in!
  - rideontime 1y ago
    But that would mean OpenAI would lose even more money on every query.
    - mdhb 1y ago
      Almost as though it’s not a sustainable business model and relies of tricking people in order to keep the lights on.
    - ModernMech 1y ago
      Better make each query count then.
  - blitzar 1y ago
    I have found my self doing the same "citation needed" loop - but with ai this is a dangerous game as it will now double down on whatever it made up and go looking for citations to justify its answer. Pre prompting to cite sources is obviously a better way of going about things.
    - deleted 1y ago
      [deleted]
  - janwl 1y ago
    I thought people here hated it when LLMs made http requests?
    - macintux 1y ago
      I don't know for certain what you're referring to, but the "bulk downloads" of the Internet that AI companies are executing for training are the problem I've seen cited, and doesn't relate to LLMs checking their sources at query time.
    - zahlman 1y ago
      It's bad when they indiscriminately crawl for training, and not ideal (but understandable) to use the Internet to communicate with them (and having online accounts associated with that etc.) rather than running them locally. It's not bad when they use the Internet at generation time to verify the output.
      - Dylan16807 1y ago
        Also for the most part this verification can use a HEAD request.
  - dingnuts 1y ago
    Gemini will lie to me when I ask it to cite things, either pull up relevant sources or just hallucinate them. IDK how you people go through that experience more than a handful of times before you get pissed off and stop using these tools. I've wasted so much time because of believable lies from these bots. Sorry, not even lies, just bullshit. The model has no conception of truth so it can't even lie. Just outputs bullshit that happens to be true sometimes.
bunderbunder 1y ago
The biggest problem with that citation isn't that the article has since been deleted. The biggest problem is that that particular Wikipedia article was never a good source in the first place. That seems to be the real challenge with AI for this use case. It has no real critical thinking skills, so it's not really competent to choose reliable sources. So instead we're lowering the bar to just asking that the sources actually exist. I really hate that. We shouldn't be lowering intellectual standards to meet AI where it's at. These intellectual standards are important and hard-won, and we need to be demanding that AI be the one to rise to meet them.
- gamerDude 1y ago
  I think this is a real challenge for everyone. In many ways potentially we need a restart of a wikipedia like site to document all the valid and good sources. This would also hopefully include things like source bias and whether it's a primary/secondary/tertiary source.
  - fullofideas 1y ago
    This is pushing the burden of proof on the society. Basically, asking everyone else to pitch in and improve sources so that ai companies can reference these trust worthy sources.
  - bunderbunder 1y ago
    Outsourcing due diligence to a tool (or a single unified source) is the problem, not the solution. For example, having a single central arbiter of source bias is inescapably the most biased thing you could possibly do. Bias has to be defined within an intellectual paradigm. So you'd have to choose a paradigm to use for that bias evaluation, and de facto declare it to be the one true paradigm for this purpose. But intellectual paradigms are inherently subjective, so doing that is pretty much the most intellectually biased thing you can possibly do.
  - ishtanbul 1y ago
    Maybe we can get AI to do this hard labor
CaptainOfCoit 1y ago
> Actually there was a Wikipedia article of this name, but it was deleted in June -- because it was AI generated. Unfortunately AI falls for this much like humans do. A recent Kurzgesagt goes into the dangers of this, and they found the same thing happening with a concrete example: They were researching a topic, tried using LLMs, found they weren't accurate enough and hallucinated, so they continued doing things the manual way. Then some weeks/months later, they noticed a bunch of YouTube videos that had the very hallucinations they were avoiding, and now their own AI assistants started to use those as sources. Paraphrased/remembered by me, could have some inconsistencies/hallucinations. https://www.youtube.com/watch?v=_zfN9wnPvU0 https://www.youtube.com/watch?v=_zfN9wnPvU0
AlienRobot 1y ago
AI-powered citogenesis!

Workaccount2 1y ago

This is likely because of the knowledge cutoff. I have seen a few cases before of "hallucinations" that turned out to be things that did exist, but no longer do.

1980phipsi 1y ago
The fix for this is for the AI to double-check all links before providing them to the user. I frequently ask ChatGPT to double check that references actually exist when it gives me them. It should be built in!
- rideontime 1y ago
  But that would mean OpenAI would lose even more money on every query.
  - mdhb 1y ago
    Almost as though it’s not a sustainable business model and relies of tricking people in order to keep the lights on.
  - ModernMech 1y ago
    Better make each query count then.
- blitzar 1y ago
  I have found my self doing the same "citation needed" loop - but with ai this is a dangerous game as it will now double down on whatever it made up and go looking for citations to justify its answer. Pre prompting to cite sources is obviously a better way of going about things.
  - deleted 1y ago
    [deleted]
- janwl 1y ago
  I thought people here hated it when LLMs made http requests?
  - macintux 1y ago
    I don't know for certain what you're referring to, but the "bulk downloads" of the Internet that AI companies are executing for training are the problem I've seen cited, and doesn't relate to LLMs checking their sources at query time.
  - zahlman 1y ago
    It's bad when they indiscriminately crawl for training, and not ideal (but understandable) to use the Internet to communicate with them (and having online accounts associated with that etc.) rather than running them locally. It's not bad when they use the Internet at generation time to verify the output.
    - Dylan16807 1y ago
      Also for the most part this verification can use a HEAD request.
- dingnuts 1y ago
  Gemini will lie to me when I ask it to cite things, either pull up relevant sources or just hallucinate them. IDK how you people go through that experience more than a handful of times before you get pissed off and stop using these tools. I've wasted so much time because of believable lies from these bots. Sorry, not even lies, just bullshit. The model has no conception of truth so it can't even lie. Just outputs bullshit that happens to be true sometimes.

bunderbunder 1y ago

The biggest problem with that citation isn't that the article has since been deleted. The biggest problem is that that particular Wikipedia article was never a good source in the first place. That seems to be the real challenge with AI for this use case. It has no real critical thinking skills, so it's not really competent to choose reliable sources. So instead we're lowering the bar to just asking that the sources actually exist. I really hate that. We shouldn't be lowering intellectual standards to meet AI where it's at. These intellectual standards are important and hard-won, and we need to be demanding that AI be the one to rise to meet them.

gamerDude 1y ago
I think this is a real challenge for everyone. In many ways potentially we need a restart of a wikipedia like site to document all the valid and good sources. This would also hopefully include things like source bias and whether it's a primary/secondary/tertiary source.
- fullofideas 1y ago
  This is pushing the burden of proof on the society. Basically, asking everyone else to pitch in and improve sources so that ai companies can reference these trust worthy sources.
- bunderbunder 1y ago
  Outsourcing due diligence to a tool (or a single unified source) is the problem, not the solution. For example, having a single central arbiter of source bias is inescapably the most biased thing you could possibly do. Bias has to be defined within an intellectual paradigm. So you'd have to choose a paradigm to use for that bias evaluation, and de facto declare it to be the one true paradigm for this purpose. But intellectual paradigms are inherently subjective, so doing that is pretty much the most intellectually biased thing you can possibly do.
- ishtanbul 1y ago
  Maybe we can get AI to do this hard labor

CaptainOfCoit 1y ago

> Actually there was a Wikipedia article of this name, but it was deleted in June -- because it was AI generated. Unfortunately AI falls for this much like humans do. A recent Kurzgesagt goes into the dangers of this, and they found the same thing happening with a concrete example: They were researching a topic, tried using LLMs, found they weren't accurate enough and hallucinated, so they continued doing things the manual way. Then some weeks/months later, they noticed a bunch of YouTube videos that had the very hallucinations they were avoiding, and now their own AI assistants started to use those as sources. Paraphrased/remembered by me, could have some inconsistencies/hallucinations. https://www.youtube.com/watch?v=_zfN9wnPvU0 https://www.youtube.com/watch?v=_zfN9wnPvU0

AlienRobot 1y ago

AI-powered citogenesis!