3 ms·

I wonder how this will go over politically with the wikipedia community. AI is such a hot button issue, and the risk of hallucinating and saying something wrong

I wonder how this will go over politically with the wikipedia community. AI is such a hot button issue, and the risk of hallucinating and saying something wrong seems more pressing in this application then most.

jszymborski 3y ago
I haven't used this, but reading the description, it sounds like it's primarily a search engine for wiki articles related to selected text. If so, I imagine it wouldn't be super susceptible to hallucinations.
- Kwpolska 3y ago
  It uses AI to parse the selected text to choose search keywords, and to parse the related Wikipedia article to decide if it agrees with the selected text. It obviously can bullshit in both cases.
  - falcor84 3y ago
    Searching for keywords shouldn't be likely to hallucinate. And I would assume they would have a subsequent step to run a quick check to see they're really in the text. And if there is some issue, I suppose we can always fall back to something like TF-IDF. The second part does seem more problematic, but still, as essentially a yes/no question, it should be significantly less likely to hallucinate/confabulate than for other tasks.
    - Kwpolska 3y ago
      The extension appears to also produce an explanation of its decision, so there is potential to bullshit: https://meta.wikimedia.org/wiki/Future_Audiences/Experiment:Citation_Needed https://meta.wikimedia.org/wiki/Future_Audiences/Experiment:... Also, if you look into their “wrong” example closer, it is a bit misleading, as both sources are correct. Joe Biden was 29 on election day, but 30 when he was sworn in. Understanding this requires more context than the LLM was apparently provided.
  - AJRF 3y ago
    Honest question - do you expect there to be hallucinations in this case? I have extensive experience in LLMs and also from talking to peers with similar experience, it is uncontroversial to say that given grounding like this LLMs won't hallucinate. I am not sure if when people say this they just don't have experience building with LLMs or they do and have experience that would make for a very popular and interesting research paper.
    - Kwpolska 3y ago
      The burden of the proof of no bullshit is on the LLM proponents.
      - hombre_fatal 3y ago
        Well wait a sec, you’d be just as guilty of confident bullshit if your claims above don’t pan out, and they didn’t even come from an LLM so it’s worse.
    - simonw 3y ago
      "it is uncontroversial to say that given grounding like this LLMs won't hallucinate" I disagree. LLMs that have been "grounded" in text that has been injected into their context (RAG style) can absolutely stolñ hallucinate. They are less likely to, but that's not the same as saying they "won't hallucinate" at all. I've spotted this myself. It's not uncommon for example for Google Gemini to include a citation link which, when followed, doesn't support the "facts" it reported. Furthermore, if you think about how most RAG implementations work you'll spot plenty of potential for hallucination. What if the text that was pulled into the context was part of a longer paragraph that started "The following is a common misconception:" - but that prefix was omitted from the extract?
deleted 3y ago
[deleted]

jszymborski 3y ago

I haven't used this, but reading the description, it sounds like it's primarily a search engine for wiki articles related to selected text. If so, I imagine it wouldn't be super susceptible to hallucinations.

Kwpolska 3y ago
It uses AI to parse the selected text to choose search keywords, and to parse the related Wikipedia article to decide if it agrees with the selected text. It obviously can bullshit in both cases.
- falcor84 3y ago
  Searching for keywords shouldn't be likely to hallucinate. And I would assume they would have a subsequent step to run a quick check to see they're really in the text. And if there is some issue, I suppose we can always fall back to something like TF-IDF. The second part does seem more problematic, but still, as essentially a yes/no question, it should be significantly less likely to hallucinate/confabulate than for other tasks.
  - Kwpolska 3y ago
    The extension appears to also produce an explanation of its decision, so there is potential to bullshit: https://meta.wikimedia.org/wiki/Future_Audiences/Experiment:Citation_Needed https://meta.wikimedia.org/wiki/Future_Audiences/Experiment:... Also, if you look into their “wrong” example closer, it is a bit misleading, as both sources are correct. Joe Biden was 29 on election day, but 30 when he was sworn in. Understanding this requires more context than the LLM was apparently provided.
- AJRF 3y ago
  Honest question - do you expect there to be hallucinations in this case? I have extensive experience in LLMs and also from talking to peers with similar experience, it is uncontroversial to say that given grounding like this LLMs won't hallucinate. I am not sure if when people say this they just don't have experience building with LLMs or they do and have experience that would make for a very popular and interesting research paper.
  - Kwpolska 3y ago
    The burden of the proof of no bullshit is on the LLM proponents.
    - hombre_fatal 3y ago
      Well wait a sec, you’d be just as guilty of confident bullshit if your claims above don’t pan out, and they didn’t even come from an LLM so it’s worse.
  - simonw 3y ago
    "it is uncontroversial to say that given grounding like this LLMs won't hallucinate" I disagree. LLMs that have been "grounded" in text that has been injected into their context (RAG style) can absolutely stolñ hallucinate. They are less likely to, but that's not the same as saying they "won't hallucinate" at all. I've spotted this myself. It's not uncommon for example for Google Gemini to include a citation link which, when followed, doesn't support the "facts" it reported. Furthermore, if you think about how most RAG implementations work you'll spot plenty of potential for hallucination. What if the text that was pulled into the context was part of a longer paragraph that started "The following is a common misconception:" - but that prefix was omitted from the extract?

deleted 3y ago

[deleted]