6 ms·

It uses AI to parse the selected text to choose search keywords, and to parse the related Wikipedia article to decide if it agrees with the selected text. It ob

It uses AI to parse the selected text to choose search keywords, and to parse the related Wikipedia article to decide if it agrees with the selected text. It obviously can bullshit in both cases.

falcor84 2y ago
Searching for keywords shouldn't be likely to hallucinate. And I would assume they would have a subsequent step to run a quick check to see they're really in the text. And if there is some issue, I suppose we can always fall back to something like TF-IDF. The second part does seem more problematic, but still, as essentially a yes/no question, it should be significantly less likely to hallucinate/confabulate than for other tasks.
- Kwpolska 2y ago
  The extension appears to also produce an explanation of its decision, so there is potential to bullshit: https://meta.wikimedia.org/wiki/Future_Audiences/Experiment:Citation_Needed https://meta.wikimedia.org/wiki/Future_Audiences/Experiment:... Also, if you look into their “wrong” example closer, it is a bit misleading, as both sources are correct. Joe Biden was 29 on election day, but 30 when he was sworn in. Understanding this requires more context than the LLM was apparently provided.
AJRF 2y ago
Honest question - do you expect there to be hallucinations in this case? I have extensive experience in LLMs and also from talking to peers with similar experience, it is uncontroversial to say that given grounding like this LLMs won't hallucinate. I am not sure if when people say this they just don't have experience building with LLMs or they do and have experience that would make for a very popular and interesting research paper.
- Kwpolska 2y ago
  The burden of the proof of no bullshit is on the LLM proponents.
  - hombre_fatal 2y ago
    Well wait a sec, you’d be just as guilty of confident bullshit if your claims above don’t pan out, and they didn’t even come from an LLM so it’s worse.
- simonw 2y ago
  "it is uncontroversial to say that given grounding like this LLMs won't hallucinate" I disagree. LLMs that have been "grounded" in text that has been injected into their context (RAG style) can absolutely stolñ hallucinate. They are less likely to, but that's not the same as saying they "won't hallucinate" at all. I've spotted this myself. It's not uncommon for example for Google Gemini to include a citation link which, when followed, doesn't support the "facts" it reported. Furthermore, if you think about how most RAG implementations work you'll spot plenty of potential for hallucination. What if the text that was pulled into the context was part of a longer paragraph that started "The following is a common misconception:" - but that prefix was omitted from the extract?

It uses AI to parse the selected text to choose search keywords, and to parse the related Wikipedia article to decide if it agrees with the selected text. It obviously can bullshit in both cases.

falcor84 2y ago

Searching for keywords shouldn't be likely to hallucinate. And I would assume they would have a subsequent step to run a quick check to see they're really in the text. And if there is some issue, I suppose we can always fall back to something like TF-IDF. The second part does seem more problematic, but still, as essentially a yes/no question, it should be significantly less likely to hallucinate/confabulate than for other tasks.

Kwpolska 2y ago
The extension appears to also produce an explanation of its decision, so there is potential to bullshit: https://meta.wikimedia.org/wiki/Future_Audiences/Experiment:Citation_Needed https://meta.wikimedia.org/wiki/Future_Audiences/Experiment:... Also, if you look into their “wrong” example closer, it is a bit misleading, as both sources are correct. Joe Biden was 29 on election day, but 30 when he was sworn in. Understanding this requires more context than the LLM was apparently provided.

AJRF 2y ago

Honest question - do you expect there to be hallucinations in this case? I have extensive experience in LLMs and also from talking to peers with similar experience, it is uncontroversial to say that given grounding like this LLMs won't hallucinate. I am not sure if when people say this they just don't have experience building with LLMs or they do and have experience that would make for a very popular and interesting research paper.

Kwpolska 2y ago
The burden of the proof of no bullshit is on the LLM proponents.
- hombre_fatal 2y ago
  Well wait a sec, you’d be just as guilty of confident bullshit if your claims above don’t pan out, and they didn’t even come from an LLM so it’s worse.
simonw 2y ago
"it is uncontroversial to say that given grounding like this LLMs won't hallucinate" I disagree. LLMs that have been "grounded" in text that has been injected into their context (RAG style) can absolutely stolñ hallucinate. They are less likely to, but that's not the same as saying they "won't hallucinate" at all. I've spotted this myself. It's not uncommon for example for Google Gemini to include a citation link which, when followed, doesn't support the "facts" it reported. Furthermore, if you think about how most RAG implementations work you'll spot plenty of potential for hallucination. What if the text that was pulled into the context was part of a longer paragraph that started "The following is a common misconception:" - but that prefix was omitted from the extract?