3 ms·

Can anyone explain to me why it is not possible for LLMs to output the reference material together with the answer? In my ideal world, an AI assistant for prof

Can anyone explain to me why it is not possible for LLMs to output the reference material together with the answer? In my ideal world, an AI assistant for professional situations would rather sound like an ideal HN answer which cites the reference, like “According to the law 12346-67, you are not allowed to add glue to food. But a researcher named John Doe in Arizona conducted an experiment with non-toxic glues with success. 67% of its participants didn’t die.” 3 sentences, 3 facts. Is it a feature that LLMs are keeping for future enterprise versions of their models, or is it entirely impossible?

ipnon 2y ago
perplexity.ai does exactly this very well. It's the best way to quickly find references for a problem you're thinking about.
dragonwriter 2y ago
> Can anyone explain to me why it is not possible for LLMs to output the reference material together with the answer? It can't do so in a particularly more reliable way (in a single pass) because every piece of input data potentially contributes to every response, and there is no deterministic way to "capture" which input work was meaningfully relevant to the output. You can ask an LLM to cite sources, which it will usually obediently do, but they may be incorrect or even completely invented-for-the-response sources. If you have access to an archive of the source data, you could do something like a semantically-aware search to try to attribute the answer to a work or multiple works in the training set, and if your setup is using RAG you can have the response (possibly bypassing the LLM, so this can be 100% reliable) identify any works that were consulted during the RAG step. You can't, of course, guarantee that the response was particularly focused on the cited source, while usually with a well-setup RAG setup the response should be informed by the retrieved documents, its possible that key aspects of the response are attributable to the general training of the model (and thus some other source or sources in the model) and not based on (and potentially even contradict) the doc(s) pulled inthe course of RAG.
- Gigachad 2y ago
  Kind of like asking an artist what the source for their painting is. The source is a lifetime of viewing and practicing art, it’s not clear which pieces contributed which details to the final result.
- venusenvy47 2y ago
  CoPilot lists reference links after responses, but I have no idea if it is directly related to the LLM data. They might just be Bing results to my question.
NoGravitas 2y ago
Bing Chat provides citations for its summaries. But the relationship between the summary and the citation is often tenuous at best. Often a statement of a specific fact will cite an article that is on the same topic, but doesn't actually say anything about that specific fact.
- cl42 2y ago
  Yeah, I feel you sometimes need to provide the actual verbatim quote the LLM is referencing for the fact, otherwise it's unclear how the LLM 'interpreted' the information for you.
cl42 2y ago
You can 100% do this. We do that at my startup (https://emergingtrajectories.com/ https://emergingtrajectories.com/). We build a fact base from reliable sources or information you give us, and we have multiple layers of checks to ensure the LLM only references those facts. It is not 100% foolproof, though, and depends on how broad your universe of knowledge needs to be. If you can only cite from 3 sources or 100 facts, this is easy to verify and force... For Google, they need to have the world's information (they are a global search engine, after all) -- deciding what is true or not is incredibly complex and there are many shades of grey. I think part of this is also just eye-catching headlines -- Google searches aren't reliable either, but we've learned to live with them and incorporate them into personal workflows that work for us. LLMs are new and we're still figuring out how to do this as users (and as product builders).

ipnon 2y ago

perplexity.ai does exactly this very well. It's the best way to quickly find references for a problem you're thinking about.

dragonwriter 2y ago

> Can anyone explain to me why it is not possible for LLMs to output the reference material together with the answer? It can't do so in a particularly more reliable way (in a single pass) because every piece of input data potentially contributes to every response, and there is no deterministic way to "capture" which input work was meaningfully relevant to the output. You can ask an LLM to cite sources, which it will usually obediently do, but they may be incorrect or even completely invented-for-the-response sources. If you have access to an archive of the source data, you could do something like a semantically-aware search to try to attribute the answer to a work or multiple works in the training set, and if your setup is using RAG you can have the response (possibly bypassing the LLM, so this can be 100% reliable) identify any works that were consulted during the RAG step. You can't, of course, guarantee that the response was particularly focused on the cited source, while usually with a well-setup RAG setup the response should be informed by the retrieved documents, its possible that key aspects of the response are attributable to the general training of the model (and thus some other source or sources in the model) and not based on (and potentially even contradict) the doc(s) pulled inthe course of RAG.

Gigachad 2y ago
Kind of like asking an artist what the source for their painting is. The source is a lifetime of viewing and practicing art, it’s not clear which pieces contributed which details to the final result.
venusenvy47 2y ago
CoPilot lists reference links after responses, but I have no idea if it is directly related to the LLM data. They might just be Bing results to my question.

NoGravitas 2y ago

Bing Chat provides citations for its summaries. But the relationship between the summary and the citation is often tenuous at best. Often a statement of a specific fact will cite an article that is on the same topic, but doesn't actually say anything about that specific fact.

cl42 2y ago
Yeah, I feel you sometimes need to provide the actual verbatim quote the LLM is referencing for the fact, otherwise it's unclear how the LLM 'interpreted' the information for you.

cl42 2y ago

You can 100% do this. We do that at my startup (https://emergingtrajectories.com/ https://emergingtrajectories.com/). We build a fact base from reliable sources or information you give us, and we have multiple layers of checks to ensure the LLM only references those facts. It is not 100% foolproof, though, and depends on how broad your universe of knowledge needs to be. If you can only cite from 3 sources or 100 facts, this is easy to verify and force... For Google, they need to have the world's information (they are a global search engine, after all) -- deciding what is true or not is incredibly complex and there are many shades of grey. I think part of this is also just eye-catching headlines -- Google searches aren't reliable either, but we've learned to live with them and incorporate them into personal workflows that work for us. LLMs are new and we're still figuring out how to do this as users (and as product builders).