4 ms·

If you train a language model on a factual dataset, it will still hallucinate.

• justinclift 3y ago
  That's why I specifically said "if they can also eliminate that...". Sure it won't come from just having a good data set... they'll need to do other stuff as well. Hopefully not impossible. :)
  • skissane 3y ago
    One technique which reduces hallucinations, is to augment the LLM with a database of facts (such as the text of an encyclopaedia). You take the prompt, and you do a semantic search on the fact database (RAG, retrieval augmented generation) to find sentences/paragraphs/documents which relate to the prompt, and you add them to the prompt. Another related approach is function-calling, where instead of just doing a query based on the text of the prompt, you let the LLM analyse the prompt and come up with search terms, and then allow the LLM to do a search for the terms it has chosen. You can prompt and fine-tune the LLM to cite all its claims to the fact database. Another approach that can help, is after you generate the response, you then submit the response to an LLM (even the same LLM), with a prompt to check it for errors (errors of reasoning, misrepresentation of citations, etc). Often, LLMs can do a decent job on catching their own errors, including hallucinations. The more advanced an LLM is, the less likely it is to hallucinate, and the more likely it is to pick up on its own mistakes. Llama-7b hallucinates a lot more than GPT-4, and is far less likely to detect itself doing so. You can't make an LLM foolproof, but you can't make humans foolproof either. Humans "hallucinate" too – e.g. students write essays with erroneous claims, cited to sources which never actually make those claims.
    • justinclift 3y ago
      > You can't make an LLM foolproof ... Heh, we need "post-LLM" AI's then that don't hallucinate at all. ;)
    • dragonwriter 3y ago
      > One technique which reduces hallucinations, is to augment the LLM with a database of facts (such as the text of an encyclopaedia). You take the prompt, and you do a semantic search on the fact database (RAG, retrieval augmented generation) to find sentences/paragraphs/documents which relate to the prompt, and you add them to the prompt. Another related approach is function-calling, where instead of just doing a query based on the text of the prompt, you let the LLM analyse the prompt and come up with search terms, and then allow the LLM to do a search for the terms it has chosen. You can prompt and fine-tune the LLM to cite all its claims to the fact database. Note that these are really very closely related, almost to the point of being slight variations on a single technique: both are “do a search on a database and include the results in a prompt to the LLM”; they differ in one uses something outside of the LLM to calculate the search query (typically, just an embedding of the user prompt used to search a vector DB), the other prompts the LLM so that it produces an appropriate search query.
      • skissane 3y ago
        > Note that these are really very closely related, almost to the point of being slight variations on a single technique: both are “do a search on a database and include the results in a prompt to the LLM”; they differ in one uses something outside of the LLM to calculate the search query (typically, just an embedding of the user prompt used to search a vector DB), the other prompts the LLM so that it produces an appropriate search query. They can produce rather different results – the LLM can do a better job than just an embedding at picking out what's truly important in the prompt, which can increase the relevance of the returned documents. I've seen before where a vector search returns documents which contain some similar language to the prompt but which are actually talking about something completely unrelated. LLMs are less likely to do that sort of thing, because they have a much better understanding of what's important in the prompt and what's just fluff/waffle