2 ms·
> One technique which reduces hallucinations, is to augment the LLM with a database of facts (such as the text of an encyclopaedia). You take the prompt, and yo
by dragonwriter 3y ago
> One technique which reduces hallucinations, is to augment the LLM with a database of facts (such as the text of an encyclopaedia). You take the prompt, and you do a semantic search on the fact database (RAG, retrieval augmented generation) to find sentences/paragraphs/documents which relate to the prompt, and you add them to the prompt. Another related approach is function-calling, where instead of just doing a query based on the text of the prompt, you let the LLM analyse the prompt and come up with search terms, and then allow the LLM to do a search for the terms it has chosen. You can prompt and fine-tune the LLM to cite all its claims to the fact database.
Note that these are really very closely related, almost to the point of being slight variations on a single technique: both are “do a search on a database and include the results in a prompt to the LLM”; they differ in one uses something outside of the LLM to calculate the search query (typically, just an embedding of the user prompt used to search a vector DB), the other prompts the LLM so that it produces an appropriate search query.
- skissane 3y ago> Note that these are really very closely related, almost to the point of being slight variations on a single technique: both are “do a search on a database and include the results in a prompt to the LLM”; they differ in one uses something outside of the LLM to calculate the search query (typically, just an embedding of the user prompt used to search a vector DB), the other prompts the LLM so that it produces an appropriate search query. They can produce rather different results – the LLM can do a better job than just an embedding at picking out what's truly important in the prompt, which can increase the relevance of the returned documents. I've seen before where a vector search returns documents which contain some similar language to the prompt but which are actually talking about something completely unrelated. LLMs are less likely to do that sort of thing, because they have a much better understanding of what's important in the prompt and what's just fluff/waffle