25 ms·

There is a confusion between hallucination and lack of information. RAGs work by finding the top N answers given some query, and then, based on this information

by _pdp_ 3y ago

There is a confusion between hallucination and lack of information. RAGs work by finding the top N answers given some query, and then, based on this information, the underlying model tries to make up some text. This is unreasonable, and it will only work out of the box for the most straightforward use cases - i.e., chatting with a single document, but there are limitations. So, I am not surprised that Copilot cannot answer information that is readily available in the right place. It is also unlikely that one can find that document, among many others, on the first try unless one is quoting a specific phrase. The fundamental architecture needs to change. Copilot needs to act more like an agent - i.e., perform multi-step research to find this information and do it fast.

zacmps 3y ago
It's possible to build RAG pipelines that support answering complex questions over multiple documents, and I don't think I would say the whole architecture needs to change. Doing it fast is another story, LLMs are pretty high latency at the moment.
- _pdp_ 3y ago
  If you have any examples I can go through it will be greatly appreciated. My main concern is that all RAGs just look for the top N best matching results. That does not mean that the information is in there.
  - lmeyerov 3y ago
    I'm not sure why all RAG implementations would do that? Ex: Louie.AI will use a combination of tools to answer a question (database queries, RAG vector index lookups, Python, interactive charts, ...), and will often do multiple attempts on the same tool until it decides it has exhausted the immediate use of that tool. Ex: Louie is even learning from usage, so as an analyst, if Louie stopped digging early and you decided to manually look further, Louie learns from this: It'll know in future sessions that it may be worth looking further in that kind of scenario. None of that, in isolation, is unique to Louie.AI: It's just part of what it means to do a 'full' agent implementation vs a langchain/llamaindex/openai wrapper. There's an interesting question around knowledge graph style questions here -- should an agent do iterative top-n vector similarity searches, or a single wider search over a knowledge graph, or maybe the documents should be combined into a knowledge graph and that's what's embedded. We're exploring a lot here in our bigger customer projects, and I can't say there's a clear universal answer...
    - davedx 3y ago
      Yup, it depends on the type of data being searched for and how the information is represented in the corpus. It also depends on how information is distributed: IME often longer legal documents have a lot of cross referencing, so you need multiple clauses to collate a single answer. It’s an interesting area.
  - treprinum 3y ago
    If you want better accuracy in the similarity search, you make RAG chunks smaller. You then need to calibrate similarity "thresholds" to know the probability distribution of relevant/correct/incorrect chunks for any given sentence and what the N should be to reach them. It's not going to be perfect but you can end up with 90% accuracy on average which tends to be better than many full-text search solutions. Moreover, querying a vector DB takes <0.5s and you can run the LLM in the streaming mode getting responses pretty quickly leading to the illusion of talking to a real human (especially if you also stream audio/video with it).
wodenokoto 3y ago
The setup that MS supplies as a one click solution on Azure [1] splits documents by page and stores a vector on the recently renamed to Azure AI Search services. From there on they use a special API that comes with Azure Open AI deployed GPT models that will look up using either cognitive search service or vector search. That API is a black box, so either they use user message or they have the LLM write a search query. I would assume 365 uses basically the same architecture. [1] if you do the one click deployment and look through the deployed web app you will see it is based of this GitHub repo: https://github.com/microsoft/sample-app-aoai-chatGPT https://github.com/microsoft/sample-app-aoai-chatGPT
dbuxton 3y ago
I don’t agree. RAG works just fine for this use case if you have a decent chunking methodology, good prompt design and some secondary safety rails. We do just this in our product (https://hrharriet.com https://hrharriet.com). The challenge Microsoft have set themselves however is that their problem domain is very large. They have to accommodate many different types of queries. We do benefit from having a specific type of usage pattern and much smaller data than a “real” Onedrive. That said I am baffled that this product has performed as badly as it did in the review. These seem like pre-alpha type issues. I guess we should congratulate MS for being willing to put these things out to test at early stage but… come on!
- rmccue 3y ago
  As an aside, I think your product site could be a bit more precise on some of the language; your homepage states "Harriet never trains on your data.", but your FAQ states "She is powered by LLMs [...] but trained on your company info" and "Harriet is trained on your documents" :) I assume what you're meaning is that the generic model itself is not trained on company data, but is using retrieval techniques with the actual company data when used.
andybak 3y ago
> RAG Retrieval Augmented Generation (apparently) It's really helpful to define acronyms. HN has a broad readership - I follow AI and I had never come across this one.
- red-iron-pine 3y ago
  thank you. i'm doing a Master's at a legit engineering school, and literally just had an assignment to survey AI and its underlaying tech, and even then I hadn't heard this acronym.
  - QuantumGood 3y ago
    For more detail, finetuning means adjusting parameters based on a smaller, specific dataset to tailor the LLM's responses, but the model's underlying knowledge is fixed at the point of its last training update. It adjusts how existing knowledge is used, but doesn't add new facts or information post-training. It's more about tweaking responses, biases, and style, rather than updating its factual database. RAG combines a language model like GPT with a real-time search component. This allows the model to pull in information from external sources during its response generation process. Now the ability to access and integrate the most recent information is gained, which the language model alone wouldn't have. But even with RAG, results can disappoint. RAG works best on smaller chunks of information, but the knowledge in the underlying model gets in the way of accuracy. On our knowledgebase, in an area where there is a lot of mis- and not-quite-accurate info on the internet, it regularly provides inaccurate information—is unusable. As @treprinium points out below, you also have to "calibrate similarity "thresholds" to know the probability distribution of relevant/correct/incorrect chunks for any given sentence and what the N should be to reach them. It's not going to be perfect but you can end up with 90% accuracy on average which tends to be better than many full-text search solutions"
  - wenc 3y ago
    Check the sibling comment for a primer on LLM concepts. I think if you survey AI at a high level you won’t encounter the term. But if you survey or keep track with LLMs it’s an important concept that is widely known and discussed. It’s very important to know about because it’s one of the few practical techniques in LLMs.

by _pdp_ 3y ago

zacmps 3y ago

It's possible to build RAG pipelines that support answering complex questions over multiple documents, and I don't think I would say the whole architecture needs to change. Doing it fast is another story, LLMs are pretty high latency at the moment.

_pdp_ 3y ago
If you have any examples I can go through it will be greatly appreciated. My main concern is that all RAGs just look for the top N best matching results. That does not mean that the information is in there.
- lmeyerov 3y ago
  I'm not sure why all RAG implementations would do that? Ex: Louie.AI will use a combination of tools to answer a question (database queries, RAG vector index lookups, Python, interactive charts, ...), and will often do multiple attempts on the same tool until it decides it has exhausted the immediate use of that tool. Ex: Louie is even learning from usage, so as an analyst, if Louie stopped digging early and you decided to manually look further, Louie learns from this: It'll know in future sessions that it may be worth looking further in that kind of scenario. None of that, in isolation, is unique to Louie.AI: It's just part of what it means to do a 'full' agent implementation vs a langchain/llamaindex/openai wrapper. There's an interesting question around knowledge graph style questions here -- should an agent do iterative top-n vector similarity searches, or a single wider search over a knowledge graph, or maybe the documents should be combined into a knowledge graph and that's what's embedded. We're exploring a lot here in our bigger customer projects, and I can't say there's a clear universal answer...
  - davedx 3y ago
    Yup, it depends on the type of data being searched for and how the information is represented in the corpus. It also depends on how information is distributed: IME often longer legal documents have a lot of cross referencing, so you need multiple clauses to collate a single answer. It’s an interesting area.
- treprinum 3y ago
  If you want better accuracy in the similarity search, you make RAG chunks smaller. You then need to calibrate similarity "thresholds" to know the probability distribution of relevant/correct/incorrect chunks for any given sentence and what the N should be to reach them. It's not going to be perfect but you can end up with 90% accuracy on average which tends to be better than many full-text search solutions. Moreover, querying a vector DB takes <0.5s and you can run the LLM in the streaming mode getting responses pretty quickly leading to the illusion of talking to a real human (especially if you also stream audio/video with it).

wodenokoto 3y ago

The setup that MS supplies as a one click solution on Azure [1] splits documents by page and stores a vector on the recently renamed to Azure AI Search services. From there on they use a special API that comes with Azure Open AI deployed GPT models that will look up using either cognitive search service or vector search. That API is a black box, so either they use user message or they have the LLM write a search query. I would assume 365 uses basically the same architecture. [1] if you do the one click deployment and look through the deployed web app you will see it is based of this GitHub repo: https://github.com/microsoft/sample-app-aoai-chatGPT https://github.com/microsoft/sample-app-aoai-chatGPT

dbuxton 3y ago

I don’t agree. RAG works just fine for this use case if you have a decent chunking methodology, good prompt design and some secondary safety rails. We do just this in our product (https://hrharriet.com https://hrharriet.com). The challenge Microsoft have set themselves however is that their problem domain is very large. They have to accommodate many different types of queries. We do benefit from having a specific type of usage pattern and much smaller data than a “real” Onedrive. That said I am baffled that this product has performed as badly as it did in the review. These seem like pre-alpha type issues. I guess we should congratulate MS for being willing to put these things out to test at early stage but… come on!

rmccue 3y ago
As an aside, I think your product site could be a bit more precise on some of the language; your homepage states "Harriet never trains on your data.", but your FAQ states "She is powered by LLMs [...] but trained on your company info" and "Harriet is trained on your documents" :) I assume what you're meaning is that the generic model itself is not trained on company data, but is using retrieval techniques with the actual company data when used.

andybak 3y ago

> RAG Retrieval Augmented Generation (apparently) It's really helpful to define acronyms. HN has a broad readership - I follow AI and I had never come across this one.

red-iron-pine 3y ago
thank you. i'm doing a Master's at a legit engineering school, and literally just had an assignment to survey AI and its underlaying tech, and even then I hadn't heard this acronym.
- QuantumGood 3y ago
  For more detail, finetuning means adjusting parameters based on a smaller, specific dataset to tailor the LLM's responses, but the model's underlying knowledge is fixed at the point of its last training update. It adjusts how existing knowledge is used, but doesn't add new facts or information post-training. It's more about tweaking responses, biases, and style, rather than updating its factual database. RAG combines a language model like GPT with a real-time search component. This allows the model to pull in information from external sources during its response generation process. Now the ability to access and integrate the most recent information is gained, which the language model alone wouldn't have. But even with RAG, results can disappoint. RAG works best on smaller chunks of information, but the knowledge in the underlying model gets in the way of accuracy. On our knowledgebase, in an area where there is a lot of mis- and not-quite-accurate info on the internet, it regularly provides inaccurate information—is unusable. As @treprinium points out below, you also have to "calibrate similarity "thresholds" to know the probability distribution of relevant/correct/incorrect chunks for any given sentence and what the N should be to reach them. It's not going to be perfect but you can end up with 90% accuracy on average which tends to be better than many full-text search solutions"
- wenc 3y ago
  Check the sibling comment for a primer on LLM concepts. I think if you survey AI at a high level you won’t encounter the term. But if you survey or keep track with LLMs it’s an important concept that is widely known and discussed. It’s very important to know about because it’s one of the few practical techniques in LLMs.