3 ms·

RAG is basically good old information retrieval with LLMs doing the querying. This can include vector search but it works without that as well. Treating vector

RAG is basically good old information retrieval with LLMs doing the querying. This can include vector search but it works without that as well. Treating vector search as magic pixie dust that makes search great without effort is not necessarily going to work that well. Also, it can add a lot of cost and complexity to the equation. And if not tuned properly, you don't necessarily get good results. The key thing with RAG is to get the right information in the context with as few queries as possible. That requires good recall (ensuring that if it is there it can be found with a reasonable query) and precision (ensuring the best stuff is on top and minimizing false positives). With search, and by extension RAG, the principle of shit in, shit out applies. Most of what search teams did before AI and RAG is still the best way to optimize the experience with RAG. And if you mess that up, search is not going to be working that well and no amount of AI can compensate for that or only at great cost in tokens and time. So, having an ETL pipeline to pre-process what you index, testing & benchmarking search quality, etc. are all helpful. The good news is that you don't need that much skills with agentic coding to build something half decent for this. This code almost writes itself. And even a little bit of effort on extracting structure before indexing can make a big difference.

dmix 2mo ago
> With search, and by extension RAG, the principle of shit in, shit out applies Similar to SEO on marketing pages, we started rewriting product docs around the idea that it will be consumed by a RAG. Mostly by putting a lot of focus on well structured headlines, thinking more carefully about technical terminology vs common human-language questions, occasionally using variations of keywords in the text, etc. This applies to pure LLM consumption too, not just hybrid search. Once you start tracking what users are asking you learn to adapt the documentation around it. And LLMs can also suggest improvements by comparing questions vs search results vs LLM responses.
- jillesvangurp 2mo ago
  It's a start. Where it gets tricky is companies with years/decades of highly unstructured data, duplicated documents, obsolete or draft versions of those documents, etc. And where it gets more tricky if the data is spread all over the place in weird tools, databases, spreadsheets, etc. that has some structure but is maybe a bit inconsistent, incomplete, or not that well documented. If you flatten all that into plain text and then create embeddings, you are effectively throwing out the baby with the bathwater. But on the other hand if you put some effort into normalizing and extracting some structured meta data, you gain a flexibility to do more sophisticated querying that get you more precise results. You can of course try to fix things at the source, which is a valid thing but usually not that practical when you have a lot of data to worry about.
IanCal 2mo ago
> RAG is basically good old information retrieval with LLMs doing the querying. No - rag is doing search before you call the llm to give it context from some corpus like your helpdesk articles.
- jillesvangurp 2mo ago
  That's a very narrow definition that might have been accurate a few years ago. But of course people mostly use things like chat gpt or claude which can use tools these days, including any search tools you make available as well as web searches. Which means they typically may do multiple searches that don't just send exactly what you type but instead interpret it, adapt to what comes back and then iterate to formulate refined queries, query different search tools you might have, etc. You can still do RAG in the narrow sense you suggest but it's probably not that optimal for answering real world questions.
  - IanCal 2mo ago
    It’s the type of rag the article is about however, it’s still very relevant.

dmix 2mo ago

> With search, and by extension RAG, the principle of shit in, shit out applies Similar to SEO on marketing pages, we started rewriting product docs around the idea that it will be consumed by a RAG. Mostly by putting a lot of focus on well structured headlines, thinking more carefully about technical terminology vs common human-language questions, occasionally using variations of keywords in the text, etc. This applies to pure LLM consumption too, not just hybrid search. Once you start tracking what users are asking you learn to adapt the documentation around it. And LLMs can also suggest improvements by comparing questions vs search results vs LLM responses.

jillesvangurp 2mo ago
It's a start. Where it gets tricky is companies with years/decades of highly unstructured data, duplicated documents, obsolete or draft versions of those documents, etc. And where it gets more tricky if the data is spread all over the place in weird tools, databases, spreadsheets, etc. that has some structure but is maybe a bit inconsistent, incomplete, or not that well documented. If you flatten all that into plain text and then create embeddings, you are effectively throwing out the baby with the bathwater. But on the other hand if you put some effort into normalizing and extracting some structured meta data, you gain a flexibility to do more sophisticated querying that get you more precise results. You can of course try to fix things at the source, which is a valid thing but usually not that practical when you have a lot of data to worry about.

IanCal 2mo ago

> RAG is basically good old information retrieval with LLMs doing the querying. No - rag is doing search before you call the llm to give it context from some corpus like your helpdesk articles.

jillesvangurp 2mo ago
That's a very narrow definition that might have been accurate a few years ago. But of course people mostly use things like chat gpt or claude which can use tools these days, including any search tools you make available as well as web searches. Which means they typically may do multiple searches that don't just send exactly what you type but instead interpret it, adapt to what comes back and then iterate to formulate refined queries, query different search tools you might have, etc. You can still do RAG in the narrow sense you suggest but it's probably not that optimal for answering real world questions.
- IanCal 2mo ago
  It’s the type of rag the article is about however, it’s still very relevant.