For more detail, finetuning means adjusting parameters based on a smaller, specific dataset to tailor the LLM's responses, but the model's underlying knowledge

For more detail, finetuning means adjusting parameters based on a smaller, specific dataset to tailor the LLM's responses, but the model's underlying knowledge is fixed at the point of its last training update. It adjusts how existing knowledge is used, but doesn't add new facts or information post-training. It's more about tweaking responses, biases, and style, rather than updating its factual database. RAG combines a language model like GPT with a real-time search component. This allows the model to pull in information from external sources during its response generation process. Now the ability to access and integrate the most recent information is gained, which the language model alone wouldn't have. But even with RAG, results can disappoint. RAG works best on smaller chunks of information, but the knowledge in the underlying model gets in the way of accuracy. On our knowledgebase, in an area where there is a lot of mis- and not-quite-accurate info on the internet, it regularly provides inaccurate information—is unusable. As @treprinium points out below, you also have to "calibrate similarity "thresholds" to know the probability distribution of relevant/correct/incorrect chunks for any given sentence and what the N should be to reach them. It's not going to be perfect but you can end up with 90% accuracy on average which tends to be better than many full-text search solutions"