If you want better accuracy in the similarity search, you make RAG chunks smaller. You then need to calibrate similarity "thresholds" to know the probability di

If you want better accuracy in the similarity search, you make RAG chunks smaller. You then need to calibrate similarity "thresholds" to know the probability distribution of relevant/correct/incorrect chunks for any given sentence and what the N should be to reach them. It's not going to be perfect but you can end up with 90% accuracy on average which tends to be better than many full-text search solutions. Moreover, querying a vector DB takes <0.5s and you can run the LLM in the streaming mode getting responses pretty quickly leading to the illusion of talking to a real human (especially if you also stream audio/video with it).