RAG can't create associations to data that isn't superficially (found by the indexing strategy) assosciated to the query. For example, you might query about one

RAG can't create associations to data that isn't superficially (found by the indexing strategy) assosciated to the query. For example, you might query about one presidential candidate and lose out on the context of all other presidential candidates (probably a bad example, but gets the point across). It is "search and summarize." It is not "glean new conclusions." That being said, "search and summarize" is probably good for 80%. LoRA is an improvement, but I have seen benchmarks showing that it struggles to make as deep inferences as regular training does. There isn't a one-size fits all... Yet.