What will happen as inference cost goes down is RAG will just be one master LLM calling a bunch of smaller LLMs spanning the context window of every document yo

What will happen as inference cost goes down is RAG will just be one master LLM calling a bunch of smaller LLMs spanning the context window of every document you are querying. Context windows went from 8k to like 128k or more in like a year. In a few years we will have practically unlimited context windows for minimal cost.