It's hard to imagine what could happen instead. Even with a model with infinite context, where we imagine you could supply e.g. your entire email archive with

It's hard to imagine what could happen instead. Even with a model with infinite context, where we imagine you could supply e.g. your entire email archive with each message in order to ask questions about one email, the inference time is still proportional to each input token. So you'd still want to use RAG as a performance optimization, even though today it's being used as more of a "there is no other way to supply enough of your own data to the LLM" must-have.