I see elements of the bitter lesson in arguments about context window size and RAG. The argument is about retrieval being the equivalent of compute/search. Just

I see elements of the bitter lesson in arguments about context window size and RAG. The argument is about retrieval being the equivalent of compute/search. Just improve them, to hell with all else. However, retrieval is not just google search. Primary key lookups in my db are also retrieval. As are vector index queries or BM25 free text search queries. It's not a general purpose area like compute/search. In summary, i don't think that RAG is dead. Context engineering is just like feature engineering - transform the swamp of data into a structured signal that is easy for in-context learning to learn. The corollory of all this is it's not just about scaling up agents - giving them more LLMs and more data via MCP. The bitter lesson doesn't apply to agents yet.