3 ms·
RAG is mostly a hack to address limited context windows, or limited use of wide context windows – some models have large windows but don’t use content that isn’
by dragonwriter 3y ago
RAG is mostly a hack to address limited context windows, or limited use of wide context windows – some models have large windows but don’t use content that isn’t near the beginning or end well, or expensive content windows (LLM-as-service typically charges by the token, so RAG can reduce cost).
“Stuffing relevant data into the context window rather than relying purely on training” is a solution to confabulation, though, just like providing relevant reference information to a person who is being pressured to answer a question is.