4 ms·
Context window + prompt caching if you can't use RAG. Can add a lot to long context models, and their needle in haystack metrics keep getting better. Why can't
by scosman 1y ago
Context window + prompt caching if you can't use RAG. Can add a lot to long context models, and their needle in haystack metrics keep getting better.
Why can't you use RAG?
- simianwords 1y agoyou lose coherence across chunks of context size. i wish i could spend compute to pre-train on some knowledge.