5 ms·
Retrieval Augmented Generation aka RAG with a relevant context. Long contexts continue to have the lost in the middle syndrome.
by dmezzetti 3y ago
Retrieval Augmented Generation aka RAG with a relevant context. Long contexts continue to have the lost in the middle syndrome.
- nl 3y agoGemini doesn't have the "lost in the middle syndrome"
- dmezzetti 3y agoWe don't have enough data on that yet to know for sure. It was just released this week. But nonetheless, sending 2M tokens to a LLM isn't an efficient way to solve most problems.
- int_19h 3y agoLLMs in general are likely to be a very inefficient way to solve many problems they're currently being used for. But so long as we don't have something better, brute force remains a valid approach.
- nl 3y ago> We don't have enough data on that yet to know for sure. It was just released this week. I think there is sufficient evidence to think it works. A bunch of people who do have access have put it through a bunch of real-world exercises to test this, and while GPT4 128K and Claude 200K both fail Gemini is passing. I agree sending 2M tokens to a LLM is costly at the moment and that RAG has a bunch of advantages in some circumstances.