3 ms·
>> hard to deny that their LLM models aren't really, really good though. The context window of Gemini 1.5 pro is incredibly large and it retains the memory of
by manishsharan 2y ago
>> hard to deny that their LLM models aren't really, really good though.
The context window of Gemini 1.5 pro is incredibly large and it retains the memory of things in the middle of the window well. It is quite a game changer for RAG applications.
- KaoruAoiShiho 2y agoIt looks like long context degraded from 1.5 to 2.0 according to the 2.0 launch benchmarks.
- verdverm 2y agoThose benchmarks are for the Flash 2.0 model, showing it improved over the 1.5 Pro model minus for that one benchmark
- caeril 2y agoBear in mind that a "1 million token" context window isn't actually that. You're being sold a sparse attention model, which is guaranteed to drop critical context. Google TPUs aren't running inference on a TERABYTE of fp8 query-key inputs, let alone TWO of fp16. Google's marketing wins again, I guess.