3 ms·
Your typical LLM benchmarks simply do not test or use large context sizes. We need benchmarks for tasks that requiere large context sizes (like recalling facts
by cypress66 3y ago
Your typical LLM benchmarks simply do not test or use large context sizes.
We need benchmarks for tasks that requiere large context sizes (like recalling facts and understanding them in long stories). I'm sure OpenAI have internal benchmarks for these tasks.
- jumpCastle 3y agohttps://github.com/google-research/long-range-arena https://github.com/google-research/long-range-arena