2 ms·
From what I got from this, they have picked a bunch of benchmarks where many LLMs can easily get to about 90%. It does not seem to be possible for an LLM to exc
by pclmulqdq 2y ago
From what I got from this, they have picked a bunch of benchmarks where many LLMs can easily get to about 90%. It does not seem to be possible for an LLM to exceed about 95% on any sort of benchmark given the randomness in the process. The lack of progress on these easy problems means that HuggingFace's leaderboard is not as meaningful.
They are sort of saying both things: LLMs are plateauing, but the benchmarks are also too easy.