3 ms·
I feel like the post doesn't relate to the title? The post seems to be about changing the Leaderboard and doesn't comment too much about whether the actual rea
by azeemba 2y ago
I feel like the post doesn't relate to the title?
The post seems to be about changing the Leaderboard and doesn't comment too much about whether the actual real-life performance of LLMs is plateauing and what can be done about it.
- pclmulqdq 2y agoFrom what I got from this, they have picked a bunch of benchmarks where many LLMs can easily get to about 90%. It does not seem to be possible for an LLM to exceed about 95% on any sort of benchmark given the randomness in the process. The lack of progress on these easy problems means that HuggingFace's leaderboard is not as meaningful. They are sort of saying both things: LLMs are plateauing, but the benchmarks are also too easy.