2 ms·
> leaderboards penalize it > socio-technical mitigation: modifying the scoring of existing benchmarks that are misaligned but dominate leaderboards Sounds mor
by baq 1y ago
> leaderboards penalize it
> socio-technical mitigation: modifying the scoring of existing benchmarks that are misaligned but dominate leaderboards
Sounds more like we need new leaderboards and old ones should be deprecated
- smallmancontrov 1y agoYeah, it's a big enough lift that I think it's fair to allow the leaderboard teams new announcements and buzzwords in exchange for doing the work :-)