4 ms·
Nice. This ranking basically matches other benchmarks, from what I can tell. Which implies this would probably also hold for the larger models, which are sadly
by andai 3mo ago
Nice. This ranking basically matches other benchmarks, from what I can tell.
Which implies this would probably also hold for the larger models, which are sadly not included in the leaderboard.
- krackers 3mo agoI think it's limited to using base models (non post-trained), because the post-training would skew the logit distribution. There are ways to "coax" post-trained models back into behaving "like" a base model, I wonder if the benchmark could be unofficially updated with those somehow.