Also from the article: "A key takeaway lesson from our journey is that evaluations are strongly tied to their implementations–down to minute details such as pr

Also from the article: "A key takeaway lesson from our journey is that evaluations are strongly tied to their implementations–down to minute details such as prompts and tokenization." In line with what you were saying about the HF LLM Leaderboard only be a starting point I wish it had more (selectable) columns with significant hyperparameters like the tokenization method or the context window size. A way to group related models would also be nice. I think that would make the Leaderboard even more useful than it is already.