4 ms·

"Leaderboards", meh. This tweet is still very true: https://twitter.com/karpathy/status/1737544497016578453 https://twitter.com/karpathy/status/173754449701657

• not2b 3y ago
  Goodhart's law would seem to apply: https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law Nevertheless, scoring so well on this benchmark is an accomplishment, though I'm not in a position to evaluate how significant it is.
• make3 3y ago
  that's why the huggingface llm arena exists
• int_19h 3y ago
  Nothing beats an actual human spending a couple hours with the model when it comes to meaningful evaluation.