3 ms·

I've seen these benchmarks over and over and I wonder how they work. Seems to me you would need something close to a test or exam like in schools to properly ra

• minimaxir 3y ago
  The Open LLM Leaderboard (https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...) is a good indicator of progress in the space that hedges on a variety of potential use cases, but for real world use it cannot replace manually experimenting with models to ensure output is what you expect for your use case, especially creative use cases which these benchmarks do not capture.