3 ms·

This isn't exactly what you're looking for, but I've found this ranked list of llms useful -- https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard http

• stavros 3y ago
  Keep in mind that the rankings are just that: an order of "best" to "worst". There's nothing on there that's an indication of how much better one model is from another.
  • WhitneyLand 3y ago
    That’s exactly what it does. It’s an Elo scoring system, like chess. By definition the difference between two entries represents how often it beats the other entry. https://en.wikipedia.org/wiki/Elo_rating_system https://en.wikipedia.org/wiki/Elo_rating_system
    • stavros 3y ago
      So, GPT-4 beats the next best LLM. Is GPT-4 twice as good, or a hundred times as good as the next one?
      • furyofantares 3y ago
        Looks like 62 points better, which under the Elo model implies about 59/41 matchup (aka GPT-4-Turbo is about 59% chance to beat Claude on a random prompt). You can look up an Elo probability calculator to plug in an Elo difference and see this number. For example plug in two Elo ratings here: https://sandhoefner.github.io/chess.html https://sandhoefner.github.io/chess.html - really only the difference between them matters. Btw the similarity of the numbers 59 and 62 here is coincidence, other differences won't be nearly the same as the probability.