4 ms·

Someone asked me, "How does OpenAI know the MMR of its bots?" I don't know. I assume it's similar to how AlphaGo measures its ELO ranking. But the strange thin

by shawn 8y ago

Someone asked me, "How does OpenAI know the MMR of its bots?" I don't know. I assume it's similar to how AlphaGo measures its ELO ranking. But the strange thing is, this is hundreds of years of self-play, not a public pool of humans playing against each other. How does MMR in simulation translate to MMR in real matches? Before pointing out that it's possible for an ELO rating, consider that Dota MMR is a bit different – every game you win, you get +25. Every game you lose, you get -25. This changes at the very high / very low levels, or if the matchups are wildly imbalanced, but that's the general setup. Or it was, a few years ago. Does anyone have a guess?

gdb 8y ago
We estimate it based on when we start being evenly-matched against teams with a given average MMR. (As you might expect, our usual pattern is to lose to a test team consistently, then start being evenly matched, then consistently beat them.) The version of this diagram on our Five page also include bars indicating when we started and stopped testing against various teams: https://openai.com/five/ https://openai.com/five/. We don't claim this is perfect, which is why we label the chart "estimated MMR".
- shawn 8y ago
  Ah, that seems like an excellent system. It removes all of the guesswork of some formula-based solution. Empirical observations tend to be more accurate. It might be worth hiring one of the top dota pros to coach OpenAI's bots: they could point out each small mistake that the bots made which pros wouldn't make. That might make it more tractable to beat the top team next year. (The actual dota coaches might be even better for this purpose than the pros, too.) Looking forward to the outcome! The goal of "beat the top dota team" is one of those ambitious ones that few companies take on. It's really interesting to see the incremental progress.
  - Buge 8y ago
    How would the pro point out the mistakes to the bot? The bot cannot understand human language.
    - shawn 8y ago
      Not to the bot, but rather write down the mistakes and decide (as an AI programmer) how best to address them. The solutions would probably take the form of adjusting weights, adding additional dimensions, removing or refactoring existing signals, or deciding that extra training would naturally solve the problem.
- deleted 8y ago
  [deleted]
anewone 8y ago
The constant +-25 is just a consequence of the matchmaking enforcing balanced MMR matchups. It's not like chess where you're more likely to have some kind of imbalance. That's why it breaks at the extremes.
Kagerjay 8y ago
Last time (like 2 years ago, haven't played much since) I checked it wasn't exactly 25 MMR per match. It does depend which team has higher overall MMR to begin with, and values adjust from that 25 value. I had an MMR of around 4.4k solo/team, I think the average player had 2.25k MMR, the standard above average player had around 3.5k to 4k MMR. Anything past 4k MMR exponentially made matches way more intense. Its always been this way as far as I remember even back in DoTA 1 days 4k MMR in DoTA skillwise is about the equivalent of Platinum in rocket league, or Diamond in Overwatch

by shawn 8y ago

gdb 8y ago

We estimate it based on when we start being evenly-matched against teams with a given average MMR. (As you might expect, our usual pattern is to lose to a test team consistently, then start being evenly matched, then consistently beat them.) The version of this diagram on our Five page also include bars indicating when we started and stopped testing against various teams: https://openai.com/five/ https://openai.com/five/. We don't claim this is perfect, which is why we label the chart "estimated MMR".

shawn 8y ago
Ah, that seems like an excellent system. It removes all of the guesswork of some formula-based solution. Empirical observations tend to be more accurate. It might be worth hiring one of the top dota pros to coach OpenAI's bots: they could point out each small mistake that the bots made which pros wouldn't make. That might make it more tractable to beat the top team next year. (The actual dota coaches might be even better for this purpose than the pros, too.) Looking forward to the outcome! The goal of "beat the top dota team" is one of those ambitious ones that few companies take on. It's really interesting to see the incremental progress.
- Buge 8y ago
  How would the pro point out the mistakes to the bot? The bot cannot understand human language.
  - shawn 8y ago
    Not to the bot, but rather write down the mistakes and decide (as an AI programmer) how best to address them. The solutions would probably take the form of adjusting weights, adding additional dimensions, removing or refactoring existing signals, or deciding that extra training would naturally solve the problem.
deleted 8y ago
[deleted]

anewone 8y ago

The constant +-25 is just a consequence of the matchmaking enforcing balanced MMR matchups. It's not like chess where you're more likely to have some kind of imbalance. That's why it breaks at the extremes.

Kagerjay 8y ago

Last time (like 2 years ago, haven't played much since) I checked it wasn't exactly 25 MMR per match. It does depend which team has higher overall MMR to begin with, and values adjust from that 25 value. I had an MMR of around 4.4k solo/team, I think the average player had 2.25k MMR, the standard above average player had around 3.5k to 4k MMR. Anything past 4k MMR exponentially made matches way more intense. Its always been this way as far as I remember even back in DoTA 1 days 4k MMR in DoTA skillwise is about the equivalent of Platinum in rocket league, or Diamond in Overwatch