3 ms·
> Most games aren't chess -- where the only variance is picking who's black and who's white -- in fact, they might include dozens of RNG mechanics (from critica
by lmm 6y ago
> Most games aren't chess -- where the only variance is picking who's black and who's white -- in fact, they might include dozens of RNG mechanics (from critical strikes to ability rolls, to spawn points). These mechanics (while fun and well-designed) might pollute your "idealized" model. There's also the problem of RPS (rock-paper-scissors) mechanics or pick-counter-pick mechanics which will also heavily skew win rates. For instance, given a slow combo Magic deck, you will most likely auto-concede to mono red aggro (regardless of skill level). If you're using Elo, this will pollute your model. (Hint: you shouldn't be using Elo.)
None of which matters? All that means is that the results of individual games are a bit higher variance. Elo handles that by design. If you lose a certain proportion of Magic games to less-skilled players then this should be considered a reflection of your skill, because the only reasonable definition of skill at the came is the rate at which you actually win it; anything else can be gamed and so should be ignored.
> Most games also don't have chess' high skill ceiling. Chess has such a high skill ceiling for a number of reasons -- it's one of the oldest games still being actively played, for one. Suppose your "game" is simply the flip of a coin (everyone wins 50% of the time). Zero skill involved. Trying to model win-loss-ratios using a sigmoid curve is silly. Obviously, no game is going to be a coin flip, but there's a world of difference between chess and DOTA.
That's also something that Elo handles just fine? If every game is a coin flip then everyone will end up with the same Elo. If player A has x more Elo points than player B, then they win y% of their games. If your game has a skill ceiling where even a complete beginner always wins, say, 20% of their games, then that just means no-one will ever be able to rise above a corresponding Elo rating.
- dvt 6y ago> That's also something that Elo handles just fine? If every game is a coin flip then everyone will end up with the same Elo. If player A has x more Elo points than player B, then they win y% of their games. If your game has a skill ceiling where even a complete beginner always wins, say, 20% of their games, then that just means no-one will ever be able to rise above a corresponding Elo rating. That's not how it works. The distribution you end up with will not be uniform, it will look like this (just ran Elo with a coinflip; 11 players, 1000 matches): https://imgur.com/9O82pRj https://imgur.com/9O82pRj On the long term, I think this will tend to a geometric distribution with a low p value.
- lmm 6y agoShow your working? If you're matchmaking players against equal-ranked players, then each match is just +/- 50 points, you'll get a binomial distribution which tends to normal as n gets large (assuming a large player pool so each player's results are independent). If players play players with different ratings then that will tend to push their rating back towards neutral. You certainly don't get a geometric distribution because the rating algorithm is completely symmetric.
- dvt 6y ago> each match is just +/- 50 points This only happens in the rare cases where you're matching players against (exactly) equally-ranked players. You can mitigate this by always trying to match as "close as possible," but it's only a mitigation. Try simulating random matchmaking with Elo, and you'll get something like this: https://i.imgur.com/1Y08jUB.png https://i.imgur.com/1Y08jUB.png (1000 players, 100,000 games). In my simulation, I set k (the Elo constant) = 50. I think it's going to tend to a geometric distribution for reasons discussed here (which is another interesting and non-intuitive result): http://www.decisionsciencenews.com/2017/06/19/counterintuitive-problem-everyone-room-keeps-giving-dollars-random-others-youll-never-guess-happens-next/ http://www.decisionsciencenews.com/2017/06/19/counterintuiti...
- LudwigNagasena 6y agoElo can be negative, so this doesn’t apply.
- lmm 6y ago> Try simulating random matchmaking with Elo I will. I was hoping you'd post the actual simulation details rather than more unlabelled graphs.
- dvt 6y agohttps://gist.github.com/dvx/e3311a984e14dca7c5eeb9214e0be3d5 https://gist.github.com/dvx/e3311a984e14dca7c5eeb9214e0be3d5 Using: https://github.com/HankSheehan/EloPy https://github.com/HankSheehan/EloPy (with a custom k-value of 50).