4 ms·

The sentiment you're expressing is the idea that one 2k player is so much better than two 1k rated players that it doesn't make sense to simply sum up each team

The sentiment you're expressing is the idea that one 2k player is so much better than two 1k rated players that it doesn't make sense to simply sum up each team's rating and keep it even, which is the false assumption that the AoE 2 devs made. Elo is not linear. However I don't think they'd have fixed it just by averaging the Elo ratings, they must have implemented something more complex, like a proximity based algorithm, otherwise if the teams were the same size you'd hit a similar problem with an average as with a sum. A 2k player + a 0 rated player would sum to 2k and average to 1k. You and your friend would also sum to 2k and average to 1k, again assuming even team sizes. That said, still very funny! Also, fun fact: Elo isn't an acronym/initialism, it's someone's name, Arpad Elo (the inventor). Also also, I don't think Elo is really a good idea for multiplayer (more than 1v1 that is) games generally. Don't know enough to say that authoritatively, but using it for something like AoE 2 is likely prone to all kinds of strange unexpected outcomes.

lordnacho 6y ago
Seems to me that you need to use the max of the team rather than the average. The better player can tell the less good player to do, up to a point.
- maybeOneDay 6y ago
  This is what rocket league uses. While it can be unpleasant as a high ranked player to jump in a lobby with your much less skilled friend, overall the matchmaking system and equality of matches is in my opinion utterly fantastic
- jerf 6y ago
  I'd expect "geometric average" to be one of the closest simple statistics for most games, assuming that you have some ranking that can be treated linearly. It would be generally unclear that rankings can be treated linearly. A subtle aspect of Elo is that as the disparity between two players increases, the Elo metric essentially becomes less and less certain that there is any relationship between the skill of the players at all, which is probably a bit deeper way of understanding "why" . Eventually you'll reach a point where if you round to integers, the winner winning will result in an increase of 0 points. (In fact one thing to watch out for is how you round; it's easy to end up with a wandering Elo basis if you truncate both sides rather than rounding properly, for instance.) It's fairly trivial to show that if you take a pool of the worst players and the best players, Elo will eventually converge to those two pools having about that amount of distance between them, but by inserting a fresh pool of middling players into the pool, the distance between the best & the worst will increase despite their skill not changing. If skill is linearly distributed... assuming you can even define that... Elo ratings might be somewhat reasonably treated linearly, but skill is very unlikely to be linearly distributed. Basically, I like Elo when used properly and I have used it to good effect a few times myself, but it is a bit dangerous in that it offers a number... but that number lacks many properties we associate with integers. (For instance, in another posting I point out you can shift the entire Elo pool by any constant you like without changing the system. Integers do not have this property.) They have a definite meaning, but it isn't what your intuition might suggest. You really shouldn't do any arithmetic on them. They only make sense in the context of the Elo computation itself.
  - noxvilleza 6y ago
    Although highly dependent on the game itself, I did some investigation back in 2008 into testing various Elo modifications on a pretty large dataset of individual players when playing in a 5v5 team game, and geometric mean of individual ratings was the best simple predictor for which team would win (the update method was essentially 10 individual matchups of each player vs the aggregate rating of the enemy team).
wongarsu 6y ago
Overwatch implements an Elo system for 6v6 competitive multiplayer. Over the years they've had to make a lot of tweaks to make it kind of work. For example groups are rated by average Elo, but the system doesn't allow you to cause large Elo differences within a team. Also it differentiates between "premade" groups and groups assembled by matchmaking, since the former are assumed to be more coordinated. It also tries to be really smart about how Elo points get distributed within the team on win/loss (which causes a heap of other problems). Overall you can make it somewhat work, but Elo really isn't well suited for anything larger than 1v1
- xxpor 6y ago
  I'm not really sure anyone's come up with a better system though for a game like OW. At least in comp I don't get rolled too often, so it sort of works.
- closed 6y ago
  It's worth noting that the general models ELO approximates, called item response theory (IRT) models, could handle larger than 1v1. Its similar to how you model situations where multiple latent skills contribute to performance (multidimensional IRT). In practice, it's a way easier though to use simple stuff like averaging / constraining groups so everyone has roughly the same skill level.
jerf 6y ago
"The sentiment you're expressing is the idea that one 2k player is so much better than two 1k rated players that it doesn't make sense to simply sum up each team's rating and keep it even, which is the false assumption that the AoE 2 devs made. Elo is not linear." It's even better than that. Elo is purely relative. The difference in score between two opponents is what is supposed to predict the likelihood of victory for the two. This means that you can add any constant you like to the entire Elo pool without affecting the Elo rankings at all. As a matter of aesthetics, when initializing an Elo pool it is nice to try to start with values that will result in nobody having a negative value, because that gives humans the feelbadz, but that's not mathematically necessary; you can run the whole thing deep in the negative billions if that floats your boat. The Elo math is very much based on two players playing head-to-head. It would be completely broken for almost any other scenario, as multiplayer introduces new degrees of freedom that Elo has nowhere to put. By that I mean that there may be one game for which the team's skill is essentially the max of the members, another for which is it essentially the minimum, and a wide variety of things in between. Elo has nowhere to put that "wide variety", and it was never designed or intended to do so, so that's not a criticism of it. It just isn't what it was for.

lordnacho 6y ago

Seems to me that you need to use the max of the team rather than the average. The better player can tell the less good player to do, up to a point.

maybeOneDay 6y ago
This is what rocket league uses. While it can be unpleasant as a high ranked player to jump in a lobby with your much less skilled friend, overall the matchmaking system and equality of matches is in my opinion utterly fantastic
jerf 6y ago
I'd expect "geometric average" to be one of the closest simple statistics for most games, assuming that you have some ranking that can be treated linearly. It would be generally unclear that rankings can be treated linearly. A subtle aspect of Elo is that as the disparity between two players increases, the Elo metric essentially becomes less and less certain that there is any relationship between the skill of the players at all, which is probably a bit deeper way of understanding "why" . Eventually you'll reach a point where if you round to integers, the winner winning will result in an increase of 0 points. (In fact one thing to watch out for is how you round; it's easy to end up with a wandering Elo basis if you truncate both sides rather than rounding properly, for instance.) It's fairly trivial to show that if you take a pool of the worst players and the best players, Elo will eventually converge to those two pools having about that amount of distance between them, but by inserting a fresh pool of middling players into the pool, the distance between the best & the worst will increase despite their skill not changing. If skill is linearly distributed... assuming you can even define that... Elo ratings might be somewhat reasonably treated linearly, but skill is very unlikely to be linearly distributed. Basically, I like Elo when used properly and I have used it to good effect a few times myself, but it is a bit dangerous in that it offers a number... but that number lacks many properties we associate with integers. (For instance, in another posting I point out you can shift the entire Elo pool by any constant you like without changing the system. Integers do not have this property.) They have a definite meaning, but it isn't what your intuition might suggest. You really shouldn't do any arithmetic on them. They only make sense in the context of the Elo computation itself.
- noxvilleza 6y ago
  Although highly dependent on the game itself, I did some investigation back in 2008 into testing various Elo modifications on a pretty large dataset of individual players when playing in a 5v5 team game, and geometric mean of individual ratings was the best simple predictor for which team would win (the update method was essentially 10 individual matchups of each player vs the aggregate rating of the enemy team).

wongarsu 6y ago

Overwatch implements an Elo system for 6v6 competitive multiplayer. Over the years they've had to make a lot of tweaks to make it kind of work. For example groups are rated by average Elo, but the system doesn't allow you to cause large Elo differences within a team. Also it differentiates between "premade" groups and groups assembled by matchmaking, since the former are assumed to be more coordinated. It also tries to be really smart about how Elo points get distributed within the team on win/loss (which causes a heap of other problems). Overall you can make it somewhat work, but Elo really isn't well suited for anything larger than 1v1

xxpor 6y ago
I'm not really sure anyone's come up with a better system though for a game like OW. At least in comp I don't get rolled too often, so it sort of works.
closed 6y ago
It's worth noting that the general models ELO approximates, called item response theory (IRT) models, could handle larger than 1v1. Its similar to how you model situations where multiple latent skills contribute to performance (multidimensional IRT). In practice, it's a way easier though to use simple stuff like averaging / constraining groups so everyone has roughly the same skill level.

jerf 6y ago

"The sentiment you're expressing is the idea that one 2k player is so much better than two 1k rated players that it doesn't make sense to simply sum up each team's rating and keep it even, which is the false assumption that the AoE 2 devs made. Elo is not linear." It's even better than that. Elo is purely relative. The difference in score between two opponents is what is supposed to predict the likelihood of victory for the two. This means that you can add any constant you like to the entire Elo pool without affecting the Elo rankings at all. As a matter of aesthetics, when initializing an Elo pool it is nice to try to start with values that will result in nobody having a negative value, because that gives humans the feelbadz, but that's not mathematically necessary; you can run the whole thing deep in the negative billions if that floats your boat. The Elo math is very much based on two players playing head-to-head. It would be completely broken for almost any other scenario, as multiplayer introduces new degrees of freedom that Elo has nowhere to put. By that I mean that there may be one game for which the team's skill is essentially the max of the members, another for which is it essentially the minimum, and a wide variety of things in between. Elo has nowhere to put that "wide variety", and it was never designed or intended to do so, so that's not a criticism of it. It just isn't what it was for.