3 ms·

Rating differences at any given point in time are comparable to rating differences at another given point in time. That is, A being 200 points better than B in

by dfan 13y ago

Rating differences at any given point in time are comparable to rating differences at another given point in time. That is, A being 200 points better than B in 2013 means the same thing as C being 200 points better than D in 1963. This is a fundamental underpinning of Arpad Elo's model. It is absolutely true that the Elo model is a relative one and does not guarantee that a given rating number represents the same ability over time, but the belief that modern ratings are inflated is not unanimous. Ratings have certainly gone up over time, but computer analyses of games have indicated that intrinsic play quality has gone up over time as well. See the work of Ken Regan, for example (search for "ken regan ipr").

tokenadult 13y ago
Is there a mathematical proof that the Elo ratings have interval properties (so that "Rating differences at any given point in time are comparable to rating differences at another given point in time" is a surely true statement) and not just ordinal properties? I ask, because this is a hot issue in psychology, which I study, and it appears that almost all psychological "measurements" have, at best, merely ordinal properties ("A is smarter than B") and not interval properties ("A is as much smarter than B as B is smarter than C"). Constructing interval measurements properly is HARD, and yet assuming numbers have interval properties when they have only rank-order properties is easy, so I wonder if there is really a rigorous proof on this issue as to chess ratings.
- Ulkopuolinen 13y ago
  The Elo rating system is based on the model that a rating difference gives x an expected result as a function of x. Therefore, the Elo ratings "have interval properties". For example, the Elo rating model states that if A's rating is 200 points higher than B's, A will get in average 0.76 points against B. However, of course the Elo model might not be perfect, and thus a lot of other things contribute into the rating difference of players A and B, not only the expected result of the players playing only against each other.
- shalmanese 13y ago
  It's built into the definition of the rating. A 200 point difference in rating always means that there's a 75% expected chance of the higher ELO winning. In less fluid arenas, the actual ratings can diverge from ideal ratings but there's enough matchups in chess that this isn't a huge concern.
- dragontamer 13y ago
  It is mathemtatically proven that the Elo Rating system converges to the Bradley Terry interval rating system over time. The difference between Elo and Bradley-Terry is that Bradley-Terry assumes to have "all" the data. So Bradley Terry is used in Psychology studies for that reason. (Eventually, you stop taking data). Elo on the other hand, is mathematically equivalent to Bradley-Terry except with infinite data. Elo allows you to incrementally "add data" to an existing Bradley-Terry-esque model and make it more accurate. BTW: Elo has been superceeded by superior models today, but all (good) ranking models are "interval properties" that have been derived from the Bradley Terry model from Psychology research. See TrueSkill (which is based on Gaussian Curves, so they aren't technically Bradley-Terry but instead the Thurstone scale), Glicko, and Glicko2.
- jerf 13y ago
  In addition to the other comments, I'd add that in the context of Chess, we have a contest in which there is a winner or a loser or a draw, and we can relatively easily verify that the system is functioning as designed with simple statistics. We also have a system that broadly speaking produces a fairly uniform ordering. Yes, at the very top of chess there's some question of who might win or whether styles have an effect, but broadly speaking in the population at large, a 50 point difference will have a certain meaning; at the macro level, we're measuring something with one dimension. I would imagine many psychology studies are measuring something a great deal less well defined and more multidimensional in practice than "A will beat B at chess."

by dfan 13y ago

tokenadult 13y ago

Is there a mathematical proof that the Elo ratings have interval properties (so that "Rating differences at any given point in time are comparable to rating differences at another given point in time" is a surely true statement) and not just ordinal properties? I ask, because this is a hot issue in psychology, which I study, and it appears that almost all psychological "measurements" have, at best, merely ordinal properties ("A is smarter than B") and not interval properties ("A is as much smarter than B as B is smarter than C"). Constructing interval measurements properly is HARD, and yet assuming numbers have interval properties when they have only rank-order properties is easy, so I wonder if there is really a rigorous proof on this issue as to chess ratings.

Ulkopuolinen 13y ago
The Elo rating system is based on the model that a rating difference gives x an expected result as a function of x. Therefore, the Elo ratings "have interval properties". For example, the Elo rating model states that if A's rating is 200 points higher than B's, A will get in average 0.76 points against B. However, of course the Elo model might not be perfect, and thus a lot of other things contribute into the rating difference of players A and B, not only the expected result of the players playing only against each other.
shalmanese 13y ago
It's built into the definition of the rating. A 200 point difference in rating always means that there's a 75% expected chance of the higher ELO winning. In less fluid arenas, the actual ratings can diverge from ideal ratings but there's enough matchups in chess that this isn't a huge concern.
dragontamer 13y ago
It is mathemtatically proven that the Elo Rating system converges to the Bradley Terry interval rating system over time. The difference between Elo and Bradley-Terry is that Bradley-Terry assumes to have "all" the data. So Bradley Terry is used in Psychology studies for that reason. (Eventually, you stop taking data). Elo on the other hand, is mathematically equivalent to Bradley-Terry except with infinite data. Elo allows you to incrementally "add data" to an existing Bradley-Terry-esque model and make it more accurate. BTW: Elo has been superceeded by superior models today, but all (good) ranking models are "interval properties" that have been derived from the Bradley Terry model from Psychology research. See TrueSkill (which is based on Gaussian Curves, so they aren't technically Bradley-Terry but instead the Thurstone scale), Glicko, and Glicko2.
jerf 13y ago
In addition to the other comments, I'd add that in the context of Chess, we have a contest in which there is a winner or a loser or a draw, and we can relatively easily verify that the system is functioning as designed with simple statistics. We also have a system that broadly speaking produces a fairly uniform ordering. Yes, at the very top of chess there's some question of who might win or whether styles have an effect, but broadly speaking in the population at large, a 50 point difference will have a certain meaning; at the macro level, we're measuring something with one dimension. I would imagine many psychology studies are measuring something a great deal less well defined and more multidimensional in practice than "A will beat B at chess."