4 ms·
Am I misunderstanding something or is there a glaring statistical error here? I think this analysis works if the matches are all between two players of truly id
by ladberg 3y ago
Am I misunderstanding something or is there a glaring statistical error here? I think this analysis works if the matches are all between two players of truly identical skill that will roughly win 50% of the time against the other, but that's not really the case here. They attempt to do that by controlling for Elo but I'm decently sure that you could pick any two players of the same Elo on lichess and one will win the majority of games.
Now if we assume there is often a slightly better player in these games, the better player will more likely get into an advantageous position early in the game and win more often, but not entirely because they got into the more advantageous position. What I'm trying to say is that someone who is up a rook will win partially because they're up a rook and partially because they are likely a better player in the first place and will continue to play better than their opponent.
I think to do this study correctly you'd need to place new players in a random position drawn from the dataset and actually evaluate the win rate without the confounding factor of having gotten into that position in the first place.
- spoonjim 3y ago> I'm decently sure that you could pick any two players of the same Elo on lichess and one will win the majority of games. I think that's true at 2000+ but at my rating (500) it's just a comedy of blunders
- fairity 3y ago> I'm decently sure that you could pick any two players of the same Elo on lichess and one will win the majority of games. Interesting hypothesis, but my intuition is that this effect is small. It should be easy to measure. Just take random matches, and study winrate when a player has beaten their opponent once before.
- yamazakiwi 3y agoPlayers can also swap back and forth between winning a majority of games, so it depends on the amount of games we're talking about.
- queuebert 3y agoThat's exactly what Elo controls for: all things, being equal, what is the skill component. Now there could be a second-order effect in which a higher Elo player is even better at winning from an advantageous position, but given large enough samples this should be in the mix.
- ladberg 3y agoYeah I guess my claim is that two people of the same Elo on lichess are likely to not be perfectly equal. If the average winrate of any random two players with the same Elo is something like 60/40 (random speculation) then you'd bias the results in the article enough to create the kind of graphs shown. The problem is that this doesn't really average out with more samples.
- BurningFrog 3y agoI accept that players with the same ELO can have unequal strength. But I don't see how that wouldn't average out over lots of games?
- ladberg 3y agoThe probability of player A being up a rook over player B is higher when player A is better than player B, even if their Elo is the same. So the author has picked scenarios where player A is already more likely to be better than player B but assumed they were of equal skill, which will cause the results to be biased. If I have time later I'll try to whip up a small coin-tossing demo in Python that demonstrates what I mean, I'm definitely not the best explainer!
- BurningFrog 3y agoOK, that's a reasonable argument at least. My gut feel is that the effect is minimal at best, but I don't have an argument for that.
- queuebert 3y ago
- inimino 3y agoUsually a chess game is started and finished by the same player, so why does this matter? The question you want to answer is "I'm up a rook, how likely am I to win?", which this analysis answers.