4 ms·

I honestly wish that people wouldn't put out rubbish analysis like this. >his conclusion is there is no reason whatsoever to suspect him of cheating. No. His

I honestly wish that people wouldn't put out rubbish analysis like this. >his conclusion is there is no reason whatsoever to suspect him of cheating. No. His conclusion is that based on this really weird brute force statistical analysis there is no statistical indicator that he cheated. Firstly, how you can state there's no reason whatsoever to suspect him of cheating is absurd. You know he has a previous record of cheating so as a starting point that statement is absurd. But secondly and more importantly, no one is suggesting that Neimann is a moron. We're not saying he's a 1200 ELO lucking out against a grand master. Any account of Neimann cheating is going to include the fact that he's already a very good chess player - he's not going to be cheating through his matches with any average player. And as other GMs have pointed out, cheating could literally involve 1 advantageous decision at a key point in a match. What we have here is very simple. Neimann was quasi-accused of cheating in 1 match, and this genius has gone off and analysed thousands of other matches and come to the conclusion that there's no statistical evidence of cheating. A methodology that is basically designed to ensure almost no one could ever be proven to be cheating using this method.

kumarvvr 4y ago
Awesome take. Reminds me of a book that teaches us how one can lie with statistics. While reading the whole article, I was convinced over the "data centric analysis". Afterall, how can data lie? But, its revealing that cheating in a chess game with a GM is literally making the right guided move at the right time.
Aromasin 4y ago
What's ever worse is that FIDE feel justified exonerating people based on this statistical analysis alone, despite the fact there seems to be a general sense within the GM community that something is wrong and has been for a while.
Ensorceled 4y ago
Yeah, the "1 in a million" hand-waving is weird. If I had a mechanism that allowed me to cheat at poker, I would only use it on a push. I certainly wouldn't be using every hand or even at all in penny ante games.
agarden 4y ago
Regan's analysis is a good deal more detailed than that. He claims that if you check the computer just three times a game, he is going to catch you within three games. Now, if you check just once, it becomes a lot harder. For the game against Carlsen, he looked at the key moves in that game. The first 20 or so moves were theory. You can't ever prove cheating in theory. After they diverged from theory, Regan said there were only two really key moves. For both of those, Niemann did not use the computers top choice. One of them he played a downright inaccurate move that could have cost him a tempo. I don't know if Niemann cheated or did not, or even what exactly Carlsen suspects, but Regan's analysis seems to me to be strong enough that it counterbalances the known character deficiency of Niemann.
- blub 4y ago
  If there are several good moves, it seems logical to cheat by not picking the computer’s top choice.
  - hrunt 4y ago
    Is it cheating if you make decisions to purposefully not gain an advantage (ignoring gambling-induced outcomes, like throwing a match)? "He was cheating! An outside influence was telling him which move to make and he purposefully didn't do it!"
    - blub 4y ago
      Like I said, it depends on the available set of moves. If #1 is +1 and #2 is +0.7, but the cheater wanted to play #5 at -0.5 then that’s an outcome-changing difference. Chess engines like Hiarcs can show a series of good, ok, questionable and bad moves for each touched piece color-coded on the board for example.
  - agarden 4y ago
    If there are three or four good moves of approximately equal strength in a position, then it just isn't a position where someone at grandmaster level needs to cheat or benefits from doing so. They were going to find one of those moves anyway.
jerf 4y ago
I agree. The number one error made by people with statistical knowledge is to assume independence of events. Even very smart and very educated people tend to make this mistake rampantly. I hypothesize it is due to the education curriculum being very slanted in the direction of assuming independence everywhere because it makes the math easier, but I can't prove this. In the real world, true and full independence should be considered an exception rather than the norm. Saying that Neimann is a very good chess player is basically a restatement of the fact that his previous records are likely clean. Nobody was accusing him of constant cheating before. (Cheating once is relatively easy. Cheating continuously, in multiple different environments, and leaving no trace, is hard.) The entire question at hand is essentially "Is Neimann's probability of cheating correlated with playing Magnus Carlson?" (and a family of similar questions can be asked) and no amount of analysis of his games not played against Magnus Carlson will answer that. As for why the probability of cheating would be correlated with playing Magnus Carlson, again, no amount of consulting the statistics will be able to answer that question, but for anyone who is a human being with human foibles the answer is fairly obvious. Someone used to double checking premises will observe that there's not much in this question statement specific to Neimann. The fact that he's cheated before does raise his particular probability, but you could ask this about anyone. And my reply to that would be, yes, that is indeed how things work, and is exactly why these high stakes games are played with more formality and more security than a pick-up game in a city park. Neimann's prior cheating does raise the question with a bit more urgency, but we generally assume a priori that anyone might be interested in cheating in a game with Magnus Carlson, and high level play in general, and act accordingly. agarden in a sibling post suggests that the analysis itself goes deeper, which sounds more interesting. I am speaking only to what is in the linked article. It would be far from the first time a reported simplified a complex and nuanced argument into something useless, and I thought the point about independence is still worth posting.
ChrisLomont 4y ago
>this really weird brute force statistical analysis Care to explain why you call the method "really weird"? Is there a flaw in the statistics that you can correct? Here's [1] his publications - all well cited, his h-index is good, and I find no refutation of complaints in the literature about his methods. He seems quite competent in this area. >how you can state there's no reason whatsoever to suspect him of cheating is absurd If cheating does not show up in performance then is it cheating? If there is any performance gain then it should show up at some statistical level. [1] https://scholar.google.com/citations?user=8nk9k5oAAAAJ&hl=en&oi=sra https://scholar.google.com/citations?user=8nk9k5oAAAAJ&hl=en...
- SilverBirch 4y ago
  It's not that the statistics are wrong, it's that you can't apply statistics to prove this at all. Specifically because the person you're analyzing knows all about chess. They know that a super strong weird move will just expose them, so instead they're going to pick lines that just slightly increase their strength. This is like the statistical analyses that show election rigging by highlighting a statistically improbable distribution of results - that analysis works if the person rigging the election doesn't consider the statistical analysis when they're doing their cheating, it is completely avoidable if you cheat competently. Which brings us back to a sort of basic question - if this guy is cheating, do we think he's doing it fairly competently or not? If he's not cheating, the statistics will show normal play, if he is cheating and he's fairly competent the statistics will show normal play. So what has this analysis done? It's proved that he's not totally incompetent, which we already knew because he's pretty well established as a good chess player even he isn't truly a 2700ELO.
  - ChrisLomont 4y ago
    >it's that you can't apply statistics to prove this at all If there are not statistical differences, then there are no performance differences. Cheating by definition should imply performance differences. If he is cheating, then at some point in the future, if that method becomes detectable and he has to stop, then his play will suddenly suffer, which would be more evidence. Claiming that statistics cannot answer this question with statistics is not true. It may be hard, or the current sample too small, but claiming stats is not usable is a misunderstanding of statistics. >This is like the statistical analyses that show election rigging by highlighting a statistically improbable distribution of results This only works on the public, and is not what professional statisticians that analyze elections do. And even here, if the event is rare enough, say 1 part in quadrillions, and the analysis is correct, then yes, we would certainly conclude there was rigging. All human knowledge is statistical. Things we claim to be true are only statistically true to large odds, so even for election rigging, if the stats reach some level of certainty, then it is completely valid proof that would hold in court. The pop idiocy of election rigging claims has never risen to that level. >it is completely avoidable if you cheat competently No, it is not. It may only lower the signal to noise ratio, but there is still detectable differences. If you continually improve the statistics and are forced to lower the signal, eventually the signal would be so low as to not affect the system, which in this case is chess games. Physics, for example, can tease events out of on the order of 1 part in trillions and demonstrate signal. Plenty of other things do the same.

kumarvvr 4y ago

Awesome take. Reminds me of a book that teaches us how one can lie with statistics. While reading the whole article, I was convinced over the "data centric analysis". Afterall, how can data lie? But, its revealing that cheating in a chess game with a GM is literally making the right guided move at the right time.

Aromasin 4y ago

What's ever worse is that FIDE feel justified exonerating people based on this statistical analysis alone, despite the fact there seems to be a general sense within the GM community that something is wrong and has been for a while.

Ensorceled 4y ago

Yeah, the "1 in a million" hand-waving is weird. If I had a mechanism that allowed me to cheat at poker, I would only use it on a push. I certainly wouldn't be using every hand or even at all in penny ante games.

agarden 4y ago

Regan's analysis is a good deal more detailed than that. He claims that if you check the computer just three times a game, he is going to catch you within three games. Now, if you check just once, it becomes a lot harder. For the game against Carlsen, he looked at the key moves in that game. The first 20 or so moves were theory. You can't ever prove cheating in theory. After they diverged from theory, Regan said there were only two really key moves. For both of those, Niemann did not use the computers top choice. One of them he played a downright inaccurate move that could have cost him a tempo. I don't know if Niemann cheated or did not, or even what exactly Carlsen suspects, but Regan's analysis seems to me to be strong enough that it counterbalances the known character deficiency of Niemann.

blub 4y ago
If there are several good moves, it seems logical to cheat by not picking the computer’s top choice.
- hrunt 4y ago
  Is it cheating if you make decisions to purposefully not gain an advantage (ignoring gambling-induced outcomes, like throwing a match)? "He was cheating! An outside influence was telling him which move to make and he purposefully didn't do it!"
  - blub 4y ago
    Like I said, it depends on the available set of moves. If #1 is +1 and #2 is +0.7, but the cheater wanted to play #5 at -0.5 then that’s an outcome-changing difference. Chess engines like Hiarcs can show a series of good, ok, questionable and bad moves for each touched piece color-coded on the board for example.
- agarden 4y ago
  If there are three or four good moves of approximately equal strength in a position, then it just isn't a position where someone at grandmaster level needs to cheat or benefits from doing so. They were going to find one of those moves anyway.

jerf 4y ago

I agree. The number one error made by people with statistical knowledge is to assume independence of events. Even very smart and very educated people tend to make this mistake rampantly. I hypothesize it is due to the education curriculum being very slanted in the direction of assuming independence everywhere because it makes the math easier, but I can't prove this. In the real world, true and full independence should be considered an exception rather than the norm. Saying that Neimann is a very good chess player is basically a restatement of the fact that his previous records are likely clean. Nobody was accusing him of constant cheating before. (Cheating once is relatively easy. Cheating continuously, in multiple different environments, and leaving no trace, is hard.) The entire question at hand is essentially "Is Neimann's probability of cheating correlated with playing Magnus Carlson?" (and a family of similar questions can be asked) and no amount of analysis of his games not played against Magnus Carlson will answer that. As for why the probability of cheating would be correlated with playing Magnus Carlson, again, no amount of consulting the statistics will be able to answer that question, but for anyone who is a human being with human foibles the answer is fairly obvious. Someone used to double checking premises will observe that there's not much in this question statement specific to Neimann. The fact that he's cheated before does raise his particular probability, but you could ask this about anyone. And my reply to that would be, yes, that is indeed how things work, and is exactly why these high stakes games are played with more formality and more security than a pick-up game in a city park. Neimann's prior cheating does raise the question with a bit more urgency, but we generally assume a priori that anyone might be interested in cheating in a game with Magnus Carlson, and high level play in general, and act accordingly. agarden in a sibling post suggests that the analysis itself goes deeper, which sounds more interesting. I am speaking only to what is in the linked article. It would be far from the first time a reported simplified a complex and nuanced argument into something useless, and I thought the point about independence is still worth posting.

ChrisLomont 4y ago

>this really weird brute force statistical analysis Care to explain why you call the method "really weird"? Is there a flaw in the statistics that you can correct? Here's [1] his publications - all well cited, his h-index is good, and I find no refutation of complaints in the literature about his methods. He seems quite competent in this area. >how you can state there's no reason whatsoever to suspect him of cheating is absurd If cheating does not show up in performance then is it cheating? If there is any performance gain then it should show up at some statistical level. [1] https://scholar.google.com/citations?user=8nk9k5oAAAAJ&hl=en&oi=sra https://scholar.google.com/citations?user=8nk9k5oAAAAJ&hl=en...

SilverBirch 4y ago
It's not that the statistics are wrong, it's that you can't apply statistics to prove this at all. Specifically because the person you're analyzing knows all about chess. They know that a super strong weird move will just expose them, so instead they're going to pick lines that just slightly increase their strength. This is like the statistical analyses that show election rigging by highlighting a statistically improbable distribution of results - that analysis works if the person rigging the election doesn't consider the statistical analysis when they're doing their cheating, it is completely avoidable if you cheat competently. Which brings us back to a sort of basic question - if this guy is cheating, do we think he's doing it fairly competently or not? If he's not cheating, the statistics will show normal play, if he is cheating and he's fairly competent the statistics will show normal play. So what has this analysis done? It's proved that he's not totally incompetent, which we already knew because he's pretty well established as a good chess player even he isn't truly a 2700ELO.
- ChrisLomont 4y ago
  >it's that you can't apply statistics to prove this at all If there are not statistical differences, then there are no performance differences. Cheating by definition should imply performance differences. If he is cheating, then at some point in the future, if that method becomes detectable and he has to stop, then his play will suddenly suffer, which would be more evidence. Claiming that statistics cannot answer this question with statistics is not true. It may be hard, or the current sample too small, but claiming stats is not usable is a misunderstanding of statistics. >This is like the statistical analyses that show election rigging by highlighting a statistically improbable distribution of results This only works on the public, and is not what professional statisticians that analyze elections do. And even here, if the event is rare enough, say 1 part in quadrillions, and the analysis is correct, then yes, we would certainly conclude there was rigging. All human knowledge is statistical. Things we claim to be true are only statistically true to large odds, so even for election rigging, if the stats reach some level of certainty, then it is completely valid proof that would hold in court. The pop idiocy of election rigging claims has never risen to that level. >it is completely avoidable if you cheat competently No, it is not. It may only lower the signal to noise ratio, but there is still detectable differences. If you continually improve the statistics and are forced to lower the signal, eventually the signal would be so low as to not affect the system, which in this case is chess games. Physics, for example, can tease events out of on the order of 1 part in trillions and demonstrate signal. Plenty of other things do the same.