7 ms·
Is Hans Niemann cheating? – Expert analyzes
- deleted 4y ago[deleted]
- jstx1 4y agoFabiano Caruana (former number 2 and World Championship contender) is skeptical of Regan's analysis: > I would take Regan’s analysis with the large grain of salt, and the reason why is not because I have any insight into his algorithm or his methods, but because I know of a case of, a very high profile case, where with absolute certainty I can say that someone was cheating in an important event. And the person was investigated and was also exonerated based on Regan’s analysis. And I am certain that there was cheating. There is no doubt in my mind that this person was cheating and they got away with it. https://www.chessdom.com/fabiano-caruana-i-would-take-regans-analysis-with-a-large-grain-of-salt/ https://www.chessdom.com/fabiano-caruana-i-would-take-regans...
- hugh-avherald 4y agoI believe Fabi, but the question must be asked: how does he know (with certainty!) that the player was cheating. Did the player confess? Was he blatantly looking on his phone in front of him?
- bpodgursky 4y agoMaking moves perceived to be outside his skill class I assume.
- parksoha 4y agoMove in a critical moment, speed of decision for said move, cheating at this level is probably extremely hard to do it alone which already creates space for leaks...
- rob74 4y agoIf it's that, then it's pretty arrogant - I mean, that's like saying: "he's too stupid to make such a good move, so he must be cheating"? Even (relatively) bad players are bound to make good moves from time to time, either through dumb luck or through occasional sparks of inspiration, so that's hardly a reason to be 100% sure they are cheating...
- bpodgursky 4y agoWhen you are #2 in the world at something you are allowed to be a bit arrogant about it. (down to #8 now, I suppose)
- deleted 4y ago[deleted]
- John23832 4y agoThere are moves and lines that only Super GM’s will calculate. Not IMs, not FMs, not GMs. It’s not arrogant, it’s the level of play.
- jfengel 4y agoNot so much "outside his skill class" as "unlikely to be played by a human". Computers have a really good idea of what the optimal move is, and it's sometimes not a move any human being would play. The main way to detect cheating is to figure out that they're playing closer to the computer's line than anybody without computer assistance. In this case, his off-the-board behavior, where he seems unable to explain some of the moves he made, is also suspicious. His skill set is such that there's no move he's not capable of playing, though he should at least be able to tell you what he was thinking when he played it.
- jstanley 4y agoMaybe it was Fabi!
- michaelt 4y agoImagine I see someone after an event, flushing evidence of cheating. I am completely certain, because I saw the evidence myself. But to anyone else? There's no hard evidence - just he-said-she-said. Maybe I was mistaken - or trying to get my rival into trouble. Knowing how it would look, maybe I keep my mouth shut rather than throwing around wild, baseless allegations about cheating.
- bombcar 4y agoProving a positive result is much easier than a negative. A positive “this move right here is not reasonable without a chess computer suggestion ♟” vs “we didn’t find anything suspicious”. And if you’re cheating, adding another layer of “try to find moves that aren’t suspicious” would be theoretically possible.
- roenxi 4y agoThat isn't really proving a positive though - maybe the player just got lucky. There aren't that many moves in any given turn, sometimes even a novice will make an inspired move.
- bombcar 4y agoTrue - but identifying a particular move gives a discussion point at least.
- deleted 4y ago[deleted]
- debacle 4y agoThe other factor is that Regan's process is designed to not produce false positives - if it says you are cheating, you are almost guaranteed to be cheating.
- bombcar 4y agoExactly - the lack of evidence isn't evidence of lack; a common mistake. All we know now is that an "obvious" method of cheating wasn't used. That may be because there was no cheating at all, or a new method of cheating has been developed.
- EarthIsHome 4y agoRegan's algorithm cannot prove that you did not cheat; it can only fail to prove that you cheated. The question that Regan's algorithm is this: "Did you cheat?" If the algorithm comes back with "Yes," then you cheated. But it cannot come back with "No." It can only come back with "I failed to prove that you were cheating." You need another hypothesis: "Did you not cheat?" But this is an impossible test.
- jtchang 4y agoLike a bloom filter!
- cosentiyes 4y agoAgreed, this is also likely a high precision, low recall approach to avoid false positives ruining a player's career
- gsinclair 4y agoHowever, allowing cheaters to get away with it could eventually ruin chess — that is, ruin all players’ careers. A delicious conundrum.
- deleted 4y ago[deleted]
- humanistbot 4y agoThe fact that Caruana has not said why he has "absolute certainty" the person cheated is cause to be skeptical of Caruana's skepticism.
- 0cVlTeIATBs 4y agoI've heard of two ways one might cheat in chess. This analysis covers receiving engine moves, but does not address the other accusation, stealing opponents' prep work.
- buzzdenver 4y agoStealing prep might work in something like a world championship game where the two participants play dozen or more games. It would be much harder to steal the prep of half of your opponents in a round robin tournament. Also, if you assume that Niemann is cheating and his real rating is 2450, not high 2600s like it is officially now, then no amount of prep stealing will make him competitive with Magnus.
- Mizza 4y agoBut that's literally the situation. He did beat Magnus, claims that he happened to study deep into a fairly esoteric line that Magnus played, then seemed unfamiliar with the variations of it when asked about it in a later interview. Magnus seems to think Neimann's coach - another cheater - got information about Magnus' team's preparation.
- buzzdenver 4y agoThat explains one game, but he consistently seems to play at the high 2600 level.
- Elhana 4y agoI guess when doing prep work, you analyze most of the top moves computer would suggest, so simply stealing it won't help that much, unless you really missed something. You'd need to find some move/line that your opponent has not prepared, analyze it deeper than he did and hope that actual game goes exactly as planned to implement it. The deeper it goes into a game - less chances you end up with that position on the board. Also accusing someone should be backed by some kind of proof. Even a subpar player with some luck might memorize something a computer would suggest and implement it, despite that it is way over his expected level, yet not getting actual suggestions during a game. Is that cheating? There are lots of videos on youtube like "This trap will help you win lots of games", that 600 elo player can memorize and then get labeled as cheater, coz it's way over his elo level. Next step would be to ponder whether computer assisted prep in general is cheating or not, really.
- twawaaay 4y agoI think the discussion misses the important problem. It is possible to design a system where the algorithm will try to imitate a human being making human errors. There is, I think, couple of ways to do it. One way would be to not be greedy and only get help from time to time. So if I was the player I would not rely for the algorithm for every move, just sometimes. Just enough so that it makes a difference. If you remember, Allies broke Enigma but decided to restrict the use of intelligence thus obtained to not alert Axis to the fact. Which is essentially the same idea here -- the question is "How much help can I use before it becomes provable that I am getting help?"
- Bakary 4y agoThere are already chessbots that can mimic medium-strength human players thanks to ML
- tbrake 4y ago> There is, I think, couple of ways to do it. One way would be to not be greedy and only get help from time to time. So if I was the player I would not rely for the algorithm for every move, just sometimes. Just enough so that it makes a difference. This is something Magnus has said before as well (paraphrasing): "If I got engine help for just one or two positions a game I would win every time."
- bombcar 4y agoGood players could cheat with a bot. Great players know exactly when they’d use a bot on a single move.
- Bakary 4y agoAt their level of play, a single one-bit message can tip the scales in dramatic fashion. All you have to know is that there is a great move to be found in a particular position, without even knowing what the move is.
- tbrake 4y agoCorrect. They're often already playing at ~95% engine matching moves and/or with negligible difference in centipawn loss. Getting the help to push that to 98% would be a dominant advantage. Funnily (to me) post-game analysis often highlights the computers finding incredible defensive resources instead of attacking ones. But it's exactly these kinds of moves - the fearless counter-thrust to equality while your king's position looks hopeless - that are the most inhuman of all and likely to draw suspicion.
- xlii 4y agoWhile I'm not interested in chess in general, I'm deeply curious how Carlsen (assuming good intent) could figure out Niemann was cheating. The ideas I have right now would be: 1. To run each early game phase (up to, let's say 3-5 moves) through multiple AI engines and find selection that is: - happening constantly in most of the engines - don't have deviation in multiple rules (i.e. have stable consequence) - aren't well known (that is, aren't well known plays described in literature) From that probably I'd have some honeypot sequence that I could try to lure cheater out. 2. Find opponent's openings that are: - Successful (i.e. usually winning) - Not recommended by any AI engine whatsoever (so probably could be described as "lucky") - Used often by opponent ...and try to get him into this exact opening seeing how it'll go. Beyond discovery I find also immensely interesting how to deal with such knowledge. Exposing method would put it in statistical argument of outcome vs chance and would be super easy to walk around in the future. Proving it would true would be almost impossible as well. In the end the only route would be to push hard (exactly as Carlsen did) with a method defendable enough that if there's push-back it can be exposed to save oneself from disgrace. I doubt we'll ever see resolution, but I find all of it amazing nonetheless.
- Bakary 4y agoThe most convincing theory around this drama is that Magnus employed just such a honeypot by playing an obscure move Niemann responded perfectly to but was unable to later explain.
- derbOac 4y agoThis has been on my mind in reading about this. It would explain a lot.
- Elhana 4y agoOn the other hand, if Niemann did exactly the same and researched less-known moves, he'd know how to play against them too. That won't actually prove he is cheating at that game. Just the fact that he analyzed them earlier. You'd need a significant number of such honeypot "incidents" to properly accuse someone of cheating.
- anothernewdude 4y agoA bell curve. That's it. Is Hans Niemann cheating? Ken Regan is cheating people into calling him an expert.
- thedstrat 4y agoAgreed. The analyses based solely on a comparison of average centipawn loss is so so flawed. It only takes using an engine move once or twice to completely demolish a much better opponent in a game. These types of analyses don't find this type of cheating. A much better analysis imo would be trying to find the probability of someone at his ELO finding surprising moves. EG I played a 1900 online recently who happened to completely turn around a game by setting up a forced mate in 6, with several branching moves a few moves down which all happened to result in mate because of incredibly lucky piece positions. I can't calculate the probability of someone at a relatively low level like that finding such a move, but I bet it's very low. This is the type of analysis which I'm guessing Magnus is using to assess Niemann as a cheater.
- ChrisLomont 4y ago>It only takes using an engine move once or twice to completely demolish a much better opponent in a game. That's not true. Pick an engine, set to a few hundred points above your strength, then try to beat it using only one or two moves from an engine. You will lose nearly every game, because so many of your other moves will be so below the other player that the 1-2 good moves cannot make it up. This is demonstrated quite often by the games where GMs are "helped" by others in multi-player games, and it shows that help against a much better opponent takes far more than 1-2 moves. Among really close players it helps. But not once you get a few 100 ELO points apart.
- agarden 4y agoRegan's analysis works, in part, the way you suggest is better. He looks for tricky moves with non-obvious consequences and looks at one's success rate in those. For the Niemann-Carlsen game, he identified two such moves and Niemann chose suboptimally for both of them. For one is those, Niemann played a nice that could have cost him a tempo. Definitely an inaccuracy.
- dandanua 4y agoChess became a joke today. People decide if you cheat or not just based on your moves and results. It's nonsense. It's like deciding if an athlete is cheating or not without doping tests - solely based on the "distribution" of success. And Magnus behavior - hinting accusations again and again - is far from ethical too.
- mmmpop 4y agoI'll give you an upvote purely because I too am curious about your premise and that's definitely going to be a 3rd rail here. However, I'm not close enough to chess to understand his accusations. When you're the greatest thing in [insert topic] for the past 10 years, you get leeway on those sorts of things.
- notafraudster 4y agoI think people decided he’s cheating because he admitted to cheating the two times he got caught cheating online, denied ever cheating besides those times, has had a meteoric rise in terms of moving from low GM to his current state, was unable to explain his play in a reasonable way, played almost perfectly, beat a substantially better player, and that player feels he cheated despite having never made accusations like that against people before. I’m not saying it’s a slam dunk because it’s not. I can respect people who come down in either side based on the current evidence. But it seems like willful ignorance to suggest that none of the above has any bearing at all.
- mmmpop 4y agoI think people are right to suspect him of cheating given everything you said. Reading through your list, I can't help but be fascinated by the amount of hubris it would take to actually cheat as such, and then proceed to live with yourself. Hell, given his comments, he seems pleased. We could be dealing with a real sicko.
- dandanua 4y ago1. Cheating in online chess as a teen is a factor, but not that strong. Also, the details of cheating are still private, so we can't rely on that. BTW, chess.com belongs to Magnus, partially. 2. It's normal for teens to have jumps in their development. 3. I watched his explanations, they are not that great, but not unreasonable either. 4. They played a couple of weeks before, with the result 3-1 for Magnus [1]. Magnus lost the first game and won other 3. Magnus losing to Niemann wasn't something extraordinary in general. Also, a player can have much better morale while playing with a King, it simply gives more motivation. 5. Magnus hinting cheating accusations is certainly a new thing. But refusing to defend his World Champion title this year is also a new thing. I guess he started to feel way too entitled in chess. Also, all this affects his position in chess business, since he owns a lot of it. He is not just a player anymore. It's a conflict of interests. [1] https://en.wikipedia.org/wiki/Champions_Chess_Tour_2022#FTX_Crypto_Cup https://en.wikipedia.org/wiki/Champions_Chess_Tour_2022#FTX_...
- spywaregorilla 4y agoI think this is being approached incorrectly. The thing that stands out to me is that people find his analysis of games to be poor. If he can't explain his reasoning, it suggests he may not have reasoned it. This is interesting as there's two sides to this problem. One is that he is good at chess and could be cheating. The other is that he is actually not great at chess and could be cheating. If people think his analysis skills are weak, then it suggests the latter case, which seems easier to test.
- frodetb 4y agoI wonder if the explanation for Carlsen's move could be far simpler than people are making it out to be. This is the guy who decided not to compete for the world title, simply because he didn't feel like it. He loves chess, and wants to keep playing it for the joy that it gives him, and nothing else. He could have resigned early simply because going through with the game would have left a bad taste in his mouth. I'll bet it's annoying to play someone with a history of cheating, regardless of whether they are still currently known to be breaking the rules or not. Imagine having to sit through a game that you love to play, only with a nagging thought in the back of your mind that sucks the joy out of the experience and turns it into a chore.
- jjulius 4y ago>Imagine having to sit through a game that you love to play, only with a nagging thought in the back of your mind that sucks the joy out of the experience and turns it into a chore. That's a completely reasonable position for Magnus to take. It would be even more reasonable for him to simply state that that's why he's acted as such, and yet...
- zug_zug 4y agoI imagine he'd open himself up to a lawsuit if he accused a player of cheating without hard evidence.
- jjulius 4y agoAccusing him of cheating is not at all what we were talking about.
- gsinclair 4y agoPerhaps it is also reasonable for Magnus to wait until the tournament is over before making such a statement.
- Chabsff 4y agoEarly on in this debacle, I remember someone pointing out that Niemann's performance could be explained relatively easily by him having forward knowledge that Carlsen intended to use the unusual opening he went for. This would amount to cheating "in spirit", by way of having an unfair advantage over his opponent, while still being tournament legal. Whatever happened to this version of events? It struck me as being a lot more credible than engine assistance.
- Bakary 4y agoIt's still widely talked about. It's just that the total amount of drama is overpowering.
- numlocked 4y agoI also thought it sounded like a plausible explanation, but it's been dismissed as exceedingly less likely than engine assistance by those who know far more than I. At the Sinquefield (the tournament in question), Magnus had just one team member with him and it's someone who has worked with for many years. The breakdown in the logic of the 'leaked prep' is that no one can come up with a plausible motivation for it to have been leaked - and no serious chess GMs seem to think it is a plausible theory.
- Chabsff 4y agoSounds about right. Thank you.
- lizardactivist 4y agoI think he is, but at this point we will never know for sure unless he admits it. But I think now that the eyes of the chess world are upon him, he will suddenly start performing much worse.
- throw7 4y agoYou either have evidence or you don't. This quasi economic modeling of chess players is ridiculous. He says his models don't know anything about playing chess. OK, let's apply your models to all sorts of areas where we suspect fraudulent play. This chess drama is corrosive. Speculation from day 1 starting with carlsen.
- maxbond 4y agoThe model described here seems. Like bunk. It set of alarm bells for me when they said it wasn't based on an understanding of chess. But regarding whether it's possible to make such an argument, I'd refer you to the more compelling case the moderators of a Minecraft speedrunning forum made against the steamer Dream. https://youtu.be/-MYw9LcLCb4 https://youtu.be/-MYw9LcLCb4 The allegation is that the drop rates of key items were manipulated. It's a statistical argument, but it's founded in the mechanics of the game. After offering counterarguments and denying it for months, Dream eventually admitted to it, though they claim it was an accident (in a manner which is plausible, though I personally doubt it).
- ChrisLomont 4y ago>You either have evidence or you don't. This quasi economic modeling of chess players is ridiculous Modeling provides the best evidence in all aspects of life (design, science, invention, marketing, medicine, ....). All human knowledge is built in modeling and statistical evidence. Nothing is 100% certain except mathematics, and even that is often fuzzier than 100%. So how is modeling yet another thing ridiculous when it provides empirically the best methods in so many other domains?
- qurt 4y agoThis is a fair point, but I think there is confusion around the meaning of the fact that Regan's models do not "know anything about chess". I believe this refers to prior domain knowledge embedded in the algorithm itself - but the idea of exploiting data from a large number of GM games is precisely so the model can extract domain "knowledge" from statistical indicators. As for "applying the model to all sorts of areas", yes conceptually if your model can infer anything from any kind of data distribution this applies. In practice it just won't. If you take a step back and look at the AlphaGo project, the story shows us a very similar model architecture could be applied to a different game, chess, with AlphaZero. And the "zero" in this case means the model is build with zero human knowledge - not even GM games, only self-play. These models do entirely different things - A0'model given a position, tries to find the best lines, while KR's tool, given a set of games from a given player, will try to price the probability of foul play - but in both cases, no programmer-introduced chess heuristics are part of the model. Yet AlphaZero performs the way it does and appears to "know" quite a bit about the game. So if a purely statistical approach can yield a very strong engine, which has been qualified as displaying some sort of "creativity" (for lack of a better term) in its play, I don't think it is too far-fetched to imagine other models extracting knowledge about what human play should look like. Totally agree that this is wrong from Magnus, and cheating from Niemann or not he is responsible for the toxic drama.
- Double_a_92 4y agoThe problem is that cheating does not only mean that he used a chess computer to calculate the next best move. E.g. just knowing when you are at a critical point and should think for a for moments longer then usual, can already tip the outcome of the game.
- SilverBirch 4y agoI honestly wish that people wouldn't put out rubbish analysis like this. >his conclusion is there is no reason whatsoever to suspect him of cheating. No. His conclusion is that based on this really weird brute force statistical analysis there is no statistical indicator that he cheated. Firstly, how you can state there's no reason whatsoever to suspect him of cheating is absurd. You know he has a previous record of cheating so as a starting point that statement is absurd. But secondly and more importantly, no one is suggesting that Neimann is a moron. We're not saying he's a 1200 ELO lucking out against a grand master. Any account of Neimann cheating is going to include the fact that he's already a very good chess player - he's not going to be cheating through his matches with any average player. And as other GMs have pointed out, cheating could literally involve 1 advantageous decision at a key point in a match. What we have here is very simple. Neimann was quasi-accused of cheating in 1 match, and this genius has gone off and analysed thousands of other matches and come to the conclusion that there's no statistical evidence of cheating. A methodology that is basically designed to ensure almost no one could ever be proven to be cheating using this method.
- kumarvvr 4y agoAwesome take. Reminds me of a book that teaches us how one can lie with statistics. While reading the whole article, I was convinced over the "data centric analysis". Afterall, how can data lie? But, its revealing that cheating in a chess game with a GM is literally making the right guided move at the right time.
- Aromasin 4y agoWhat's ever worse is that FIDE feel justified exonerating people based on this statistical analysis alone, despite the fact there seems to be a general sense within the GM community that something is wrong and has been for a while.
- Ensorceled 4y agoYeah, the "1 in a million" hand-waving is weird. If I had a mechanism that allowed me to cheat at poker, I would only use it on a push. I certainly wouldn't be using every hand or even at all in penny ante games.
- binarymax 4y agoIf you play at a grandmaster level, then sometimes you only need to cheat on one or two critical moves in the game. The rest you can handle yourself. A smart and talented cheater wouldn't use stockfish for 10 to 20 moves in a 90 move game, you only use the game changing move or two. This is likely impossible to catch and prove.
- agarden 4y agoRegan claims that if you consult the computer just three times during a game, he will catch you after three games. If you consult just once, it gets a lot harder.
- puffoflogic 4y agoHe seems to define "consult" as "receive a specific move from the engine". GMs have stated that they could throw a game simply by being alerted that a good move existed. I doubt that would ever show up on a statistical analysis of moves. It would at the very least require an analysis correlating think time, move, and depth of analysis required to correctly analyze the current position. You'd have to say "hey wait player doesn't usually notice there's some X depth analysis opportunity and actually take it." And that doesn't seem to be the analysis which happened here by a long shot.
- Der_Einzige 4y agoAs a guy who tried writing my own chess engine, it's pretty trivial to modify a chess engine in such a way that looking for "chess engine like moves" won't work. Take a high end engine, start messing with it's piece square tables. Modify or turn off the opening book and end game tablebases, and add slight changes to the evaluation function (e.g. slightly modify piece values. Even doing just two of the things listed above is enough to destroy fingerprinting techniques.
- Overtonwindow 4y agoI do not believe Hans Niemann is cheating. I used to play competitive chess, and while I never got above an ELO of 1100, I think Carlsen has made a tremendous blunder.