3 ms·

What it looks like to cheat is more than just playing relatively accurate moves. Average move time, centipawn loss over multiple games, blunder/mistake frequenc

What it looks like to cheat is more than just playing relatively accurate moves. Average move time, centipawn loss over multiple games, blunder/mistake frequency across multiple games, strengh of moves while in time trouble, etc. Cheaters tend to stand out when you look at a short history of games.

personjerry 4y ago
- Snowflame 4y ago
  Are you trolling? No it isn't? If nothing else, the time issue is a huge one. Cheaters have very consistent seconds-per-move while actual masters make obvious moves instantly and pause on tougher moves.
  - hgazx 4y ago
    I suppose that only completely newbie cheaters would think of taking the same amount of time to make each move. It’s like the first rule of pretending to be a human: add a random delay to all of your actions.
    - synu 4y ago
      In general you’re right, but there are some tells because humans don’t have a random delay. It’s connected with the complexity (from the perspective of how humans think, which is different from engine calculation) of the position. One of the things that makes cheaters stand out is they will have a random weird delay on various obvious moves.
    - IntrepidWorm 4y ago
      And yet time and time again, chess streamers and content producers run across people with newly formed accounts and perfect win records playing the best engine move at regular measured intervals. If you are cunning enough to hold off on the best move for a few extra seconds to appear unaided by an engine, or if you blunder X% moves in your game (or simply play the 3rd or 4th rated move which still probably wins against most humans), chances are you'd do fine learning some chess strategy and playing the game unaided. People keep asking why one would cheat at chess. I'm sure there are some bad actors who aim to disrupt the game, in a manner consistent with cheater motivations in other games. I'd imagine many cheaters are simply looking for some quick dopamine after being frustrated by a plateau in their skill.
    - kelnos 4y ago
      A random delay is also a tell, though. An obvious move shouldn't take 15 seconds, but if that's what the random delay for that move is, that looks suspicious. A more difficult move shouldn't take only 3 seconds, but if that's what the random delay for that move is, that also looks suspicious.
      - xh-dude 4y ago
        Definitely, random time is adding no correlation where it would count. Move time is an easily captured psychometric observation, the clever bit is that it’s intermingled with automated chess analysis. Feels like there could be a lot of surprising inferences to think about here … just a few quick thoughts - how long does someone pause after a blunder, how does one react to unpredictable moves. Can definitely imagine AI being of significant utility here.
- CharlesW 4y ago
  I thought it was a reasonably good answer considering that the question doesn't explicitly spell out why it might flag someone like Magnus Carlsen (who I learned is a grandmaster and World Chess Champion). To rephrase it, Irwin doesn't flag people for playing well — it alerts moderators when it finds suspicious signals across several dimensions.
- mlyle 4y ago
  It is a comprehensive answer. Top human players tend to not make the moves engines make. They also stay relatively strong when under time pressure, where lots of humans reentering moves from a computer fail. Playing someone taking 10 seconds for each move— whether there is only one valid move to capture back or the situation is complicated— you get suspicious. And then when they forget how to play when they have 3 seconds per move stands out.
  - Nerada 4y ago
    Chess.com has a measurement for how accurate a player plays compared to top engine moves called Computer Aggregated Precision Score (or "CAPS"), and top humans do play incredibly accurately. Magnus has CAPS of 98.36 for example. https://www.chess.com/article/view/who-was-the-best-world-chess-champion-in-history https://www.chess.com/article/view/who-was-the-best-world-ch...
    - mlyle 4y ago
      Note that even his "top engine match" is 85%. This is not a fantastically high number, given that maybe 40% of moves are obvious/must-moves. I bet I match the engine ~60%. I also think this is his classical games, not rapid, blitz, or bullet. Top human players tend to make safer, lower variance moves than things engines prove super safe on evaluation. They may give up a few centipawns for ease of analysis.
    - DSMan195276 4y ago
      I think that as a measure of skill vs. a computer that statistic doesn't tell you all that much. Note that even in that measure Magnus only picks the top move ~85% of the time, and that includes many moves where the top move is "obvious" which will inflate the average a bit. And it needs to be kept in mind that Magnus cannot win against a modern computer, so clearly that 85% only means so much, computer play is still on a whole different level from human play. Beyond that, time is a very important factor, running out the clock is one of the most effective ways to win against cheating opponents. Magnus only finds the top move ~85% of the time and takes probably 10x to 100x the time to do it. And like the other commenter noted, variance in the move time is an extremely obvious tell. Magnus will blitz out certain moves (even in classical chess) and then think for ~40 minutes in other complicated positions. A computer can do both positions in basically the same amount of time, and cheating players typically don't know when they "should" be thinking vs. playing fast.
tzs 4y ago
That would clearly work for someone who is cheating by letting an engine do all the work, but how about someone who mostly plays the game themselves just using the engine rarely? Give Nakamura a minute with Stockfish any two times of his choosing in each game, and he would have probably won the Candidates. Heck, just give a good player a blunder alert that tells them after they have made a blunder that they have done so and it could make a big difference. There were games in the Candidates where a player would make a blunder that would completely turn the game around if the opponent found the one move that exploited it, but the opponent didn't see it. The first player could have then saved themselves but had not yet realized they blundered so didn't. Then the other player realized what was going on and exploited the blunder.
- Rodeoclash 4y ago
  I don't think it's that helpful if you're letting Stockfish make a move or two for you per game, or at least at the level I'm at. The engine is so good that it often makes moves that are incomprehensible, setting itself up for an attack in n moves where n is often 10+. If you did want to cheat (but what's the point?) a chrome extension that prevented you from making moves where you lost more then some certain amount of centipawns would be the way to do it.
  - throwaway81523 4y ago
    Anand talked about that a while back. He said even 1 bit of information from Stockfish per game would result in a significant amount of rating points, and be quite hard to detect. I.e. you are in a position where you could choose to play a solid move, or alternatively to launch a risky combination. Stockfish explores the combination and gives you a 1 or 0 saying whether it will work. I believe cheat detection is done partly by humans. Computers flag something as suspicious and then they show the suspicious moves to some grandmasters who might immediately say "no human would do that", or else "yeah that move looks weird but I could imagine someone making it", that type of thing. There was a youtube of Nakamura looking at such a position a while back. The person had a chance to sac some material in order to simplify to a trivially winning endgame, but instead carried out a ridiculously complicated maneuver that kept the material. Just the sort of thing a computer would do, and Naka pointed it out.

personjerry 4y ago

Snowflame 4y ago
Are you trolling? No it isn't? If nothing else, the time issue is a huge one. Cheaters have very consistent seconds-per-move while actual masters make obvious moves instantly and pause on tougher moves.
- hgazx 4y ago
  I suppose that only completely newbie cheaters would think of taking the same amount of time to make each move. It’s like the first rule of pretending to be a human: add a random delay to all of your actions.
  - synu 4y ago
    In general you’re right, but there are some tells because humans don’t have a random delay. It’s connected with the complexity (from the perspective of how humans think, which is different from engine calculation) of the position. One of the things that makes cheaters stand out is they will have a random weird delay on various obvious moves.
  - IntrepidWorm 4y ago
    And yet time and time again, chess streamers and content producers run across people with newly formed accounts and perfect win records playing the best engine move at regular measured intervals. If you are cunning enough to hold off on the best move for a few extra seconds to appear unaided by an engine, or if you blunder X% moves in your game (or simply play the 3rd or 4th rated move which still probably wins against most humans), chances are you'd do fine learning some chess strategy and playing the game unaided. People keep asking why one would cheat at chess. I'm sure there are some bad actors who aim to disrupt the game, in a manner consistent with cheater motivations in other games. I'd imagine many cheaters are simply looking for some quick dopamine after being frustrated by a plateau in their skill.
  - kelnos 4y ago
    A random delay is also a tell, though. An obvious move shouldn't take 15 seconds, but if that's what the random delay for that move is, that looks suspicious. A more difficult move shouldn't take only 3 seconds, but if that's what the random delay for that move is, that also looks suspicious.
    - xh-dude 4y ago
      Definitely, random time is adding no correlation where it would count. Move time is an easily captured psychometric observation, the clever bit is that it’s intermingled with automated chess analysis. Feels like there could be a lot of surprising inferences to think about here … just a few quick thoughts - how long does someone pause after a blunder, how does one react to unpredictable moves. Can definitely imagine AI being of significant utility here.
CharlesW 4y ago
I thought it was a reasonably good answer considering that the question doesn't explicitly spell out why it might flag someone like Magnus Carlsen (who I learned is a grandmaster and World Chess Champion). To rephrase it, Irwin doesn't flag people for playing well — it alerts moderators when it finds suspicious signals across several dimensions.
mlyle 4y ago
It is a comprehensive answer. Top human players tend to not make the moves engines make. They also stay relatively strong when under time pressure, where lots of humans reentering moves from a computer fail. Playing someone taking 10 seconds for each move— whether there is only one valid move to capture back or the situation is complicated— you get suspicious. And then when they forget how to play when they have 3 seconds per move stands out.
- Nerada 4y ago
  Chess.com has a measurement for how accurate a player plays compared to top engine moves called Computer Aggregated Precision Score (or "CAPS"), and top humans do play incredibly accurately. Magnus has CAPS of 98.36 for example. https://www.chess.com/article/view/who-was-the-best-world-chess-champion-in-history https://www.chess.com/article/view/who-was-the-best-world-ch...
  - mlyle 4y ago
    Note that even his "top engine match" is 85%. This is not a fantastically high number, given that maybe 40% of moves are obvious/must-moves. I bet I match the engine ~60%. I also think this is his classical games, not rapid, blitz, or bullet. Top human players tend to make safer, lower variance moves than things engines prove super safe on evaluation. They may give up a few centipawns for ease of analysis.
  - DSMan195276 4y ago
    I think that as a measure of skill vs. a computer that statistic doesn't tell you all that much. Note that even in that measure Magnus only picks the top move ~85% of the time, and that includes many moves where the top move is "obvious" which will inflate the average a bit. And it needs to be kept in mind that Magnus cannot win against a modern computer, so clearly that 85% only means so much, computer play is still on a whole different level from human play. Beyond that, time is a very important factor, running out the clock is one of the most effective ways to win against cheating opponents. Magnus only finds the top move ~85% of the time and takes probably 10x to 100x the time to do it. And like the other commenter noted, variance in the move time is an extremely obvious tell. Magnus will blitz out certain moves (even in classical chess) and then think for ~40 minutes in other complicated positions. A computer can do both positions in basically the same amount of time, and cheating players typically don't know when they "should" be thinking vs. playing fast.

tzs 4y ago

That would clearly work for someone who is cheating by letting an engine do all the work, but how about someone who mostly plays the game themselves just using the engine rarely? Give Nakamura a minute with Stockfish any two times of his choosing in each game, and he would have probably won the Candidates. Heck, just give a good player a blunder alert that tells them after they have made a blunder that they have done so and it could make a big difference. There were games in the Candidates where a player would make a blunder that would completely turn the game around if the opponent found the one move that exploited it, but the opponent didn't see it. The first player could have then saved themselves but had not yet realized they blundered so didn't. Then the other player realized what was going on and exploited the blunder.

Rodeoclash 4y ago
I don't think it's that helpful if you're letting Stockfish make a move or two for you per game, or at least at the level I'm at. The engine is so good that it often makes moves that are incomprehensible, setting itself up for an attack in n moves where n is often 10+. If you did want to cheat (but what's the point?) a chrome extension that prevented you from making moves where you lost more then some certain amount of centipawns would be the way to do it.
- throwaway81523 4y ago
  Anand talked about that a while back. He said even 1 bit of information from Stockfish per game would result in a significant amount of rating points, and be quite hard to detect. I.e. you are in a position where you could choose to play a solid move, or alternatively to launch a risky combination. Stockfish explores the combination and gives you a 1 or 0 saying whether it will work. I believe cheat detection is done partly by humans. Computers flag something as suspicious and then they show the suspicious moves to some grandmasters who might immediately say "no human would do that", or else "yeah that move looks weird but I could imagine someone making it", that type of thing. There was a youtube of Nakamura looking at such a position a while back. The person had a chance to sac some material in order to simplify to a trivially winning endgame, but instead carried out a ridiculously complicated maneuver that kept the material. Just the sort of thing a computer would do, and Naka pointed it out.