6 ms·

An astonishing insert: >A good example of human exploitation of the engine's failings is GM Hikaru Nakamura's defeat of Rybka in the following three-minute bli

by airza 5y ago

An astonishing insert: >A good example of human exploitation of the engine's failings is GM Hikaru Nakamura's defeat of Rybka in the following three-minute blitz game from the Internet Chess Club. Nakamura cleverly locks the position so that progress is impossible for Rybka, then he offers two exchange sacrifices to the engine. With the position locked, the engine's rooks have no value, but the engine thinks it has a winning material advantage. With a draw by the 50-move rule approaching, the engine sacrifices a pawn to avoid a draw, but this proves a huge mistake as Nakamura is then able to win the game, an incredible achievement!

someguy101010 5y ago
There is a video of this on YouTube https://youtu.be/DTMnrUdd8Tw https://youtu.be/DTMnrUdd8Tw
- Clewza313 5y ago
  Thanks for the link. The commentary notes that this was possible because Rybka is an algorithmic engine with hard constraints like "must try to win in 50 moves to avoid tie", and this is what Nakamura exploited to win. DeepMind and other deep learning/neural network approaches don't have hard rules like this.
  - gpm 5y ago
    > because Rybka is an algorithmic engine with hard constraints like "must try to win in 50 moves to avoid tie" That rule isn't the engine's constraint, that's a rule of chess. 50 moves without a pawn move or piece capture (basically "irrevocable progression of the game") is a draw. The chess engine thought it was winning, so it wanted to avoid a draw, even if it meant putting itself in a still winning but otherwise worse situation. Unfortunately for the chess engine it was mistaken, in the original position it wasn't winning, but drawing, and in the new position it wasn't still winning, but losing.
    - nvader 5y ago
      Thank you.
    - aaron695 5y ago
      > 50 moves without a pawn move or piece capture (basically "irrevocable progression of the game") is a draw. A player may call a draw if they wish. This game was also in 2008, I don't think cell phones could even beat all humans beings back then. These days a human couldn't draw against a cellphone, contrary to the article.
      - stouset 5y ago
        And Rybka thought, materially, it was winning, so didn’t want to allow Nakamura to be able to call the game a draw. So it lost a “small” amount of material which transposed the game from one that was an obvious draw to one that was obviously winning for Nakamura. That’s the “contempt” effect that Nakamura kept referring to: the computer would rather continue the game in a slightly-less-good position than allow a draw from a better position. Nakamura exploited the fact that it was wrong about its assessment due to the horizon effect. A traditional chess engine evaluates a position by starting from the current position and evaluating promising moves downward along the tree. In a locked-up position like the one linked, the computer has to evaluate an enormous number of positions, none of which will materially affect the game due to the inability for either side to force meaningful progress. A human can understand the themes in a position and understand where they want the pieces to go, then work backward from there to see a path to achieve it. This is particularly easy when the opponent has no achievable setup of their pieces where they could counter you once you’ve achieved that position. In such a situation, you can (almost) completely discount any moves they make and only consider the path for you to get to your ideal position. Even an beginner-intermediate chess player (1600+) could analyze this type of position easily. This was such a situation. Rybka had no moves to progress things (other than the game-losing sacrifice Nakamura wanted to induce). So Nakamura didn’t have to evaluate any of Rybka’s moves, he just had to put his pieces in the right place to be ready for the inevitable blunder when Rybka decided to sacrifice a pawn to keep its “winning” position thanks to contempt.
  - OhNoMyqueen 5y ago
    Was it actually configured with "no draw" policy? The chess.com explains that Rybka thinks it's winning because of material advantage.
anitil 5y ago
Might need a glitchless chess league!
why_only_15 5y ago
Why did Rybka sacrifice the pawn to avoid the draw if the resulting position was bad? Was its contempt setting really high or something? Edit: looks like in the video Nakamura says the contempt factor was too high, and later it was changed.
- murat131 5y ago
  Correct. Rybka believes it has the piece superiority and doesn't consider sacrificing a pawn to be an issue but the only pawn that it can sacrifice opens up that side of the board for black. Nakamura explains a little before the sacrifice that if they exchanged queens it'd be almost certain that Rybka would have to sacrifice that pawn to avoid draw.
kohlerm 5y ago
Looks to me that a modern AlphaZero based engine like Leela would understand this position easily. Even SF with its new neural net based eval function might understand it. My Leela reports a close to 0 evaluation after Nakamuras first sacrify.

by airza 5y ago

someguy101010 5y ago

There is a video of this on YouTube https://youtu.be/DTMnrUdd8Tw https://youtu.be/DTMnrUdd8Tw

Clewza313 5y ago
Thanks for the link. The commentary notes that this was possible because Rybka is an algorithmic engine with hard constraints like "must try to win in 50 moves to avoid tie", and this is what Nakamura exploited to win. DeepMind and other deep learning/neural network approaches don't have hard rules like this.
- gpm 5y ago
  > because Rybka is an algorithmic engine with hard constraints like "must try to win in 50 moves to avoid tie" That rule isn't the engine's constraint, that's a rule of chess. 50 moves without a pawn move or piece capture (basically "irrevocable progression of the game") is a draw. The chess engine thought it was winning, so it wanted to avoid a draw, even if it meant putting itself in a still winning but otherwise worse situation. Unfortunately for the chess engine it was mistaken, in the original position it wasn't winning, but drawing, and in the new position it wasn't still winning, but losing.
  - nvader 5y ago
    Thank you.
  - aaron695 5y ago
    > 50 moves without a pawn move or piece capture (basically "irrevocable progression of the game") is a draw. A player may call a draw if they wish. This game was also in 2008, I don't think cell phones could even beat all humans beings back then. These days a human couldn't draw against a cellphone, contrary to the article.
    - stouset 5y ago
      And Rybka thought, materially, it was winning, so didn’t want to allow Nakamura to be able to call the game a draw. So it lost a “small” amount of material which transposed the game from one that was an obvious draw to one that was obviously winning for Nakamura. That’s the “contempt” effect that Nakamura kept referring to: the computer would rather continue the game in a slightly-less-good position than allow a draw from a better position. Nakamura exploited the fact that it was wrong about its assessment due to the horizon effect. A traditional chess engine evaluates a position by starting from the current position and evaluating promising moves downward along the tree. In a locked-up position like the one linked, the computer has to evaluate an enormous number of positions, none of which will materially affect the game due to the inability for either side to force meaningful progress. A human can understand the themes in a position and understand where they want the pieces to go, then work backward from there to see a path to achieve it. This is particularly easy when the opponent has no achievable setup of their pieces where they could counter you once you’ve achieved that position. In such a situation, you can (almost) completely discount any moves they make and only consider the path for you to get to your ideal position. Even an beginner-intermediate chess player (1600+) could analyze this type of position easily. This was such a situation. Rybka had no moves to progress things (other than the game-losing sacrifice Nakamura wanted to induce). So Nakamura didn’t have to evaluate any of Rybka’s moves, he just had to put his pieces in the right place to be ready for the inevitable blunder when Rybka decided to sacrifice a pawn to keep its “winning” position thanks to contempt.
- OhNoMyqueen 5y ago
  Was it actually configured with "no draw" policy? The chess.com explains that Rybka thinks it's winning because of material advantage.

anitil 5y ago

Might need a glitchless chess league!

why_only_15 5y ago

Why did Rybka sacrifice the pawn to avoid the draw if the resulting position was bad? Was its contempt setting really high or something? Edit: looks like in the video Nakamura says the contempt factor was too high, and later it was changed.

murat131 5y ago
Correct. Rybka believes it has the piece superiority and doesn't consider sacrificing a pawn to be an issue but the only pawn that it can sacrifice opens up that side of the board for black. Nakamura explains a little before the sacrifice that if they exchanged queens it'd be almost certain that Rybka would have to sacrifice that pawn to avoid draw.

kohlerm 5y ago

Looks to me that a modern AlphaZero based engine like Leela would understand this position easily. Even SF with its new neural net based eval function might understand it. My Leela reports a close to 0 evaluation after Nakamuras first sacrify.