5 ms·

Unfortunately, Stockfish is not doing too well in the more recognized TCEC unofficial world computer chess championship at http://tcec.chessdom.com/live.php htt

by tromp 9y ago

Unfortunately, Stockfish is not doing too well in the more recognized TCEC unofficial world computer chess championship at http://tcec.chessdom.com/live.php http://tcec.chessdom.com/live.php, due to lackluster performance against the weaker engines. With just over 3 rounds to go, it looks like it's out of the final: N Engine Rtng Pts Gm SB Ho Ko St Fi Ch Gi Bo An 1 Houdini 6.02 3184 17.0 25 187.25 ···· =01 === ==1= 111= 1=1= 1=1 =1== 2 Komodo 1959.00 3232 17.0 25 183.25 =10 ···· ==0 ==1= =1=1 11= =1=1 1=11 3 Stockfish 051117 3228 15.0 24 173.25 === ==1 ···· =1= ==== =11= =1= 1=== 4 Fire 6.2 3112 13.5 25 152.25 ==0= ==0= =0= ···· ==1 ===1 =1= 1==1 5 Chiron 251017 3013 9.5 25 114.00 000= =0=0 ==== ==0 ···· === ==== === 6 Ginkgo 2.01 3052 9.5 25 110.00 0=0= 00= =00= ===0 === ···· ==== 1== 7 Booot 6.2 3091 9.0 24 104.75 0=0 =0=0 =0= =0= ==== ==== ···· === 8 Andscacs 0.921 3100 8.5 25 107.25 =0== 0=00 0=== 0==0 === 0== === ···· It had better win its current game against Booot to keep some very slim chances. Which it now seems to be doing...

_0ffh 9y ago
I suppose I depends on the format. It seems like in a knockout style tournament Stockfish would surely win, as it's the only engine with zero losses. Looks like it might benefit from a more aggressive play style, in order to get more wins against weaker opponents.
- brownbat 9y ago
  There's interesting metagame theory here... It should get more aggressive whenever it's behind its goal rank in the tournament. You can often trade variance for performance in competing strategies. The amount you're trading depends on your opponent's skill and strategy. If an engine has a way to learn the skill of its opponent, that might be another way to determine the ideal level of aggressiveness too. Also if you're behind in a game, cranking variance is often your best shot. Do any engines metagame yet? That could make things crazy.
  - Scarblac 9y ago
    The rules of the tournament forbid it, it's the same program in all rounds and current standings aren't an input. So there is also no luck for any engine getting a favourable order of opponents or so.
    - brownbat 9y ago
      Even just with the input from the current game, the engine could get more aggressive by estimating the skill of its opponent.
tephra 9y ago
Worth noting that Stockfish is actually the only engine that hasn't lost a game. The lackluster performance is it just not beating the weaker engines, something that Komodo and Houdini have done (but they have also lost against stronger engines).
- umanwizard 9y ago
  So can we conclude that Stockfish just plays more "drawishly" than the other top-tier engines? Or is this effect just random chance so far?
  - Recursing 9y ago
    It might have something to do with the "contempt" setting used by the developers for this tornaument
  - pizza234 9y ago
    I'm not an expert, but it may be the equivalent of a strong human defensive player, e.g. Petrosian (https://en.wikipedia.org/wiki/Tigran_Petrosian https://en.wikipedia.org/wiki/Tigran_Petrosian).
  - kregasaurusrex 9y ago
    Scoring conditions for chess tournaments cause forcing a draw to be more favorable than a loss, likely causing the programmers to heavily weigh draws over losses. Contrast this with Go engines where there are no draws, the incentives for finishing chess games in a draw that would otherwise end in a loss is much higher.
  - thom 9y ago
    This is true, Stockfish expects its opponent to play as well as itself (outside of the 'contempt' setting, not sure if that was used or tuned per game). So it won't play edgy lines that invite blunders if they are even a centipawn worse than what it thinks is the 'correct' line. Funnily enough, I feel there's a lesson for startup people in this. I have seen many people (myself included) talk themselves out of big, bold bets just because they are very good at thinking up reasons something should never work.
schoen 9y ago
Although Stockfish beat Booot, it then tied against Komodo, giving Komodo 17.5 Houdini 17 Stockfish 16.5 It's quite bad for Stockfish now because each of these three top engines has 1 remaining remaining match it's pretty likely to win (against Ginkgo, Booot, and Fire, respectively), which if it happens wouldn't change the relative ranking of the top three at all (they would be at 18.5, 18, and 17.5). Also, Stockfish has a match against Houdini, while Houdini and Komodo have a match against each other. Stockfish can still in principle reach the final this way if Houdini and Komodo have a draw, because that would give Komodo 19.5 Stockfish 18.5 Houdini 18.0 assuming all three engines also win their respective remaining matches against non-top-3 engines, and Stockfish defeats Houdini. Very tough for Stockfish at the moment!
- schoen 9y ago
  Booot achieved a draw against Houdini and Ginkgo achieved a draw against Komodo! Therefore Stockfish's potential paths to the final have increased; for example, if Stockfish can beat Houdini and draw against Fire, while Komodo beats Houdini, then Stockfish would be in the final against Komodo. Or if Houdini defeats Komodo and Stockfish wins against both Houdini and Fire, Stockfish would be in the final against Houdini. Edit: But now I'm not sure of the original comment that only the two top engines from this round will go to the final; in previous years there were more rounds before the final, but I don't understand how many of the rounds have been eliminated this year.
  - schoen 9y ago
    People in the TCEC chat definitely seem to agree with the interpretation that only the top two will go to the final. It's kind of down to the wire now: currently Komodo and Houdini lead Stockfish by exactly one point, while Komodo is playing Houdini in the final game for both of them. Stockfish has exactly one game remaining (against Fire). In order to reach the final, Stockfish needs to win its game against Fire, and also Komodo and Houdini need not to draw. Then Stockfish would be tied with whichever of Komodo and Houdini loses the current game, and I guess there would be a tiebreaker prior to the final. If Komodo and Houdini draw now, they're both guaranteed to face each other in the final. People in the TCEC chat are pointing out that if they were human players, they would face a strong incentive to deliberately play for a draw or to accept one if one were offered. But presumably the software has no idea of this!
tromp 9y ago
With only one round left, Stockfish is hanging by a thread: N Engine Rtng Pts Gm SB Ho Ko St Fi Ch Gi An Bo 1 Houdini 6.02 3184 18.0 27 219.00 ···· =01 ==== ==1= 111= 1=1= =1== 1=1= 2 Komodo 1959.00 3232 18.0 27 214.75 =10 ···· ==0= ==1= =1=1 11== 1=11 =1=1 3 Stockfish 051117 3228 17.0 27 215.75 ==== ==1= ···· =1= ==== =11= 1=== =1=1 4 Fire 6.2 3112 15.0 27 180.50 ==0= ==0= =0= ···· ==1= ===1 1==1 =1=1 It needs to beat Fire, while Komodo vs Houdini needs to be decisive. Fingers crossed...

by tromp 9y ago

_0ffh 9y ago

I suppose I depends on the format. It seems like in a knockout style tournament Stockfish would surely win, as it's the only engine with zero losses. Looks like it might benefit from a more aggressive play style, in order to get more wins against weaker opponents.

brownbat 9y ago
There's interesting metagame theory here... It should get more aggressive whenever it's behind its goal rank in the tournament. You can often trade variance for performance in competing strategies. The amount you're trading depends on your opponent's skill and strategy. If an engine has a way to learn the skill of its opponent, that might be another way to determine the ideal level of aggressiveness too. Also if you're behind in a game, cranking variance is often your best shot. Do any engines metagame yet? That could make things crazy.
- Scarblac 9y ago
  The rules of the tournament forbid it, it's the same program in all rounds and current standings aren't an input. So there is also no luck for any engine getting a favourable order of opponents or so.
  - brownbat 9y ago
    Even just with the input from the current game, the engine could get more aggressive by estimating the skill of its opponent.

tephra 9y ago

Worth noting that Stockfish is actually the only engine that hasn't lost a game. The lackluster performance is it just not beating the weaker engines, something that Komodo and Houdini have done (but they have also lost against stronger engines).

umanwizard 9y ago
So can we conclude that Stockfish just plays more "drawishly" than the other top-tier engines? Or is this effect just random chance so far?
- Recursing 9y ago
  It might have something to do with the "contempt" setting used by the developers for this tornaument
- pizza234 9y ago
  I'm not an expert, but it may be the equivalent of a strong human defensive player, e.g. Petrosian (https://en.wikipedia.org/wiki/Tigran_Petrosian https://en.wikipedia.org/wiki/Tigran_Petrosian).
- kregasaurusrex 9y ago
  Scoring conditions for chess tournaments cause forcing a draw to be more favorable than a loss, likely causing the programmers to heavily weigh draws over losses. Contrast this with Go engines where there are no draws, the incentives for finishing chess games in a draw that would otherwise end in a loss is much higher.
- thom 9y ago
  This is true, Stockfish expects its opponent to play as well as itself (outside of the 'contempt' setting, not sure if that was used or tuned per game). So it won't play edgy lines that invite blunders if they are even a centipawn worse than what it thinks is the 'correct' line. Funnily enough, I feel there's a lesson for startup people in this. I have seen many people (myself included) talk themselves out of big, bold bets just because they are very good at thinking up reasons something should never work.

schoen 9y ago

Although Stockfish beat Booot, it then tied against Komodo, giving Komodo 17.5 Houdini 17 Stockfish 16.5 It's quite bad for Stockfish now because each of these three top engines has 1 remaining remaining match it's pretty likely to win (against Ginkgo, Booot, and Fire, respectively), which if it happens wouldn't change the relative ranking of the top three at all (they would be at 18.5, 18, and 17.5). Also, Stockfish has a match against Houdini, while Houdini and Komodo have a match against each other. Stockfish can still in principle reach the final this way if Houdini and Komodo have a draw, because that would give Komodo 19.5 Stockfish 18.5 Houdini 18.0 assuming all three engines also win their respective remaining matches against non-top-3 engines, and Stockfish defeats Houdini. Very tough for Stockfish at the moment!

schoen 9y ago
Booot achieved a draw against Houdini and Ginkgo achieved a draw against Komodo! Therefore Stockfish's potential paths to the final have increased; for example, if Stockfish can beat Houdini and draw against Fire, while Komodo beats Houdini, then Stockfish would be in the final against Komodo. Or if Houdini defeats Komodo and Stockfish wins against both Houdini and Fire, Stockfish would be in the final against Houdini. Edit: But now I'm not sure of the original comment that only the two top engines from this round will go to the final; in previous years there were more rounds before the final, but I don't understand how many of the rounds have been eliminated this year.
- schoen 9y ago
  People in the TCEC chat definitely seem to agree with the interpretation that only the top two will go to the final. It's kind of down to the wire now: currently Komodo and Houdini lead Stockfish by exactly one point, while Komodo is playing Houdini in the final game for both of them. Stockfish has exactly one game remaining (against Fire). In order to reach the final, Stockfish needs to win its game against Fire, and also Komodo and Houdini need not to draw. Then Stockfish would be tied with whichever of Komodo and Houdini loses the current game, and I guess there would be a tiebreaker prior to the final. If Komodo and Houdini draw now, they're both guaranteed to face each other in the final. People in the TCEC chat are pointing out that if they were human players, they would face a strong incentive to deliberately play for a draw or to accept one if one were offered. But presumably the software has no idea of this!

tromp 9y ago

With only one round left, Stockfish is hanging by a thread: N Engine Rtng Pts Gm SB Ho Ko St Fi Ch Gi An Bo 1 Houdini 6.02 3184 18.0 27 219.00 ···· =01 ==== ==1= 111= 1=1= =1== 1=1= 2 Komodo 1959.00 3232 18.0 27 214.75 =10 ···· ==0= ==1= =1=1 11== 1=11 =1=1 3 Stockfish 051117 3228 17.0 27 215.75 ==== ==1= ···· =1= ==== =11= 1=== =1=1 4 Fire 6.2 3112 15.0 27 180.50 ==0= ==0= =0= ···· ==1= ===1 1==1 =1=1 It needs to beat Fire, while Komodo vs Houdini needs to be decisive. Fingers crossed...