4 ms·

Genuinely enjoyable and timely game. Out of prep early, machine evaluations useless throughout, plenty to play for and Magnus's quality and stamina showed. I'm

by thom 5y ago

Genuinely enjoyable and timely game. Out of prep early, machine evaluations useless throughout, plenty to play for and Magnus's quality and stamina showed. I'm of the opinion that there's nothing to gain from fans having engines open during a game, despite working in sports analytics and ostensibly being on the side of the machines. Obviously engines have a lot to say about the modern game, and the match so far had clearly been dominated by excellent prep by both teams. But the excellent commentary on Levitov Chess World by the peerless Peter Svidler, Vladimir Kramnik and Evgenij Miroshnichenko was firmly of the opinion that we had a game on our hands throughout. They were correct. I looked into some stats from the most recent TCEC superfinal just to get a feel for the error bars in engine evaluations. I think it's clear that even up to about +1.5, a draw is still on the cards between the superhuman engines, let alone humans. About a quarter of the superfinal games were decisive, despite us treating them as oracles. They're far more accurate in their draw predictions - in games that reached a 0.0 evaluation, only 3 ended with a win. But in this game, even when we hit the tablebase results, where the result is a foregone conclusion for the machines, they still don't reflect the outcome of a human game, especially when the time controls start to get more stressful. Anyway, what a wonderful, reassuring game. The players will be exhausted and it sets up a fascinating weekend of play. I couldn't be happier.

alexeiz 5y ago
I also like Svindler's et al. commentary better because they don't resort to computer analysis. They explore ideas on the chessboard from the human point of view and that makes their analysis much better than what you see on chess.com or chess24.
pedrocr 5y ago
>I'm of the opinion that there's nothing to gain from fans having engines open during a game Isn't this game a great argument for engines. Using a threshold of 2 to be well above your 1.5 this game had a total of 4 moments where one of the players had a very good chance for a win, and one of those moments was for Nepo: https://analysis.sesse.net/ https://analysis.sesse.net/ Having the engine show you that is a great complement to hearing the commentary from top super GMs. Why would you throw away that part of the coverage? >But in this game, even when we hit the tablebase results, where the result is a foregone conclusion for the machines, they still don't reflect the outcome of a human game, especially when the time controls start to get more stressful. While true I think the solution is to double down and extract metrics from the engine search that reflect that. Not only the evaluation of the position but also how narrow the path is for white and black, to better understand how even if it's a draw from the machine's point of view, how hard of a draw it will be for the human to hold.
- thom 5y ago
  If I were to make a serious point it would be this: sometimes a machine learning model shouldn’t be optimised for accuracy, it should be optimised for teaching humans an effective policy. Often those are the same goal, for example in opening preparation in chess. But often they’re not, and just having 1000 fans suggesting lines from Stockfish without really thinking about a position or its human contestants isn’t particularly edifying for me. It feels like shopping for great art based on the price tag alone. So yes, I agree with your second point, the engines/metrics/stats could be doing more here. Just relying on the evaluation at ridiculous depths isn’t very interesting, and I don’t believe those scores necessarily translate to a very good chance to win. 33 Rcc2 would have been very hard to calculate and involves a double pawn sacrifice. 36… Bxb4 probably was the move most humans would have made but I’m still not convinced Nepo always wins from there. I’m sure there were other swings I’m forgetting. But outside of “this line exists at depth 30 if you see it” there’s a lot to learn about the components of that evaluation, like mobility and king safety etc that do mirror some of the human thought process and which you can apply yourself. I’d love to see more detailed metrics like that, especially if they could capture why the humans felt it was a very good Catalan for white with lots of options, and not just zeroes. But as of today, I’d rather hear what Kramnik thinks than Stockfish. And the difficulty of holding a position is far more accurate coming from humans than the machines (game 5 for example). Even when the computer thinks there’s only one move in a position, that’s often based on the opponent playing perfectly to a great depth afterwards. I don’t want to sound too much of a curmudgeon here, I like engines, I write my own and of course I have Sesse and my own copy of Leela open most games. But I think too often they short circuit one’s brain, and I’ve found my enjoyment increases when I follow the human thought process and not the machine one. I’d love to see a richer synthesis of the two though!

by thom 5y ago

alexeiz 5y ago

I also like Svindler's et al. commentary better because they don't resort to computer analysis. They explore ideas on the chessboard from the human point of view and that makes their analysis much better than what you see on chess.com or chess24.

pedrocr 5y ago

>I'm of the opinion that there's nothing to gain from fans having engines open during a game Isn't this game a great argument for engines. Using a threshold of 2 to be well above your 1.5 this game had a total of 4 moments where one of the players had a very good chance for a win, and one of those moments was for Nepo: https://analysis.sesse.net/ https://analysis.sesse.net/ Having the engine show you that is a great complement to hearing the commentary from top super GMs. Why would you throw away that part of the coverage? >But in this game, even when we hit the tablebase results, where the result is a foregone conclusion for the machines, they still don't reflect the outcome of a human game, especially when the time controls start to get more stressful. While true I think the solution is to double down and extract metrics from the engine search that reflect that. Not only the evaluation of the position but also how narrow the path is for white and black, to better understand how even if it's a draw from the machine's point of view, how hard of a draw it will be for the human to hold.

thom 5y ago
If I were to make a serious point it would be this: sometimes a machine learning model shouldn’t be optimised for accuracy, it should be optimised for teaching humans an effective policy. Often those are the same goal, for example in opening preparation in chess. But often they’re not, and just having 1000 fans suggesting lines from Stockfish without really thinking about a position or its human contestants isn’t particularly edifying for me. It feels like shopping for great art based on the price tag alone. So yes, I agree with your second point, the engines/metrics/stats could be doing more here. Just relying on the evaluation at ridiculous depths isn’t very interesting, and I don’t believe those scores necessarily translate to a very good chance to win. 33 Rcc2 would have been very hard to calculate and involves a double pawn sacrifice. 36… Bxb4 probably was the move most humans would have made but I’m still not convinced Nepo always wins from there. I’m sure there were other swings I’m forgetting. But outside of “this line exists at depth 30 if you see it” there’s a lot to learn about the components of that evaluation, like mobility and king safety etc that do mirror some of the human thought process and which you can apply yourself. I’d love to see more detailed metrics like that, especially if they could capture why the humans felt it was a very good Catalan for white with lots of options, and not just zeroes. But as of today, I’d rather hear what Kramnik thinks than Stockfish. And the difficulty of holding a position is far more accurate coming from humans than the machines (game 5 for example). Even when the computer thinks there’s only one move in a position, that’s often based on the opponent playing perfectly to a great depth afterwards. I don’t want to sound too much of a curmudgeon here, I like engines, I write my own and of course I have Sesse and my own copy of Leela open most games. But I think too often they short circuit one’s brain, and I’ve found my enjoyment increases when I follow the human thought process and not the machine one. I’d love to see a richer synthesis of the two though!