3 ms·

>I'm of the opinion that there's nothing to gain from fans having engines open during a game Isn't this game a great argument for engines. Using a threshold of

>I'm of the opinion that there's nothing to gain from fans having engines open during a game Isn't this game a great argument for engines. Using a threshold of 2 to be well above your 1.5 this game had a total of 4 moments where one of the players had a very good chance for a win, and one of those moments was for Nepo: https://analysis.sesse.net/ https://analysis.sesse.net/ Having the engine show you that is a great complement to hearing the commentary from top super GMs. Why would you throw away that part of the coverage? >But in this game, even when we hit the tablebase results, where the result is a foregone conclusion for the machines, they still don't reflect the outcome of a human game, especially when the time controls start to get more stressful. While true I think the solution is to double down and extract metrics from the engine search that reflect that. Not only the evaluation of the position but also how narrow the path is for white and black, to better understand how even if it's a draw from the machine's point of view, how hard of a draw it will be for the human to hold.

thom 5y ago
If I were to make a serious point it would be this: sometimes a machine learning model shouldn’t be optimised for accuracy, it should be optimised for teaching humans an effective policy. Often those are the same goal, for example in opening preparation in chess. But often they’re not, and just having 1000 fans suggesting lines from Stockfish without really thinking about a position or its human contestants isn’t particularly edifying for me. It feels like shopping for great art based on the price tag alone. So yes, I agree with your second point, the engines/metrics/stats could be doing more here. Just relying on the evaluation at ridiculous depths isn’t very interesting, and I don’t believe those scores necessarily translate to a very good chance to win. 33 Rcc2 would have been very hard to calculate and involves a double pawn sacrifice. 36… Bxb4 probably was the move most humans would have made but I’m still not convinced Nepo always wins from there. I’m sure there were other swings I’m forgetting. But outside of “this line exists at depth 30 if you see it” there’s a lot to learn about the components of that evaluation, like mobility and king safety etc that do mirror some of the human thought process and which you can apply yourself. I’d love to see more detailed metrics like that, especially if they could capture why the humans felt it was a very good Catalan for white with lots of options, and not just zeroes. But as of today, I’d rather hear what Kramnik thinks than Stockfish. And the difficulty of holding a position is far more accurate coming from humans than the machines (game 5 for example). Even when the computer thinks there’s only one move in a position, that’s often based on the opponent playing perfectly to a great depth afterwards. I don’t want to sound too much of a curmudgeon here, I like engines, I write my own and of course I have Sesse and my own copy of Leela open most games. But I think too often they short circuit one’s brain, and I’ve found my enjoyment increases when I follow the human thought process and not the machine one. I’d love to see a richer synthesis of the two though!

thom 5y ago

If I were to make a serious point it would be this: sometimes a machine learning model shouldn’t be optimised for accuracy, it should be optimised for teaching humans an effective policy. Often those are the same goal, for example in opening preparation in chess. But often they’re not, and just having 1000 fans suggesting lines from Stockfish without really thinking about a position or its human contestants isn’t particularly edifying for me. It feels like shopping for great art based on the price tag alone. So yes, I agree with your second point, the engines/metrics/stats could be doing more here. Just relying on the evaluation at ridiculous depths isn’t very interesting, and I don’t believe those scores necessarily translate to a very good chance to win. 33 Rcc2 would have been very hard to calculate and involves a double pawn sacrifice. 36… Bxb4 probably was the move most humans would have made but I’m still not convinced Nepo always wins from there. I’m sure there were other swings I’m forgetting. But outside of “this line exists at depth 30 if you see it” there’s a lot to learn about the components of that evaluation, like mobility and king safety etc that do mirror some of the human thought process and which you can apply yourself. I’d love to see more detailed metrics like that, especially if they could capture why the humans felt it was a very good Catalan for white with lots of options, and not just zeroes. But as of today, I’d rather hear what Kramnik thinks than Stockfish. And the difficulty of holding a position is far more accurate coming from humans than the machines (game 5 for example). Even when the computer thinks there’s only one move in a position, that’s often based on the opponent playing perfectly to a great depth afterwards. I don’t want to sound too much of a curmudgeon here, I like engines, I write my own and of course I have Sesse and my own copy of Leela open most games. But I think too often they short circuit one’s brain, and I’ve found my enjoyment increases when I follow the human thought process and not the machine one. I’d love to see a richer synthesis of the two though!