3 ms·
If I were to make a serious point it would be this: sometimes a machine learning model shouldn’t be optimised for accuracy, it should be optimised for teaching
by thom 5y ago
If I were to make a serious point it would be this: sometimes a machine learning model shouldn’t be optimised for accuracy, it should be optimised for teaching humans an effective policy. Often those are the same goal, for example in opening preparation in chess. But often they’re not, and just having 1000 fans suggesting lines from Stockfish without really thinking about a position or its human contestants isn’t particularly edifying for me. It feels like shopping for great art based on the price tag alone.
So yes, I agree with your second point, the engines/metrics/stats could be doing more here. Just relying on the evaluation at ridiculous depths isn’t very interesting, and I don’t believe those scores necessarily translate to a very good chance to win. 33 Rcc2 would have been very hard to calculate and involves a double pawn sacrifice. 36… Bxb4 probably was the move most humans would have made but I’m still not convinced Nepo always wins from there. I’m sure there were other swings I’m forgetting. But outside of “this line exists at depth 30 if you see it” there’s a lot to learn about the components of that evaluation, like mobility and king safety etc that do mirror some of the human thought process and which you can apply yourself. I’d love to see more detailed metrics like that, especially if they could capture why the humans felt it was a very good Catalan for white with lots of options, and not just zeroes. But as of today, I’d rather hear what Kramnik thinks than Stockfish. And the difficulty of holding a position is far more accurate coming from humans than the machines (game 5 for example). Even when the computer thinks there’s only one move in a position, that’s often based on the opponent playing perfectly to a great depth afterwards.
I don’t want to sound too much of a curmudgeon here, I like engines, I write my own and of course I have Sesse and my own copy of Leela open most games. But I think too often they short circuit one’s brain, and I’ve found my enjoyment increases when I follow the human thought process and not the machine one. I’d love to see a richer synthesis of the two though!