3 ms·

They also had a headline for Alphazero that convinced everyone that they crushed Stockfish and that classical chess engines were stuff of the past, when in fact

They also had a headline for Alphazero that convinced everyone that they crushed Stockfish and that classical chess engines were stuff of the past, when in fact it was about 50 elo better than the Stockfish version they were testing against, or roughly the same as how much Stockfish improves each year.

thom 2y ago
And then what happened is AlphaZero changed the professional game in various interesting ways, and all its ideas were absorbed into Stockfish. A little bombast is forgivable for technology that goes on to have a big impact, and I don’t doubt it’s the same story here.
- thealig 2y ago
  > all its ideas were absorbed into Stockfish don't think that is true, Stockfish incorporated NNUE techniques through a fork https://www.chess.com/news/view/stockfishnnue-strongest-chess-engine-ever-to-compete-in-cccc https://www.chess.com/news/view/stockfishnnue-strongest-ches... being transparent with the setup of your invention is always a good thing.
- GaggiX 2y ago
  >all its ideas were absorbed into Stockfish That's not true at all, Stockfish still uses only human heuristics for search and NNUE for eval, a completely different architecture than alphazero and derived from the Yu Nasu Shogi engine.
  - thom 2y ago
    It's a neural network trained on self-play games (many of them lifted from Leela Zero). I get that it's a different shape of network, but people really seem touchy about crediting Google with the kick up the bum that led us here. AlphaZero had a massive effect on chess globally, whatever people think about its press releases. My main point is that people should update the heuristic that wastes energy arguing about bold claims when clearly something amazing has happened that everyone in the industry will react to and learn from.
    - nybsjytm 2y ago
      I don't have any particular thoughts about DeepMind's board game algorithms or how they were advertised, but even if I happened to think it was the most innovative and influential research in years, I'd still ask for honest communication about the work. It's part of being a healthy research community - although clearly the AI community falls well short on this, and nobody could say it's only DeepMind's fault.
    - GaggiX 2y ago
      >It's a neural network trained on self-play games It's not, it's just supervised learning on evaluations. There is no self-play involved when training the model.
      - thom 2y ago
        Where do the evaluations come from? The idea that Stockfish isn't benefiting hugely from Google having created and advertised AlphaZero is preposterous, can we please just stop?
        GaggiX 2y ago
        >Where do the evaluations come from? Good datasets are selected empirically, they are usually a mix of different sources, not a single engine. >The idea that Stockfish isn't benefiting hugely from Google having created and advertised AlphaZero is preposterous, can we please just stop? I have not said anything about AlphaZero, I am just reporting where you are wrong. Your arguments are simply not very convincing.
        thom 2y ago
        Okay, well, no sale I guess. Stockfish's training dataset is mostly self-play games from an engine directly inspired by AlphaZero. It moved to neural network evaluation after a fork based on a paper that cites AlphaZero. It plays chess more like AlphaZero than Stockfish 11. Yes, it's extremely interesting that it continues to edge out Leela with a fast, rough approximation of the latter's evaluation but much faster search. But it (and human chess) wouldn't be where it is today without AlphaZero, and I was originally responding to someone dismissing it based on the perceived over-zealousness of its marketing, as people seem to want to do with TFA. I merely submit that both of these Google innovations are exciting and impactful, and we should forgive their presentation, which nevertheless has been kind enough to link to the original papers which have all the information we need to help change the world.
kriro 2y ago
I think Alphazero is a lot more interesting than Stockfish though. Most notably it lead me to reevaluate positional play. Iirc A0 at around 2-3 ply is still above SuperGM Level which is pretty mind-blowing. Based on this I have increased my strategy to tactics ratio quite a bit. FWIW Stockfish is always evolving and adapting and has incorporated ideas from A0.
- RUnconcerned 2y ago
  Stockfish has not incorporated ideas from AlphaZero. Stockfish's NN eval technology, NNUE, comes from Shogi and it predates Alphazero there. The 2nd strongest engine, Leela Chess Zero, is indeed directly inspired by AlphaZero, though, and did surpass Stockfish until NNUE was introduced.
  - abecedarius 2y ago
    Hmm: NNUE was introduced in 2018, the AlphaZero preprint 2017, AlphaGo 2015-2016. I checked this because my memory claimed that it was AlphaGo's success that sparked the new level of interest in NN evaluation. Wouldn't surprise me if AlphaZero's improvements had no influence in that timeline, but for AlphaGo it would.
    - thom 2y ago
      The original NNUE paper cites AlphaZero[0]. The architectures are different because NNUE is optimized for CPUs and uses integer quantization and a much smaller network. I don't think one could credibly claim that it would have come about if not for Google making so much noise about their neural network efforts in Go, Chess and Shogi. 0: https://github.com/asdfjkl/nnue/blob/main/nnue_en.pdf https://github.com/asdfjkl/nnue/blob/main/nnue_en.pdf
  - thom 2y ago
    For whatever it's worth, the NNUE training dataset contains positions from Leela games and several generations of self-play. Stockfish wouldn't be where it is if not for Google's impact. AlphaFold will likely have a similar impact on our understanding of protein structure. I don't know why everyone is so offended by them puffing their chests out a little bit here, the paper's linked in the article.
mda 2y ago
Alphazero indeed crushed stockfish with a novel technique, I think it deserved all the praise.
- hibikir 2y ago
  It definitely deserved a lot of praise, but the testing situation wasn't really against a fully fledged stockfish running on similar hardware, but one that, among other things, had no opening library. The issue is not whether alphazero was impressive, but that we should be careful about the specific claims of the press releases, as they are known to oversell. The whole thing would have been impressive enough if the games had been against the last release of stockfish with good hardware, just for the way it played.

thom 2y ago

And then what happened is AlphaZero changed the professional game in various interesting ways, and all its ideas were absorbed into Stockfish. A little bombast is forgivable for technology that goes on to have a big impact, and I don’t doubt it’s the same story here.

thealig 2y ago
> all its ideas were absorbed into Stockfish don't think that is true, Stockfish incorporated NNUE techniques through a fork https://www.chess.com/news/view/stockfishnnue-strongest-chess-engine-ever-to-compete-in-cccc https://www.chess.com/news/view/stockfishnnue-strongest-ches... being transparent with the setup of your invention is always a good thing.
GaggiX 2y ago
>all its ideas were absorbed into Stockfish That's not true at all, Stockfish still uses only human heuristics for search and NNUE for eval, a completely different architecture than alphazero and derived from the Yu Nasu Shogi engine.
- thom 2y ago
  It's a neural network trained on self-play games (many of them lifted from Leela Zero). I get that it's a different shape of network, but people really seem touchy about crediting Google with the kick up the bum that led us here. AlphaZero had a massive effect on chess globally, whatever people think about its press releases. My main point is that people should update the heuristic that wastes energy arguing about bold claims when clearly something amazing has happened that everyone in the industry will react to and learn from.
  - nybsjytm 2y ago
    I don't have any particular thoughts about DeepMind's board game algorithms or how they were advertised, but even if I happened to think it was the most innovative and influential research in years, I'd still ask for honest communication about the work. It's part of being a healthy research community - although clearly the AI community falls well short on this, and nobody could say it's only DeepMind's fault.
  - GaggiX 2y ago
    >It's a neural network trained on self-play games It's not, it's just supervised learning on evaluations. There is no self-play involved when training the model.
    - thom 2y ago
      Where do the evaluations come from? The idea that Stockfish isn't benefiting hugely from Google having created and advertised AlphaZero is preposterous, can we please just stop?
      - GaggiX 2y ago
        >Where do the evaluations come from? Good datasets are selected empirically, they are usually a mix of different sources, not a single engine. >The idea that Stockfish isn't benefiting hugely from Google having created and advertised AlphaZero is preposterous, can we please just stop? I have not said anything about AlphaZero, I am just reporting where you are wrong. Your arguments are simply not very convincing.
        thom 2y ago
        Okay, well, no sale I guess. Stockfish's training dataset is mostly self-play games from an engine directly inspired by AlphaZero. It moved to neural network evaluation after a fork based on a paper that cites AlphaZero. It plays chess more like AlphaZero than Stockfish 11. Yes, it's extremely interesting that it continues to edge out Leela with a fast, rough approximation of the latter's evaluation but much faster search. But it (and human chess) wouldn't be where it is today without AlphaZero, and I was originally responding to someone dismissing it based on the perceived over-zealousness of its marketing, as people seem to want to do with TFA. I merely submit that both of these Google innovations are exciting and impactful, and we should forgive their presentation, which nevertheless has been kind enough to link to the original papers which have all the information we need to help change the world.

kriro 2y ago

I think Alphazero is a lot more interesting than Stockfish though. Most notably it lead me to reevaluate positional play. Iirc A0 at around 2-3 ply is still above SuperGM Level which is pretty mind-blowing. Based on this I have increased my strategy to tactics ratio quite a bit. FWIW Stockfish is always evolving and adapting and has incorporated ideas from A0.

RUnconcerned 2y ago
Stockfish has not incorporated ideas from AlphaZero. Stockfish's NN eval technology, NNUE, comes from Shogi and it predates Alphazero there. The 2nd strongest engine, Leela Chess Zero, is indeed directly inspired by AlphaZero, though, and did surpass Stockfish until NNUE was introduced.
- abecedarius 2y ago
  Hmm: NNUE was introduced in 2018, the AlphaZero preprint 2017, AlphaGo 2015-2016. I checked this because my memory claimed that it was AlphaGo's success that sparked the new level of interest in NN evaluation. Wouldn't surprise me if AlphaZero's improvements had no influence in that timeline, but for AlphaGo it would.
  - thom 2y ago
    The original NNUE paper cites AlphaZero[0]. The architectures are different because NNUE is optimized for CPUs and uses integer quantization and a much smaller network. I don't think one could credibly claim that it would have come about if not for Google making so much noise about their neural network efforts in Go, Chess and Shogi. 0: https://github.com/asdfjkl/nnue/blob/main/nnue_en.pdf https://github.com/asdfjkl/nnue/blob/main/nnue_en.pdf
- thom 2y ago
  For whatever it's worth, the NNUE training dataset contains positions from Leela games and several generations of self-play. Stockfish wouldn't be where it is if not for Google's impact. AlphaFold will likely have a similar impact on our understanding of protein structure. I don't know why everyone is so offended by them puffing their chests out a little bit here, the paper's linked in the article.

mda 2y ago

Alphazero indeed crushed stockfish with a novel technique, I think it deserved all the praise.

hibikir 2y ago
It definitely deserved a lot of praise, but the testing situation wasn't really against a fully fledged stockfish running on similar hardware, but one that, among other things, had no opening library. The issue is not whether alphazero was impressive, but that we should be careful about the specific claims of the press releases, as they are known to oversell. The whole thing would have been impressive enough if the games had been against the last release of stockfish with good hardware, just for the way it played.