5 ms·

I would like to see a rematch with Stockfish configured correctly. I give Stockfish at least 1 GB of hash per thread. In AlphaZero match they had 64 threads and

I would like to see a rematch with Stockfish configured correctly. I give Stockfish at least 1 GB of hash per thread. In AlphaZero match they had 64 threads and only 1 GB of hash for all of them. No one knows how Stockfish behaves with that many search threads, since no one tested it. I don't know if there is any data on how Stockfish scales with number of CPUs but I seem to remember that being one of the weaknesses of the engine, and that commercial engines like Komodo scaled better with larger number of CPUs. Anecdotally on 2.8 GHz 8 core 64bit CPU with 8 threads and 16 GB hash size it calculates about 7-8 million ply per second at the beginning of the game, and much more later when there are less peaces on the board. AlphaZero's setup, with 64 threads on 32 physical CPU cores, calculated about 70 million ply per second, with tiny 1 GB hash (i.e. the engine could remember less of what it calculated previously). But my Stockfish on computationally weaker setup clearly flags some of the moves in the match as mistakes on the 64 threaded Stockfish. I would really like to understand why. Is it because if you have more resources to see deeper you see how hopeless the situation is, or is it something else? I am sure we will see more of these matches. It would be nice if Google volunteered some computing resources and entered TCEC regularly.

roganp 9y ago
It seems to me that that someone can prove that the configuration mattered by pitting a correctly configured Stockfish against the configuration used by AlphaZero, and see what the outcome is for 100 games. I haven't read the paper, but some of the constraint choices seem odd to me (1 minute max per move?).
- super_mario 9y ago
  Unfortunately, AlphaZero is private and no one has access to either the code (as commercial product) or resources it needs to run (TPUs). So, only Google can do this experiment.
  - thisacctforreal 9y ago
    OP is suggesting properly-configured-stockfish versus how-alphazero-configured-stockfish for 100 games.
- forgot-my-pw 9y ago
  The paper suggest that the more time you give Alphazero, the stronger it becomes too.
sireat 9y ago
I too was dissapointed at the 1GB of hash given to Stockfish. Turns out that there is a good reason for 1GB of hash : it is an easy way to get a high number nodes searches per second (ie high kn/sec) on many threads. http://support.stockfishchess.org/discussions/questions/655-which-settings-are-best http://support.stockfishchess.org/discussions/questions/655-... Stockfish is a major PIA to configure to use correctly and I do not blame AlphaZero for taking the easy way out.. I turned on 30 threads for Stockfish 8 on my 2x Xeon 2670 with 64GBs of RAM and tried to adjust hash memory. When going past 1GB hash my kn/sec dropped from 25kn/sec to less than 10kn/sec It might have to do with how 2x CPU configurations work, locality, who knows. A0 should have let Stockfish use its opening book though.
- vatueil 9y ago
  Interesting information on reasons for choosing a 1GB hash size, thanks. > A0 should have let Stockfish use its opening book though. Has it been confirmed that it didn't? As far as I can tell, DeepMind never said that Stockfish didn't use an opening book or endgame tablebase, only that AlphaZero didn't: https://twitter.com/demishassabis/status/938347604462542849 https://twitter.com/demishassabis/status/938347604462542849 The only time opening books are mentioned in the arxiv paper is when they are described as standard components of chess engines; while it doesn't explicitly state whether Stockfish was allowed to use them in the tests, I would think it likely: https://arxiv.org/abs/1712.01815 https://arxiv.org/abs/1712.01815 The rumor that Stockfish didn't have an opening book seems widespread but unsourced.
forgot-my-pw 9y ago
Don't think Google/Deepmind is interested in TCEC at all. They got bigger and better things to do with the research. Also, current TCEC hardware does not support Alphazero. Eventually people will be able to replicate the results and we'll have a few different versions of Alphazero successors. Also read this comment from Stockfish author: https://www.reddit.com/r/chess/comments/7igro1/alphazero_reactions_from_top_gms_stockfish_author/dqyt0fr/?context=10000 https://www.reddit.com/r/chess/comments/7igro1/alphazero_rea...

roganp 9y ago

It seems to me that that someone can prove that the configuration mattered by pitting a correctly configured Stockfish against the configuration used by AlphaZero, and see what the outcome is for 100 games. I haven't read the paper, but some of the constraint choices seem odd to me (1 minute max per move?).

super_mario 9y ago
Unfortunately, AlphaZero is private and no one has access to either the code (as commercial product) or resources it needs to run (TPUs). So, only Google can do this experiment.
- thisacctforreal 9y ago
  OP is suggesting properly-configured-stockfish versus how-alphazero-configured-stockfish for 100 games.
forgot-my-pw 9y ago
The paper suggest that the more time you give Alphazero, the stronger it becomes too.

sireat 9y ago

I too was dissapointed at the 1GB of hash given to Stockfish. Turns out that there is a good reason for 1GB of hash : it is an easy way to get a high number nodes searches per second (ie high kn/sec) on many threads. http://support.stockfishchess.org/discussions/questions/655-which-settings-are-best http://support.stockfishchess.org/discussions/questions/655-... Stockfish is a major PIA to configure to use correctly and I do not blame AlphaZero for taking the easy way out.. I turned on 30 threads for Stockfish 8 on my 2x Xeon 2670 with 64GBs of RAM and tried to adjust hash memory. When going past 1GB hash my kn/sec dropped from 25kn/sec to less than 10kn/sec It might have to do with how 2x CPU configurations work, locality, who knows. A0 should have let Stockfish use its opening book though.

vatueil 9y ago
Interesting information on reasons for choosing a 1GB hash size, thanks. > A0 should have let Stockfish use its opening book though. Has it been confirmed that it didn't? As far as I can tell, DeepMind never said that Stockfish didn't use an opening book or endgame tablebase, only that AlphaZero didn't: https://twitter.com/demishassabis/status/938347604462542849 https://twitter.com/demishassabis/status/938347604462542849 The only time opening books are mentioned in the arxiv paper is when they are described as standard components of chess engines; while it doesn't explicitly state whether Stockfish was allowed to use them in the tests, I would think it likely: https://arxiv.org/abs/1712.01815 https://arxiv.org/abs/1712.01815 The rumor that Stockfish didn't have an opening book seems widespread but unsourced.

forgot-my-pw 9y ago

Don't think Google/Deepmind is interested in TCEC at all. They got bigger and better things to do with the research. Also, current TCEC hardware does not support Alphazero. Eventually people will be able to replicate the results and we'll have a few different versions of Alphazero successors. Also read this comment from Stockfish author: https://www.reddit.com/r/chess/comments/7igro1/alphazero_reactions_from_top_gms_stockfish_author/dqyt0fr/?context=10000 https://www.reddit.com/r/chess/comments/7igro1/alphazero_rea...