6 ms·

The match between Stockfish and AlphaZero was played with certain unjustified parameters (time control, ponder off, different hardware, no opening book or endga

The match between Stockfish and AlphaZero was played with certain unjustified parameters (time control, ponder off, different hardware, no opening book or endgame tablebase for Stockfish etc.). By "unjustified," I mean that the authors of the paper did not justify their choice of parameters in the paper as being designed to implement a fair match. At a glance, the parameters of the match seem unfair to me -- and tilted heavily towards AlphaZero. If the code, were open source, this would not matter; anyone could run a rematch. As it is, I haven't seen any convincing evidence that AlphaZero is stronger than Stockfish when Stockfish is allowed to use its full breadth of knowledge and run on equal hardware.

noir_lord 8y ago
That was the first paper. Second paper they played fair even giving stockfish a 10 to 1 advantage on same hardware as tcec uses.
- vkerner 8y ago
  Paper is patient. We won't know until AlphaZero participates in a fair tournament, like Leela, which recently came in third after Stockfish.
- thom 8y ago
  You can still run the games past the exact commit of Stockfish they used and it finds blunders in its own play, so it still feels like there's a lack of transparency. But I don't think anyone strongly believes AlphaZero isn't the best at this point.
  - mda 8y ago
    Like which move? I wonder why stockfish developers do not claim any of this?
    - ag1 8y ago
      glinscott said "We need a public exhibition match to settle the score, ideally with some GM commentary." above. As for the GM side, Nakamura basically said the same.
      - mda 8y ago
        Still, zero proof of foul play. But sure i would very much enjoy watching Alphazero destroy Stockfish in a public match.
        thom 8y ago
        Yeah, I'm certainly not claiming AlphaZero isn't the stronger engine (probably by quite a substantial margin). The first match _was_ a pretty clear fix, but the second paper is an entirely reasonable setup, so I have no real suspicions about the process on an ongoing basis, just a shame nobody can prove it for themselves.
    - thom 8y ago
      You can checkout the exact commit of Stockfish from the paper and perform the analysis of the published games yourself. I doubt the Stockfish developers have bothered because it's an old version. Can I just state again, because of your aggressive tone in multiple comments now, that I do believe AlphaZero is stronger, I don't believe there are real shenanigans going on, but it's _still_ sad that we can't reliably, publicly verify this stuff.
      - thom 8y ago
        (And I know in the past, before reading the paper, I've assumed they were cheating this time because they made so little effort the first time around, but I'm happy to admit I was wrong).
      - mda 8y ago
        You are the one making this claim: "You can still run the games past the exact commit of Stockfish they used and it finds blunders in its own play, so it still feels like there's a lack of transparency." This claim implies foul play, I asked for a shred of evidence. Call me aggresive if you want, I just can't stand this kind of bullshit.
yesenadam 8y ago
There has been a rematch recently vs Stockfish, with a couple of hundred games. AlphaZero won 155-6! [0] There are fascinating videos with grandmasters commentating on some of the games. They're played in an exciting, sacrificial, swashbuckling style, nothing like any other top computer engine, and it seems that may affect the play of top (human) players for the better. e.g. see Matthew Sadler on chess24 https://www.youtube.com/watch?v=JacRX6cKIaY&list=PLAwlxGCJB4NchyTBYik8FBbnzLXpCCO79&index=6 https://www.youtube.com/watch?v=JacRX6cKIaY&list=PLAwlxGCJB4... Daniel King https://www.youtube.com/watch?v=pFtY7gNRVRI&index=3&list=PLhyM8toCZs_qWKCohlxdcRUo3O9h7g61w https://www.youtube.com/watch?v=pFtY7gNRVRI&index=3&list=PLh... [0] https://en.wikipedia.org/wiki/AlphaZero#Chess_2 https://en.wikipedia.org/wiki/AlphaZero#Chess_2
- fairplay2 8y ago
  The Wikipedia link says this was against Stockfish 8. Could people please stop spreading FUD here? If there's no public tournament, this might as well not have happened. I do not understand why Google is always special. Other engines are open, Google can test against Stockfish but not vice versa. All these web companies take, take, take from Open Source and rarely give back. I'll write a paper now that I beat Carlsen, but I'll refuse to do so in public.
  - yesenadam 8y ago
    >Could people please stop spreading FUD here? Do you mean me? If so, why not say so. "Could people please stop X-ing here?" is very passive-aggressive. I looked up 'FUD' - Fear, uncertainty and doubt. Not sure how what I said counts as any of those. Your comment certainly seems to want to spread FUD, however. (And why is this your account's only comment on HN?) I don't know the significance of your first sentence either. I know nothing about the versions of Stockfish used for this or anything else. Maybe you're taking too much for granted. i.e. that I know enough of the minutiae to understand your comments. Could you fill in the dots a bit? And who are 'all these web companies'? I'm not super-interested in AlphaZero (or computer chess generally) - haven't read any of the papers, for example. But there's a lot of talk about the various Alphas in the online chess world, since before it was playing chess, and I found these videos very impressive. And it's ridiculous to say "This might as well not have happened". Maybe true for you, but not for the chess world, at all.

noir_lord 8y ago

That was the first paper. Second paper they played fair even giving stockfish a 10 to 1 advantage on same hardware as tcec uses.

vkerner 8y ago
Paper is patient. We won't know until AlphaZero participates in a fair tournament, like Leela, which recently came in third after Stockfish.
thom 8y ago
You can still run the games past the exact commit of Stockfish they used and it finds blunders in its own play, so it still feels like there's a lack of transparency. But I don't think anyone strongly believes AlphaZero isn't the best at this point.
- mda 8y ago
  Like which move? I wonder why stockfish developers do not claim any of this?
  - ag1 8y ago
    glinscott said "We need a public exhibition match to settle the score, ideally with some GM commentary." above. As for the GM side, Nakamura basically said the same.
    - mda 8y ago
      Still, zero proof of foul play. But sure i would very much enjoy watching Alphazero destroy Stockfish in a public match.
      - thom 8y ago
        Yeah, I'm certainly not claiming AlphaZero isn't the stronger engine (probably by quite a substantial margin). The first match _was_ a pretty clear fix, but the second paper is an entirely reasonable setup, so I have no real suspicions about the process on an ongoing basis, just a shame nobody can prove it for themselves.
  - thom 8y ago
    You can checkout the exact commit of Stockfish from the paper and perform the analysis of the published games yourself. I doubt the Stockfish developers have bothered because it's an old version. Can I just state again, because of your aggressive tone in multiple comments now, that I do believe AlphaZero is stronger, I don't believe there are real shenanigans going on, but it's _still_ sad that we can't reliably, publicly verify this stuff.
    - thom 8y ago
      (And I know in the past, before reading the paper, I've assumed they were cheating this time because they made so little effort the first time around, but I'm happy to admit I was wrong).
    - mda 8y ago
      You are the one making this claim: "You can still run the games past the exact commit of Stockfish they used and it finds blunders in its own play, so it still feels like there's a lack of transparency." This claim implies foul play, I asked for a shred of evidence. Call me aggresive if you want, I just can't stand this kind of bullshit.

yesenadam 8y ago

There has been a rematch recently vs Stockfish, with a couple of hundred games. AlphaZero won 155-6! [0] There are fascinating videos with grandmasters commentating on some of the games. They're played in an exciting, sacrificial, swashbuckling style, nothing like any other top computer engine, and it seems that may affect the play of top (human) players for the better. e.g. see Matthew Sadler on chess24 https://www.youtube.com/watch?v=JacRX6cKIaY&list=PLAwlxGCJB4NchyTBYik8FBbnzLXpCCO79&index=6 https://www.youtube.com/watch?v=JacRX6cKIaY&list=PLAwlxGCJB4... Daniel King https://www.youtube.com/watch?v=pFtY7gNRVRI&index=3&list=PLhyM8toCZs_qWKCohlxdcRUo3O9h7g61w https://www.youtube.com/watch?v=pFtY7gNRVRI&index=3&list=PLh... [0] https://en.wikipedia.org/wiki/AlphaZero#Chess_2 https://en.wikipedia.org/wiki/AlphaZero#Chess_2

fairplay2 8y ago
The Wikipedia link says this was against Stockfish 8. Could people please stop spreading FUD here? If there's no public tournament, this might as well not have happened. I do not understand why Google is always special. Other engines are open, Google can test against Stockfish but not vice versa. All these web companies take, take, take from Open Source and rarely give back. I'll write a paper now that I beat Carlsen, but I'll refuse to do so in public.
- yesenadam 8y ago
  >Could people please stop spreading FUD here? Do you mean me? If so, why not say so. "Could people please stop X-ing here?" is very passive-aggressive. I looked up 'FUD' - Fear, uncertainty and doubt. Not sure how what I said counts as any of those. Your comment certainly seems to want to spread FUD, however. (And why is this your account's only comment on HN?) I don't know the significance of your first sentence either. I know nothing about the versions of Stockfish used for this or anything else. Maybe you're taking too much for granted. i.e. that I know enough of the minutiae to understand your comments. Could you fill in the dots a bit? And who are 'all these web companies'? I'm not super-interested in AlphaZero (or computer chess generally) - haven't read any of the papers, for example. But there's a lot of talk about the various Alphas in the online chess world, since before it was playing chess, and I found these videos very impressive. And it's ridiculous to say "This might as well not have happened". Maybe true for you, but not for the chess world, at all.