3 ms·
Given their track history, I don't think it's likely the DeepMind team are trying to be sneaky here. They'd be found out eventually given how big their claim is
by seanwilson 9y ago
Given their track history, I don't think it's likely the DeepMind team are trying to be sneaky here. They'd be found out eventually given how big their claim is.
When working in academia, I found it very common for research papers to not come with source code or enough information to allow you to replicate experiments yourself. You usually have to pester the author. I don't find the (valid) criticisms here that unusual. I'm not sure why they wouldn't release the moves for all the test games played though seeing is that should be simple to do.
As for Stockfish and AlphaZero running on different hardware...AlphaZero's approach is built around taking full advantage of what TPUs can do quickly and Stockfish doesn't utilise TPUs so how are you meant to make this fair? Does Stockfish eventually level out when you throw enough hardware at it? Doesn't DeepMind's claim about AlphaZero evaluating significantly less moves per turn invalidate the criticism about the hardware used?
- deleted 9y ago[deleted]
- pdpi 9y agoMore hardware still means better results — or there would be no need to cap the time per move. It’s always been a trade off between using more expensive heuristics on fewer moves or cheaper heuristics on more moves. The “number of moves” thing only says that they went all-in on the better heuristics angle. In fact, there was something posted recently about test games where they played go by just using the first move suggested by the search heuristic and that was pretty strong.
- seanwilson 9y ago> More hardware still means better results — or there would be no need to cap the time per move. OK, but AlphaZero essentially runs on different hardware so how are you suppose to make the comparison fair? You could give Stockfish access to the TPUs but wouldn't do anything with them.
- forgot-my-pw 9y agoThey hinted that a full paper will be released in the future.
- vatueil 9y agoYes. And not just a hint; Demis Hassabis (DeepMind founder) explicitly stated "full paper coming soon, will have things like early games": https://twitter.com/demishassabis/status/938347604462542849 https://twitter.com/demishassabis/status/938347604462542849 I'm sure they're saving some stuff for the full paper. Don't know how many games that will include, but hopefully it'll have some shogi game records too.