8 ms·
Deepmind Produces a General-Purpose Games-Playing Machine
- mindgam3 8y agoProps to DeepMind for an impressive result. And for fixing the unfair conditions from the previous match where they “crushed” a Stockfish that was severely handicapped due to time control and no opening books. Note that Stockfish score was significantly improved this time around. That said, can someone please explain what the logic is for DeepMind only releasing 20 games out of a 1000 game match? That is a relatively outstanding degree of cherry picking results. As a former chess prodigy and software engineer, I want to believe these results but they would be much more credible if DeepMind open sourced the games. I don’t see how that would expose any valuable proprietary IP, but I’m a total machine learning noob so maybe I’m missing something.
- CJefferson 8y agoI agree. I'm annoyed how companies like Google get to release selected parts of their work, keep the rest secret, yet get accepted by top Journals. I am positive (and have seen reviews saying such) that academics would never get away with this kind of filtering of results.
- deleted 8y ago[deleted]
- ACow_Adonis 8y agoI hate to break it to you, but academics certainly do get away with this practice. You simply don't have to report the 1000 results or hypothese that didn't generate positive findings: also see publication bias. Indeed not only is it present, but due to the various ways academia ranks citations and papers, it's fairer to say it's actually incentivised by academic culture.
- skybrian 8y agoI agree they should do it, but it looks like other researchers do sometimes get away without publishing their data? "Data sharing is a common requirement of funding or publication, though this obligation may come as a surprise to some authors—and to their colleagues who have had trouble acquiring data from other laboratories." https://www.nature.com/articles/nn0807-931 https://www.nature.com/articles/nn0807-931
- newen 8y agoAcademics regularly get away with just posting results as graphs, without even the actual numbers in the graphs given out. And certainly not data, code, etc. I've done that for multiple papers in top journals in my field, and lots of others do the same. I give out the data and sometimes the code when people email me, but it's never required during the peer review process. Reviewers are just expected to believe the results in the submitted paper. I always find it funny when people hold so much faith in the peer review process, like the reviewers have some special knowledge or insight that let's them legitimize a paper. Majority of the time, the peer reviewers read the exact same thing everyone else reads, then provide suggestions for change, etc and eventually accept the paper for publication. But in terms of actual verification that the authors of the paper are not fabricating results, there is nothing remotely close to that.
- dmurray 8y agoThere are at least 230 games released according to https://deepmind.com/research/alphago/alphazero-resources/ https://deepmind.com/research/alphago/alphazero-resources/ Agreed that it would be nice to get all 1000 (or maybe there are more? Since these seem to have come from two matches played under different conditions, and they mention pitting AZ against Stockfish in many other sets of conditions) and it really shouldn't compromise DeepMind's IP.
- WestCoastJustin 8y agoMaybe you could partially reverse the model if you had this data + timeline? I'm not sure though. Probably just didn't think anyone would want to dig through a 1000+ games.
- mindgam3 8y ago"Maybe you could partially reverse the model" Even with my rudimentary understanding of machine learning, I'm fairly certain that reverse engineering a model presumably based on millions of training games would be utterly impossible just by looking at a few thousand games. "Probably just didn't think anyone would want to dig through a 1000+ games." Actually, I think a lot of people would be interested: - chess players like myself - skeptics, academic researchers, and other "data scientists" looking for actual data to back up these impressive claims
- usgroup 8y agoStill think there’s a total hardware mismatch here. AlphaGo got 2 TPUs and 44 cores vs Stockfish on just 44 cores. AlphaGo is undoubtedly impressive but would love to see an FPGA version of Stockfish vs AlphaGo.
- aerovistae 8y agoAnyone else see the first sentence of this, think about passing props to Deepmind, and reflect that they've spent too much time writing JS lately?
- Twirrim 8y agoNot sure if this is in your wheel-house, but there's an open-source community driven re-implementation of AlphaZero, targeting Chess: https://github.com/LeelaChessZero/ https://github.com/LeelaChessZero/, http://lczero.org/ http://lczero.org/ It's reaching quite a competitive state, having trained to an ELO of ~3500 or so (based on evaluations against stockfish et al)
- mindgam3 8y agoThanks, I’ve heard about LeelaChess a bit but haven’t gotten a chance to check it out in depth. It sounds like a really interesting project. I’m glad somebody is doing some open source work in this area, just wish Google would join the party a bit more in this area.
- skybrian 8y agoApparently the news is that they published a new paper about AlphaZero?
- buboard 8y agoyes. reviews take a lot of time
- deleted 8y ago[deleted]
- backpropaganda 8y agoComparing AlphaZero to AlphaGo Lee seems problematic. I don't think transitivity holds in this case, i.e. AlphaZero can beat AlphaGo Lee almost surely, and AlphaGo Lee can beat Lee Sedol almost surely, but it could be possible that AlphaZero is not able to beat Lee Sedol at all. This is because the state spaces reached in computer-computer games are probably very different from the state space reached in human-computer games. I could be wrong, but at the very least this should be discussed by Deepmind.
- Wowfunhappy 8y ago> This is because the state spaces reached in computer-computer games are probably very different from the state space reached in human-computer games. Why would this be true? If we were talking about ai-generated music that would be one thing, but it isn't intuitively obvious to me why a computer would play Chess or Go all that differently from a human.
- backpropaganda 8y agoWe do know that AlphaZero's chess playing style is different. AlphaZero doesn't care as much about material, while Stockfish and humans (presumably learning from Stockfish) do. This is because Stockfish uses a material heuristic to search for moves. Consider rock-paper-scissor, and agent X which plays X every move. Agent rock beats agent scissor all the time, and agent scissor beats agent paper all the time. But we cannot conclude that agent rock will beat agent paper all the time. Chess and Go might have this intransitivity property. The fact that one game exists which is not transitive means the burden of proof that Chess and Go are transitive rests on the authors. My own suspicion is that AlphaZero should be weaker against humans compared to AlphaGo Lee since the former is not trained on human games while the latter is.
- gliptic 8y agoBut that isn't true. AlphaGo Zero seemed to be considerably stronger against humans than AlphaGo Lee. Lee managed to win one game against AlphaGo Lee, but AlphaGo Zero won 60 games in a row against top players, as well as all games against Ke Jie.
- j_m_b 8y agoNow just do Magic: The Gathering!
- ArtWomb 8y agoFeels like an historical moment. DeepMimic is a great companion to this: simulated robots using self-play to "discover" the art of Kung Fu ;) https://www.youtube.com/watch?v=vppFvq2quQ0 https://www.youtube.com/watch?v=vppFvq2quQ0 The quote from Lee about creativity is haunting. If the current debate centers on using human reinforcement for robot meta-learning. Particularly using one-shot learning from a single video demonstration. It's possible the main competitor to emerge. Given advances in near future TPU Cloud performance and AI accelerators. An emergence of end-to-end transfer learning directly from simulation. With the unexpected result of novel strategies beyond the capacity of expert level humans to devise. Ilya Sutskever at AI Frontiers 2018: Recent Advances in Deep Learning and AI from OpenAI https://www.youtube.com/watch?v=ElyFDUab30A https://www.youtube.com/watch?v=ElyFDUab30A
- dmoy 8y agoSo what's up with throwing boxes at everything all the time? Is that just an easy way to put varying amounts of external complications in to see how it handles?
- machiaweliczny 8y agoChecking robustness to noise I think. These algorithms are meant for stochastic, not entirely predictable environments (so your action might not always work and you should be able to carry on task even after some action failure (eg. in case of walking - you could slip, wind blows harder or something hits you etc.))
- budadre75 8y agoDoes anyone know what's the progress on explaining AlphaGo/Zero's moves?
- andreyk 8y agoGeneral purpose only if you have access to a simulator and can do MCTS with it :) Just fyi, there are many caveats for how this relates to progress in AI as a whole (self promotion warning): https://www.skynettoday.com/editorials/is-alphago-zero-overrated https://www.skynettoday.com/editorials/is-alphago-zero-overr... (yes the article is for AlphaGo Zero, but it largely applies to AlphaZero as well).
- quotemstr 8y agoWhy is this surprising? AlphaGo Zero mastered Go without any prior idea of what Go is, so why wouldn't the same kind of system be able to learn and play other sorts of games? As I've been saying for years now: if you can model a problem as a complete-information adversarial game, that problem is solved.
- dagw 8y agothat problem is solved. While what they've achieved is super impressive, it has nothing to do with solving chess in a formal sense.
- roymurdock 8y agoNow that perfect information games have been "solved" it will be interesting to see how these teams move up to exponentially harder (IMO) problems where limited information gives the cognitive abilities of humans a massive edge over the brute force skill of computers. Look at how Blizzard did computer opponent AI for games like Starcraft and Warcraft in the early 2000s. The "insane" AI builds its base and units well, controls its troops quickly, and can micro manage different types of units. But its real strength is that it gets 2x the normal amount of resources that a human player gets, allowing it to build structures sooner than human players and build 2x the units. This is the necessary tradeoff to make up for the fact that a human can recognize and exploit computer weaknesses, and the computer generally can't infer what its opponent is doing through the subtle hints that humans pick up on. For example if you catch even a slight glimpse of a certain type of unit an enemy has, or see that an enemy has moved into a certain part of the map early on, you can guess with a reasonable confidence level what build or strategy they are going for without seeing the full playing board. “Those multiplayer games are harder than Go, but not that much higher,” Campbell tells IEEE Spectrum. “A group has already beaten the best players at Dota 2, though it was a restricted version of the game; Starcraft may be a little harder. I think both games are within 2 to 3 years of solution.” 2-3 years would be amazing to solve these problems IMO for non-perfect info games.
- spdionis 8y agoDota 2 is probably harder to beat than starcraft 2, and no, the AI didn't actually manage to beat a pro team, it failed.
- ilaksh 8y agoI wonder when they will start attempting educational games that teach math and language. Those types of skills could lead to general purpose AIs.
- SilverSlash 8y ago"And Stockfish, in turn, is a piker next to AlphaZero, which crushed it after a mere 24 hours of self-training." Isn't this controversial?
- xcvl 8y agoStockfish 8. The paper is completely out of date. Play against version 10 under fair conditions.