9 ms·
DeepMind has open-sourced the heart of AlphaGo and AlphaZero
- mtlmtlmtlmtl 4y agoObligatory note that AlphaZero has long since been surpassed by its independently developed cousin Leela Chess Zero(which is also open source btw), and also Stockfish has more than caught up and remains competitive with MCTS engines.
- bsaul 4y agoDid stockfish use any tech from alphazero & co ? or has it managed to catch up keeping itself "pure" ?
- FartyMcFarter 4y agoThey use neural networks since Stockfish 12, but not in the same way that AlphaZero did: https://www.chessprogramming.org/Stockfish_NNUE https://www.chessprogramming.org/Stockfish_NNUE
- thom 4y agoStockfish now uses a neural network in some situations. Last I checked, it uses the classical, fast, tactical search and evaluation for active, tactical positions, and uses the network for quieter, more strategic positions.
- mtlmtlmtlmtl 4y agoStockfish has added neural networks for evaluation, much simpler than the large networks in MCTS engines, and efficiently CPU computable. The technique actually originated in a Shogi engine and was later ported to mainline Stockfish, so there's no direct inheritance from AlphaZero in that sense. The search algorithms have been incrementally improved, but still follow the same Alpha Beta and a heap of heuristics approach.
- robertlagrant 4y agoI can't help but wonder when this stuff happens that the devs (or the leads) are worried about being let go, so they open source their stuff to be able to take it elsewhere. Anyone here from DeepMind?
- Jensson 4y agoThe people who left probably said that they would want to work on stuff usable by the public, this seems like an answer to that.
- throwaway81523 4y agoThese particular programs were bleeding edge when they were announced, but have been recreated and improved on outside of DeepMind since then. For example, LCZero today is stronger at chess than any of the DeepMind chess programs that were shown to the public. Of course we don't know if they got further behind the scenes. Of course LCZero and everything else relied on the published papers of DeepMind. That is, the papers were more important than the code.
- largepeepee 4y agoAgreed, most "newly released" "open sourced" products have typically been in the pipeline for at least half a decade. That said many have been retooled for a more specific purpose before public release.
- mtlmtlmtlmtl 4y agoThere's an interesting bias I've noticed on HN where a lot of people still believe AlphaZero is the state of the art in computer chess, when that hasn't been true except for a short while after the release. AZ still gets posted and upvoted today while newer improvements in other engines are discussed far less. I guess it's because AZ came out of Google?
- bluecalm 4y agoWhat is interesting is that current state of the art comes from another place: https://en.m.wikipedia.org/wiki/Efficiently_updatable_neural_network https://en.m.wikipedia.org/wiki/Efficiently_updatable_neural... Lc0 is based on AlphaZero ideas and is significantly weaker than NNUE based modern Stockfish.
- kubb 4y agoMust feel pretty bad being a rank and file dev at DeepMind and realizing that your leads had all the talent and funding in the world that they could have used for building language model products and instead they heavily invested in solving board games. Then you realize they get paid >x10 of what you are and they're fine, but the next layoff is likely gonna get you.
- est 4y agoThey have their own LLMs such as PaLM
- tyler33 4y agowhy they dont make public apps?(at least)
- ogoparootbbo 4y agoThe human in the loop reinforcement learning paper that powered chatgpt's training arose from deepmind's experiments with boardgames (and games). AGI is still an unsolved problem and deep RL that arose from the success of deepmind's experiments wth boardgames and games so far, will likely play a huge part in it
- leereeves 4y agoLike Bell Labs of old, Google has excellent AI researchers but they haven't built many (any?) AI products people can use.
- Jensson 4y agoSearch? Gmail completing your emails? They have lots of AI products, they just wait until they are more solid before releasing stuff. Now they will rush out a competitor to ChatGPT before it is ready, but they would use it for something given time.
- 4y ago
- codeflo 4y ago“The heart of” can mean many things, is this runnable or not? Also, they should open source the weights as well so that their claims can finally be verified independently.
- Y_Y 4y agoTotally agree. I don't even know what benefit they'd get at this point from keeping some parts locked up. Anyway if you want something runnable Leela has a nice reimplementation: https://github.com/leela-zero/leela-zero https://github.com/leela-zero/leela-zero
- mourner 4y agoI'd suggest KataGo, which is much stronger and more actively developed than Leela Zero https://github.com/lightvector/KataGo https://github.com/lightvector/KataGo
- d4rkp4ttern 4y agoAnd let us not forget those secret random seeds!
- codeflo 4y agoIs this a joke I don't get? Because it's a bit unlikely there's something interesting in random seeds.
- Synaesthesia 4y agoPlease do the same for Alphastar, the starcraft AI, that would be great.
- notTooFarGone 4y agoThis would probably be against some Blizzard ToS
- Synaesthesia 4y agoI thought that they had an open API, and I also thought we as players and AI enthusiasts would benefit somehow from this venture, it was all rather exciting. But the research was all behind closed doors, and nothing much came of it except for some impressive demonstrations.
- mtlmtlmtlmtl 4y agoIn case you didn't know, SC2 does have a healthy AI bot community: https://sc2ai.net/ https://sc2ai.net/
- Synaesthesia 4y agoI am aware of that and follow it but its way behind what Alphastar was doing.
- knicholes 4y agoI talked to the guy who led AlphaStar at Blizzcon. I asked if I could get the weights. He said that balance changes and map changes makes keeping the model updated prohibitive.
- swyx 4y agosure but even frozen outdated weights would be very instructive to study
- 4y ago
- londons_explore 4y agoWorth noting that while AlphaGo and AlphaZero are incredible achievements, the amount of actual code to implement them isn't very much. If you have the research paper, someone in the field could reimplement them in a few days. Then there is the large compute cost for training them to produce the trained weights. So, opensourcing these bits of work without the weights isn't as major a thing as you might imagine.
- tiagod 4y ago> Then there is the large compute cost for training them to produce the trained weights. And as far as I understand, the training code is where the secret sauce lies.
- londons_explore 4y agogenerally no... Secret sauce is in the ML compiler and accelerator used, but all those improvements simply lower the cost of training a model. You could still do it on a regular GPU, it would just take you more time. In the case of Google, they probably used TPU chips that you can't get direct 'bare metal' access to anyway, so none of that code would have helped. The actual optimizer used and parameters (like the learning rate schedule) is normally published in the research paper.
- deleted 4y ago[deleted]
- amj2 4y agoYou should pencil out on a napkin just how long "more time" is. Here, i'll get you started: 1600 inferences per move * 1ms per inference * 250 moves/game * 30M games played = 12B seconds. 140k days; muzero with gumbel brought down the 1600 to ~40, but either way, you need some more scale. It turns out a lot of the difficulties, judgment calls, and implementation details involve data pipelining. Some of those choices affect the final skill ceiling you reach. Which ones? How much? Are they path dependent? Well, you'll need to run it more than once...
- xcombelle 4y agoactually it is not the original code of alphazero , the original code was tensorflow/TPU https://en.wikipedia.org/wiki/AlphaGo_Zero#Training https://en.wikipedia.org/wiki/AlphaGo_Zero#Training and this code is in JAX
- nmca 4y agoNote that JAX was created after the success of Alpha(Go|Zero), so the claim that this powered those papers is not accurate. It is a DM MCTS library though.
- banach 4y ago...and announce that they are changing name to OpenMind?
- nohat 4y agoI feel like this headline is deceptive. This is a Jax reimplementation, and it was released a year ago. It is a cool library though. The basic operation of muzero is very simple, but training it efficiently is tricky.
- fnbr 4y agoYeah. DeepMind has released various MCTS implementations (eg in OpenSpiel). Not the same as releasing AlphaZero.
- Recursing 4y agoInteresting approach to private variables https://github.com/deepmind/mctx/blob/577fc77a3cda1b796e277e8cd19a77e5811d2bf9/mctx/__init__.py#L69 https://github.com/deepmind/mctx/blob/577fc77a3cda1b796e277e...
- FartyMcFarter 4y agoIs there a better way to do it in Python?
- brilee 4y agoSo the naive MCTS implementation in Python is ridiculously inefficient. Of course, you could reimplement it in C++ but this then requires you to use the C wrappers of Tensorflow/JAX to do the MCTS/neural network interop. I came up with a nifty implementation in Python that outperforms the naive impl by 30x, allowing a pure python MCTS/NN interop implementation. See https://www.moderndescartes.com/essays/deep_dive_mcts/ https://www.moderndescartes.com/essays/deep_dive_mcts/ MCTX comes up with an even niftier implementation in JAX that runs the entire MCTS algorithm on the TPU. This is quite a feat because tree search is typically a heavily pointer based algorithm. It uses the object pool pattern described in https://gameprogrammingpatterns.com/object-pool.html https://gameprogrammingpatterns.com/object-pool.html to serialize all of the nodes of the search tree into one flat array (which is how it manages to fit into JAX formalisms). I suspect it's not a particularly efficient use of the TPU, but it does cut out all of the CPU-TPU round trip latency, which I'm sure more than compensates.
- cgreerrun 4y ago> I came up with a nifty implementation in Python that outperforms the naive impl by 30x, allowing a pure python MCTS/NN interop implementation. See https://www.moderndescartes.com/essays/deep_dive_mcts/ https://www.moderndescartes.com/essays/deep_dive_mcts/ Great post! Chasing pointers in the MCTS tree is definitely a slow approach. Although typically there are ~ 900 "considerations" per move for alphazero. I've found getting value/policy predictions from a neural network (or GBDT[1]) for the node expansions during those considerations is at least an order of magnitude slower than the MCTS tree-hopping logic. [1] https://github.com/cgreer/alpha-zero-boosted https://github.com/cgreer/alpha-zero-boosted