4 ms·
Worth noting that while AlphaGo and AlphaZero are incredible achievements, the amount of actual code to implement them isn't very much. If you have the researc
by londons_explore 4y ago
Worth noting that while AlphaGo and AlphaZero are incredible achievements, the amount of actual code to implement them isn't very much.
If you have the research paper, someone in the field could reimplement them in a few days.
Then there is the large compute cost for training them to produce the trained weights.
So, opensourcing these bits of work without the weights isn't as major a thing as you might imagine.
- tiagod 4y ago> Then there is the large compute cost for training them to produce the trained weights. And as far as I understand, the training code is where the secret sauce lies.
- londons_explore 4y agogenerally no... Secret sauce is in the ML compiler and accelerator used, but all those improvements simply lower the cost of training a model. You could still do it on a regular GPU, it would just take you more time. In the case of Google, they probably used TPU chips that you can't get direct 'bare metal' access to anyway, so none of that code would have helped. The actual optimizer used and parameters (like the learning rate schedule) is normally published in the research paper.
- deleted 4y ago[deleted]
- amj2 4y agoYou should pencil out on a napkin just how long "more time" is. Here, i'll get you started: 1600 inferences per move * 1ms per inference * 250 moves/game * 30M games played = 12B seconds. 140k days; muzero with gumbel brought down the 1600 to ~40, but either way, you need some more scale. It turns out a lot of the difficulties, judgment calls, and implementation details involve data pipelining. Some of those choices affect the final skill ceiling you reach. Which ones? How much? Are they path dependent? Well, you'll need to run it more than once...
- Yuioup 4y agoIt's already been done: https://github.com/lightvector/KataGo https://github.com/lightvector/KataGo
- ta_tunestub 4y agoYep, there are many reimplementations. Here is a reimplementation that swaps out a neural net with a GBDT to address compute costs: https://github.com/cgreer/alpha-zero-boosted https://github.com/cgreer/alpha-zero-boosted
- woah 4y agoHow does the performance of this version compare?
- ArtWomb 4y agoImagine its perfect for Computer Backgammon, but overfits higher dimensional spaces ;)
- cgreerrun 4y agoDepends on game/environment and—since it's using a GBDT and not a NN—how good you are at feature extraction/selection for your problem. High level, I'd say it's a good way to test a new environment w/out spending time/effort on GPUs until you understand the problem well, and then you can switch to the time/money costly GPU world.
- rhdunn 4y agoThere is Leela Zero (https://github.com/leela-zero/leela-zero https://github.com/leela-zero/leela-zero) for Go and lc0/Leela Chess (https://github.com/orgs/LeelaChessZero/repositories https://github.com/orgs/LeelaChessZero/repositories) for Chess, where both provide trained weights. The Leela Chess project specifically have been working for a long time on training and refining the weights for Chess, as well as providing the code -- they allow you to see the history and performance over time for the various trained models.
- deleted 4y ago[deleted]
- CSMastermind 4y agoIf I'm not mistaken Stockfish has it's own neural network implementation as well correct?
- ragnarsson 4y agoyes https://cp4space.hatsya.com/2021/01/08/the-neural-network-of-the-stockfish-chess-engine/ https://cp4space.hatsya.com/2021/01/08/the-neural-network-of...
- sebstefan 4y agoI'm moderately into chess but I have never heard of Leela I'm shocked to discover it's been rated higher than AlphaZero & Komodo and just slightly below Stockfish
- ArtWomb 4y agoModels can be massive, but also totally doable. Just to put things in perspective: ProcMaze solving using DeepMind MCTX converges <1M steps. Whereas a physically based agent such as HalfCheetah may require >100M steps to learn to run. Q-learning Pac-Man on snapdragon chromeos is ~1hr for 1000 epochs ;)
- mensetmanusman 4y agoIf only there existed a distributed way to incentivize calculation of AI weightings while also providing a currency to encourage scale…
- eternalban 4y agonot an entirely bad idea.
- lern_too_spel 4y agoAs many people have pointed out before, the distributed currency part adds energy waste. BOINC accomplishes the same without the waste.
- mensetmanusman 4y agoOf course there will be trade-offs of inefficiency, that is always the case with distribution. Is it valuable to have open source AI systems is the countering question to that…
- lern_too_spel 4y agoThe point is you can develop open source AI cheaper without blockchain.
- brilee 4y ago> If you have the research paper, someone in the field could reimplement them in a few days. Hi I did this while I was at Google Brain and it took our team of three more like a year. The "reimplementation" part took 3 months or so and the rest of the time was literally trying to debug and figure out all of the subtleties that were not quite mentioned in the paper. See https://openreview.net/forum?id=H1eerhIpLV https://openreview.net/forum?id=H1eerhIpLV
- westurner 4y agoReplication crisis: https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis : > The replication crisis (also called the replicability crisis and the reproducibility crisis) is an ongoing methodological crisis in which the results of many scientific studies are difficult or impossible to reproduce. Because the reproducibility of empirical results is an essential part of the scientific method,[2] such failures undermine the credibility of theories building on them and potentially call into question substantial parts of scientific knowledge. People should publish automated tests. How does a performance-optimizer know that they haven't changed the output of there are no known-good inputs and outputs documented as executable tests? Pytest-hypothesis seems like a nice compact way to specify tests. AlphaZero: https://en.wikipedia.org/wiki/AlphaZero https://en.wikipedia.org/wiki/AlphaZero GH topic "AlphaZero" https://github.com/topics/alphazero https://github.com/topics/alphazero I believe ther are one or more JAX implementations of AlphaZero? Though there's not yet a quantum-inference-based self-play (AlphaZero) algorithm? TIL about the modified snow plow problem is a variation on TSP, and there are already quantum algos capable of optimally solving TSP.
- dllthomas 4y agoI think I agree with everything you've said here, but just want to note that while we absolutely should (where relevant) expect published code including automated tests, we should not typically consider reproduction that reuses that code to be "replication" per se. As I understand it, replication isn't merely a test for fraud (which rerunning should typically detect) and mistakes (which rerunning might sometimes detect) but also a test that the paper successfully communicates the ideas such that other human minds can work with them.