3 ms·
Forgive me if this I'm being waaay too naive, but it doesn't seem like the parallelism side of this is a big deal. Playing N games in parallel and then feeding
by slivym 9y ago
Forgive me if this I'm being waaay too naive, but it doesn't seem like the parallelism side of this is a big deal. Playing N games in parallel and then feeding them back in for training doesn't seem like an incredibly innovative idea?
On the timeline issue of course, I don't it's reasonable to make any assumptions about what that implies about the algorithms. It may be that Chess was solved, the dev team went on a 6 month binge, then shoved a Go board in front of the machine and it learned it in a weekend. Alternatively they may have gotten to a point where 5 of their top target games had special cases to learn before they were ready to show off the Chess engine, and that it took 7 months to get the other applications over the line.
- dsptri 9y agoPlaying the games simultaneously is actually innovative in how it let's them learn faster. If you play one game at a time consecutive moves are highly correlated which makes it difficult to make learning updates after each move. If you're playing 1250 parallel games that are starting and restarting so that they're at all at different stages of play, you can do an update after every move in every game and make better progress faster. This is known as Asynchronous Advantage Actor-Critic (A3C) and is a highly cited method discovered by the DeepMind team [1]. [1] https://arxiv.org/abs/1602.01783 https://arxiv.org/abs/1602.01783