10 ms·
Another way of thinking about how efficient the brain is: By the article’s numbers, about 5.5 million TPU hours were required to train the machine to play as we
by hjnilsson 6y ago
Another way of thinking about how efficient the brain is: By the article’s numbers, about 5.5 million TPU hours were required to train the machine to play as well as a Go champion.
A Go champion might have trained for 8 hours a day, for 15 years (age 5 to 20). That is about 40 000 hours.
In other words, machines required 137 times longer to learn the game, and at twice the power consumption! There is still a lot of room for improvement.
- FartyMcFarter 6y agoGo champions don't learn from zero. They learn from teachers, books, and playing against each other. This knowledge is built over hundreds, or thousands of years.
- simonh 6y agoAlphago didn't learn from zero either. It has a pre-processor that identifies sets of patterns with known features, and also: "AlphaGo was initially trained to mimic human play by attempting to match the moves of expert players from recorded historical games, using a database of around 30 million moves".
- joycian 6y agoAlphaGo != AlphaGo Zero
- PeterisP 6y agoThat's for an earlier system (which also used less compute). AlphaGo was followed by AlphaGo Zero (which is the topic of this article) which did not use the process that you describe, it used only the rules of the game and the winning condition.
- simonh 6y agoOops, my mistake. Thanks for the correction.
- 29athrowaway 6y agoAnd long collaborative study sessions.
- hjnilsson 6y agoYes! So perhaps one way to make the machine more efficient, is by one of pre-programmed “general” models, that can be attuned to a particular problem in a much shorter time?
- phreeza 6y agoThis comparison is not entirely fair because the human brain also benefits from priors baked in over the entire course of evolution.
- Drakim 6y agoSo did the machine, albeit indirectly.
- ben_w 6y agoOTOH, I expect that avoiding human evolutionary priors is necessary for superhuman performance.
- darepublic 6y agoYeah only the last few human layers needed to be trained for the GO expansion pack, all the early layers were frozen during GO training.
- andbberger 6y agoThat's a pretty big claim. One could argue that the topology of the brain is a prior, analogous to the architecture of a neural net. But considering that we really have no idea how learning happens in the brain on a large scale, you really can't say.
- amelius 6y agoTrue, but a spider doesn't figure out how to build a web all by itself. That's to say, there is a lot that evolution can provide us with as a prior.
- andbberger 6y agoYes it is clearly possible to encode behavioral priors, there are many examples from different species. But humans aren't spiders. We've got the big brain, it's kind of our thing
- PeterisP 6y ago
- vbezhenar 6y agoHuman does not learn Go from the scratch on himself. He's using teachers, books which present compressed knowledge which was crystallized from many millions of human hours. If you would ask someone to learn Go, but only present him rules of the game, he'll likely be weak player (although probably with some original strategies).
- elcritch 6y agoIt'd be really interesting if a research group could calculate an entropic calculation on how efficient training any given neural network would be. As in what is the thermodynamic limit of the most optimal NN training could be in terms of watts per bit trained. My hunch would be that human brains would operate close to this limit. At least in our standard environmental conditions. Based on how near optimal biomaterials are in terms of strength to weight ratios it wouldn't surprise me much.
- simiones 6y agoI think the problem you'd find is that "bit trained" is probably highly non-trivial. For example, I expect that the training required to go from 7-year-old child to Go grand master requires a completely different number of bits of information than the training required to go from blanks-late NN to NN Go Grand master. I also suspect that the difference in what is being learned may well dominate the difference in training efficiency. Both the prior knowledge and the mechanism of learning are so different that I doubt you could get a meaningful comparison based on current understanding. You should remember that we have no idea basically how human beings actually learn things, and no idea how much prior knowledge we have encoded. Just for an example, I once saw a documentary that claimed chess grandmasters seem to recognize valid chess positions using the parts of the brain that usually recognize faces. Assuming that was true (I'm not claiming it is) perhaps a part of their chess learning consisted in taking a built-in face recognizing NN and training it to recognize chess boards. How much did the built-in knowledge of recognizing faces help? I don't think it would be possible to calculate.
- 29athrowaway 6y agoRather than Go champion I would rather use the term Go professional. There is a difference between being a professional and winning professional tournaments. Now, the bot has many advantages. It never sleeps, never gets distracted, never dies and can be copied to another system to obtain a copy of the bot with the same playing performance. The bot is also more accessible. Any player now can train with a bot, all day if you want, for almost free. You cannot do that with a professional.
- Scarblac 6y agoBut there are also many other people spending time studying Go who didn't reach that level. We ran all that studying in parallel and then selected the best person by running a world championship. You can't only count his effort alone.
- hjnilsson 6y agoTrue. But that single brain, in that person was that efficient. And represents the theoretical gap in efficiency to the machine. There are for example, other NNs also being trained to play Go, should all unsuccessful attempts be counted into the machine total? The comparison is almost impossible then.
- visarga 6y ago> In other words, machines required 137 times longer to learn the game, and at twice the power consumption! This comparison is a bit unfair. Humans are the result of evolution on a grand scale. Human Go is the result of millennia of gameplay. A human does not become grand master in isolation. AlphaGo is the result of an evolutionary tournament style competition of a much smaller duration and breadth. AG is also a population, not just one agent, and it would be silly to take just one agent and evaluate it on its own as if it could be created without the others. Should we include the human costs as well in AG, why just the electricity and CPU?
- superkuh 6y agohttps://github.com/lightvector/KataGo https://github.com/lightvector/KataGo >KataGo's latest run used about 29 GPUs, rather than thousands (like AlphaZero and ELF), first reached superhuman levels on that hardware in perhaps just three to six days, and reached strength similar to ELF in about 14 days. With minor adjustments and a few more GPUs, starting around 40 days it roughly began to match or surpass Leela Zero in some tests with different configurations, time controls, and hardware. And finally after about four months of training time, the current run may be wrapping up fairly soon, but we hope to be able to continue it or begin another run in the future.