11 ms·
Another Way of Looking at Lee Sedol vs. AlphaGo
- wodenokoto 11y agoDeep blue using ~900 watts vs AlphaGo ~1.000.000 watts doesn't sound right.
- zamalek 11y agoThe assumption is 200W per CPU and 200W per GPU. 1920 × 200 + 200 × 280 = 440kW. That's 1.76MWh for the entire match.
- wodenokoto 11y agoThank you for the clarification.
- zamalek 11y agoTechnically you are right to question "1MW" because the author omitted the hours unit. 440kW is the correct wattage if you aren't talking about time.
- MagnumOpus 11y agoThe assumption is very obviously wrong, by as much as an order of magnitude. The TDP of the 24 core Xeon systems that were most likely used is 6.8W per core.
- zamalek 11y agoLet's assume that he actually meant 200W per blade, so: (1920core × 200W/machine ÷ 24core/machine + 200gpu × 280W/gpu) × 4h = 288kWh The original is still somewhat acceptable as a Fermi estimation and argument, as AlphaGo still uses more energy (3600 times more) with this conservative estimate.
- ricraz 11y agoI'm not sure why having a "fair" competition matters. The point of technological progress is to get better outcomes, not to get better outcomes using massively constrained hardware. Of course the latter is useful, but it's totally arbitrary to just pick energy usage as the one constraint to put on computers, there are hundreds of other ways in which they're advantaged. The important thing is that it can be done at all.
- emodendroket 11y agoI mean, it's cool now but not really practical to set up in your home, for instance.
- mcherm 11y agoThere are VERY few situations where I care even remotely how energy efficient someone's thinking is. The benefits of better thinking are nearly always far more valuable than the energy inputs. We are frequently limited by the fact that no more intelligence can be applied to the problem no matter what we do. In many other cases we are limited by cost -- we would happily hire a team of three planners or one expensive genius, whichever was smart enough to lead our project at the cheaper price. But even in extreme cases like manned spaceflight, I've never heard of hiring someone dumber because at least they ate less.
- smitherfield 11y agoEnergy efficiency does matter for small-scale stuff like mobile devices, and for medium-to-large-scale stuff like servers (unless money is no object). But yes, it's not really all that important for something like AI research. Especially considering that at the rate AlphaGo has been improving,* by some time in the next year or two it should be able to beat Sedol running on a smartphone. *And yes, there's no guarantee it will continue improving at the same rate.
- ekianjo 11y agoThat's not the point at all. The point is that if you decide tomorrow to run thousands of AIs like AlphaGo to replace humans to do several stuff, does that scale well? Well, if it consumes as much power as AlphaGo, it certainly does not. The point about having something running on low watts is critical for scalability in the longer term, because no matter how you look at it, energy production is finite (you can increase it over time but there are clear limits anyway).
- visarga 11y agoMachine learning is compatible with low energy applications. There is a body of research in slimming down heavy neural nets to run on cell phones with very little accuracy loss. AlphaGo uses so much power because it additionally uses a brute force approach to improve its results.
- ajuc 11y agoStill other way of looking at this - could the whole humanity playing in one team win against AlphaGo? If no - computers still win because they scale better (play better as a team).
- sarreph 11y agoLet's set the entire world population up as a huge deep neural network (each person acting as a node) and put that theory to the test! ;)
- smitherfield 11y agoAlphaGo is the product of the combined human effort to play Go as well as possible.
- pmontra 11y agoIMHO the whole humanity won't contribute much to the challenge. You take the best 5 pros and that's pretty much the best team you can pit against the AI. Ten pros would probably need too much time to discuss and coordinate. A striking analogy with software dev teams :-)
- ajuc 11y agoThat's the point - humans don't scale, and 1 person vs energetic equivalent computer is arbitrary (and human-prefering) level of comparison.
- visarga 11y agoOr if they played 5 second Go, that would reduce the time the computer can do Monte Carlo tree search, which is the brute force aspect of AlphaGo.
- DennisP 11y agoSomething like that was actually tried in 1999, when Kasparov played against the world: http://michaelnielsen.org/blog/kasparov-versus-the-world/ http://michaelnielsen.org/blog/kasparov-versus-the-world/ Given how well collective intelligence worked in that game, it'd be pretty interesting to try it against AlphaGo.
- faitswulff 11y agoThis is a great observation. It makes me curious as to what about the human brain allows it to do calculations so efficiently?
- chronial 11y agoThe human brain is in this situation much closer to an ASIC than a CPU. An average human can't play go on this level of skill – you need to use 20 years to turn your brain into the right kind of ASIC to play go. Also comparing the human brain and cpus in general, it should be noted that we are just good at different things. When it comes to basic arithmetic, human brains are much less energy-efficient than cpus.
- chongli 11y agoCalling a human brain an ASIC seems ridiculous to me. Lee Sedol, in addition to being a marvellous Go player, is also a typical human being. He can speak a language, read and write, probably drive a car, take care of his wife and child. AlphaGo is incapable of all those things. If you ask me, it's closer to an ASIC than a human is.
- Ciantic 11y agoIf chips were built for neural networks it would require significantly less power. One interesting thing I found by Googling FPGA (which is the easisest way to build a new chip) and neural networks, Microsoft is building one which requires considerabley less power: https://gigaom.com/2015/02/23/microsoft-is-building-fast-low-power-neural-networks-with-fpgas/ https://gigaom.com/2015/02/23/microsoft-is-building-fast-low...
- wheresmypasswd 11y agoThe AlphaGo team tweeted that the single node version wins 25% of the time against the cluster version. The bulk of the compute is in training the model... I would bet that a cell phone AlphaGo is <200 ELO (or whatever passes in the Go world) weaker than the massively distributed version--good enough to be competitive with Lee Sedol.
- andrepd 11y agoIt doesn't really matter what you bet. I want to see a Lee Sedol vs AlphaGo on a single consumer chip before I draw any conclusions.
- kotach 11y agoIf it works for chess, it'll work for Go. Chess has lots of games that you can learn from, Komodo wins any grandmaster or draws. The problem with Go was lack of evaluation function that would guide the policy. So it had to be learned simultaneously. You can leave AlphaGo to play a billion games and then learn a policy that requires little to no search but has almost perfect evaluation (local optimality of minimizing future regret). Same positional play is exhibited by Komodo, and it requires not that much of depth searching, while currently AlphaGo rolls out a whole game for every move.
- Ulti 11y agoI want to see Lee Sedol win with just a single neurone from his neocortex then...
- chillacy 11y agoGive it a few years and you can draw your conclusions, guaranteed. 60 years ago: "The ENIAC uses so much more energy and takes up more space than a human to multiply numbers, I want to see a calculator multiply faster than a human running on a 5V watch battery before I draw conclusions"
- chronial 11y ago> In the one corner: Human, all of 175 pounds of extremely well trained runner. And in the other corner, a Formula 1 racecar with a remote control running down a straight track. > > Nonsense, you’d say, that’s not a fair comparison No, I (and I think most sensible people) would say that's pointless – we know who's going to win. If you have a sufficiently smooth surface or it's worth creating one, cars are way better at their specific task than humans are. Everybody knows that and nobody uses humans for transportation or walks when they care about getting there fast. Why is everybody so scared that computers might now “be better” than humans?
- visarga 11y agoBecause humans tend to identify with those traits that are still superior to computers.
- andrepd 11y agoBecause they aren't. Not yet at least. They probably will be someday, but not for a long time.
- chronial 11y agoNo, the correct question is “better at what”? There are tons of tasks at which humans don't stand a chance against computers.
- 11y ago
- visarga 11y agoShifting goalposts. And humans cost much more energy than the watts contained in their food. Humans take 20 years to "train" and need lots of water, electricity, food, clothes, transportation and teaching time. If they were to use only the policy net part of AlphaGo they would need a few milliseconds per move and only one computer to run it, achieving the level of 1p. Also, if DeepMind used specialized Go hardware maybe they could reduce the power usage a lot, but they didn't optimize for that. For example, it would be interesting to calculate how much power is used in image recognition, man vs machine. The brain uses 20-40W and a cell phone only 5W, and we know cell phones can do image recognition, so, probably the computer uses less than the human given that human reaction time is 200ms and the computer could classify much more than 5 images per second, so it uses less time and consequently less energy per image.
- andrepd 11y ago>The brain uses 20-40W and a cell phone only 5W, and we know cell phones can do image recognition, so, probably the computer uses less than the human given that human reaction time is 200ms and the computer could classify much more than 5 images per second, so it uses less time and consequently less energy per image. I'm all for praising the enormous developments being made in modern computing, but this is absolutely needless exaggeration. Computers are far, far, far weaker at image recognition than even an 8 year old child. The amount of things humans can continually process, using a fraction of the brain (the brain is constantly doing visual processing without shutting the other centers down), whereas a computer can barely do simple tasks and takes longer to do that than "200ms", as you claim. The brain is a mind boggling machine. Our computers are getting pretty kickass, but there is still a long long way to go.
- emodendroket 11y agoI mean, I think he's right that it'd be a great achievement to be able to do this with standard hardware and not a purpose-built supercomputer.
- yohui 11y agoIBM's Deep Blue used custom hardware. DeepMind's AlphaGo used Google Cloud Platform: http://googleresearch.blogspot.com/2016/01/alphago-mastering-ancient-game-of-go.html http://googleresearch.blogspot.com/2016/01/alphago-mastering... It still uses a lot of hardware, but not hardware built for the sole purpose of playing Go.
- andrepd 11y agoAnother way of looking at this is that Lee Sedol wasn't playing against one machine, but against a team of thousands of players.
- Vraxx 11y agoDo you think a thousand amateurs working together could be Lee Sedol? I'm inclined to believe that having 1000 of them would only make an impossible feat harder. Strategy quality doesn't, in fact, scale with the number of individual actors creating the strategy.
- ikeboy 11y agoYou could also look at cost. How much would it cost to rent the alphago processing power from EC2, and how much does it cost to get a top player to agree to play you? (The actual match here doesn't count because of publicity, but I imagine general costs should be easily calculated.)
- guilhas 11y agoThe machine had the time to train against humans. But did Lee Sedol had the opportunity to train against the Algorithm?
- jbergens 11y agoIt's not simply an algorithm. It as learning computer. It remembers and learns from previous matches. He can probably play more matches against AlphaGo if he wants to but the computer will keep getting better and probably faster than he is.
- Strom 11y agoThere seems to be a lack of basic fact checking in this post. In the world of Chess it took until 1996 before a computer won against the then reigning world champion, Gary Kasparov in a series of 6 matches. 1) His name is Garry Kasparov 2) Kasparov won the 1996 series against Deep Blue 4-2, it was in 1997 that Kasparov lost 2.5-3.5
- robbiemitchell 11y agoI don't understand why a Joule-for-Joule "fair" competition will seem like a more important achievement. Computers can trade efficiency for raw performance, humans can't -- it's an unfair advantage. How the AlphaGo team balances those two over time will change, but for now I'm delighted to see what happens when they optimize for something that is beyond human potential.
- chillacy 11y agoDoubling alphaGo's compute will improve Go performance (up to a limit) Doubling a human's food intake will only lead to cardiac issues ;)
- debacle 11y ago> on the most sophisticated game we have constructed. Starcraft, anyone?
- ajuc 11y agoIt's comparable, I'd say. And go is much more elegant.
- johanneskanybal 11y agoWell what an irrelevant way of looking at it.
- tommynicholas 11y agoAgreed. The amount of energy it takes to do this will soon fit in my phone easily without having an impact on my battery life (as is the case with chess).
- emodendroket 11y agoMaybe, depending on how you define "soon."
- tommynicholas 11y agoTrue!
- roel_v 11y agoWhy do we count only the amount of energy the human uses during the match? After the match we can turn the machine off, making it use no energy. Humans need to spend 50+% of their time doing stuff that isn't even related to the activities we're measuring (eat, sleep, do laundry, drink wine with friends), and 99% of the rest on 'training'. I fail to see why energy-parity comparisons are 'more fair' or 'better'. We don't have soccer matches for people who eat nothing but two sandwiches a day, do we? Has a marathon winner cheated if he ate a few bananas during the race and the runner up hasn't? Of course not (well, 'of course' to me, it seems that the OP might disagree?). I guess we could move on to comparing sporting wins by people who used steroids, but that would just stray from the actual point.
- emodendroket 11y agoWell, motor vehicle races are generally broken up by how much horsepower the car has, right? Or boxing matches by weight class?
- roel_v 11y agoInteresting angle that I hadn't considered. That said, I don't think it's the same. Vehicles and fighters are classified in various ways (power/weight, but also experience, e.g. pro and amateur classes) because otherwise the competition becomes 'who has the best engineering team' and not 'who drives best'; or 'who has most fighting skills' and not 'who can overpower his opponent through sheer force/mass'. It's purely to keep it interesting and/or exciting. In this case, I interpreted the argument made in the OP as 'it only becomes an achievement when the computer constrains itself to the boundaries the human has by its nature'. I don't see the point of that. An F1 car is unarguably faster than 50cc kart. If we talk about 'who can drive the fastest', it doesn't make sense to say 'it's not a 'fair' comparison'. It is, and the F1 car is faster, period. The nature of the past match was 'who is the best Go player, no holds barred'. Call it the UFC 1 of Go. Back in 1994 in UFC 1 there were no weight classes, no restrictions on techniques - the question was 'who is the best fighter'. It's not like after that event the sumo guy said 'yeah well I can't move as fast because I'm 200kg, so we'll only know who is the best fighter after a jiu jitsu guy fights me but doesn't move faster than I do'. I guess there is more nuance to the debate than I originally thought. I still think it doesn't make sense to impose artificial limits. We might have tournaments for humans and tournaments for machines in the future, just like we have nascar and F1. But there is no mistake who is the best player.
- mbubb 11y agoAnother comparison would be how would the Formula 1 car do in terms of sourcing its own energy to keep going. Humans - in our primitive state were adapted for 'persistence hunting' - we ran in packs and drove animals into hyperthermia. The Formula 1 car would do quite well for the first few hundred miles. But you would eventually get an Aesopian 'tortoise and hare' effect. In the real world the OP's point is a good one. Lee SeDol could outlast AlphaGo by making it use too much in the way of resources (energy, human work, maintenance) untill he could get close enough to fell the beast with a stone arrow or hardened wood spear... https://en.wikipedia.org/wiki/Endurance_running_hypothesis https://en.wikipedia.org/wiki/Endurance_running_hypothesis
- mikeash 11y agoF1 cars are good at causing humans to tend to their needs, though! Much better than most humans. The average F1 car gets far more resources than, for example, I do. Here's a fun idea. Chess is thoroughly lost to the computers at this point, but what if we add in an element that would give the humans an advantage? There's a real actual sport called Chess boxing. The competitors spend three minutes playing chess, then three minutes pounding the crap out of each other, alternating back and forth. Let's see how well a modern computer can do at this game!
- mbubb 11y agoHeh, I found out about this recently. I have a coworker who is simultaneously quite good at chess and Brasilian JiuJitsu. I suggested a combination of the two and he told me about Chess boxing: https://en.wikipedia.org/wiki/Chess_boxing https://en.wikipedia.org/wiki/Chess_boxing
- dagw 11y agoI'd probably bet on this: https://www.robots.com/images/R-2000ia.JPG https://www.robots.com/images/R-2000ia.JPG vs. a human, especially since all it has to do is not get knocked out.
- mikeash 11y ago
- KnightOfWords 11y agoGood point, power consumption is often overlooked by proponents of AI.
- Vraxx 11y agoMy main complaint with this line of thought is this. It is claimed that it is not fair due to energy usage comparisons, but if you asked people before AlphaGo if it was a "fair" competition to pit a Go AI versus a world class professional, the answer would be hilariously no, because the world class professional would destroy the AI. I imagine telling them that you'll use more energy than them wouldn't change the consensus either. But now that a computer using so much energy did beat a world class professional, now this same scenario is framed as unfair for the professional. There's plenty of back and forth arguments that can be made as to whether the energy argument is a true test of fairness, but I don't think it matters if it's a good yardstick for fairness or not. It was an amazing feat to create an AI of any size that could beat a professional.
- mikeash 11y agoThe post starts with an analogy of a race between a runner and a race car. It says we'd call this unfair because the two are clearly not comparable. But I think it gets the reasons wrong. It says we'd call it unfair because the race car uses so much more energy. I think we'd call it unfair because the outcome is such a foregone conclusion. We know the race car is going to be way faster, so the race is pointless. It's a foregone conclusion, so why even bother unless you do something to make it more even? Go back to the very early days of automobiles, when they were so slow that a human runner would be faster. Then a new one comes along that can beat a human. The first car faster than a person! Would you be amazed that machine has beaten man? Or would you say that it somehow doesn't count, because machine used more fuel than man? I'm pretty sure most people would not go for the second one.
- placebo 11y agoI find people using various tricks to somehow console themselves of the fact that a machine beat a human to be quite funny. What is it exactly that they're trying to protect? Perhaps at the core is a fear that if someone or some thing does something better than me than I'm not worth as much, or more generally, if "they" (the machines) do things better than "us" then we're not worth as much, and what wouldn't we do to feel worthy... but I feel this whole nonsense stems from a belief that anything in existence could really be unworthy. I remember a short passage, from one of Raymond Smullyan's books noting two different responses from humans the day it is discovered that it is possible to make machines that are indistinguishable from humans. The first says glumly: "So, we are just machines?" The other says with joy: "I didn't know machines could be so wonderful". It's not a contest of worthiness - it's reality beautifully playing out and there are no unworthy players.
- jacquesm 11y ago> I find people using various tricks to somehow console themselves of the fact that a machine beat a human to be quite funny. I find it quite funny that you'd take what I wrote and that you'd manage to wrangle from it that we need to be consoled. I simply observed that there is another challenge, one that in chess has since the first computer beat a world champion been more than met and that in Go there is still some distance left to cover. Put another way, if the size and energy consumption would not matter do you think that there would have been improvement in the Chess programs post the point where they could beat any chess player in the world? Clearly the Chess programming community thought otherwise and we've seen a major improvement in algorithms and this resulted in a huge decrease in the required computing horsepower. That's smarter programming, not using larger computers to achieve the win through brute force and to me - feel free to disagree - that's an interesting prospect.
- placebo 11y agoOk, I made an unnecessary leap from knowing that many people do feel threatened by the increasing problem solving abilities of computers to assuming you are one of them. so I stand corrected, and in light of your response understand that my sermon was misplaced in this context. I don't disagree with you at all. In fact, I think probably we're headed for the same improvements over time with machines playing Go as was with machines playing Chess. I think that in addition to better algorithms, Chess had the advantage of having Moore's law being in full swing over these years, which will probably slow down, but there's always something new and unexpected coming up, so beyond multiple processors and GPU's with lower energy consumption I'm sure there will be even further improvements in the machine learning methods and other things that I can't yet imagine.
- Ulti 11y agoThis isn't a better view though. The number of Watts is not important. Biology is efficient in energy use, so what? If you tax a human brain it doesn't especially consume vast amounts more energy. I don't work up a sweat thinking. What's more interesting is if the computational "power" has parity. Deep Blue wasn't as computationally powerful as a human brain. It was specialised which is why it's not as interesting a victory as AlphaGo which is based on a more general principal.
- jacquesm 11y ago> Biology is efficient in energy use, so what? That's a very bold statement to make coming from the perspective of an industry that does just about everything it can to save energy. Better programming means lower energy consumption, to achieve a win like this on 1% of the energy budget would be a major game changer (no pun intended). AlphaGo is still using brute force quite a bit, the stage is set for a much improved batch of software that will focus less on brute force as a main strategy.
- verytrivial 11y agoThis is exactly the question I was asking myself this morning! I find the parallels to robotics interesting too: whenever another creepy beast lurches out of Boston Dynamics I wonder how such machines will function 50 or 100 year in the future when the world is operating under a different energy cost equation, when the fruits of millions of years of fossil fuel deposit have been spanked. Edit: I see lots of "no fair!" comments here. I think people are failing to grasp how supremely efficient biological systems are at dealing with exactly the sort of messy/noisy/lossy reasoning required for real-world problem solving, the sort of reasoning I see linked (incorrectly IMHO) by various media to the advancement demonstrated by this hyper-specialized Go playing machine. Edit2: (I removed that "little to show for it" bit before I saw your comment because it was leading to a digression I realised I would need to defend which seemed unwarranted in this context. Sorry about that!)
- pavel_lishin 11y ago> with little to show for it. Those fossil fuels allowed us to rapidly bootstrap ourselves through the industrial revolution, and to get to the point where renewable and carbon-neutral energy isn't just feasible, but already being used in many places. I'd say we've got quite a bit to show for it.
- jacquesm 11y agoAnd now we're telling the rest of the world they should focus on sustainability and the environment. The hypocrisy is quite incredible at times.
- pavel_lishin 11y agoIt would be less hypocritical if we were helping them set up a renewable/carbon-neutral infrastructure.
- Vraxx 11y agoIt is hypocritical, but that doesn't mean it's not a good idea. That's also not to say it isn't necessarily unfair. I tend to see it like progression in a game's tech tree, in some sense the prior steps (oil) were needed to get to a point where we can actually create energy in a clean and renewable way. In the future, there will likely be a tipping point where the energy created from clean and renewable sources can be used to help create new sources, but it likely wouldn't have been possible without the oil stepping stone. In an ideal and non-hypocritical scenario, the advancements in energy production would be shared to avoid the dirty oil intermediary step, but while there is still scarcity and separate national entities, there will still be hoarding and greed. A great implementation of this is type of thinking is Morocco's solar plant in the Sahara [0]. Though I don't think this would have happened if it wasn't also thought to be a good investment, and urging a bad investment that you're not willing to participate in would definitely be hypocritical. [0]:http://www.npr.org/sections/thetwo-way/2016/02/04/465568055/morocco-unveils-a-massive-solar-power-plant-in-the-sahara http://www.npr.org/sections/thetwo-way/2016/02/04/465568055/...
- iofj 11y agoBig problem in this way of looking at this: energy is measured in Joules, not in Watts. So this is not a fair comparison. Yes Lee was pretty cheap energy-wise to "run" during the game itself, but needed 150W or so constantly for 33 years to get trained to be as good as he is (brain is cheap, but obviously you have to keep the body alive too). Plus the massive support structure needed to raise him, teach him, get him interested in Go, provide opponents to train against. As you can't have a grandmaster. You need 10 grandmasters, 1000 really good players, 1000000 good players, and so on to train someone to get to this level. AlphaGo needed 1MW, probably about the same for training. If AlphaGo got trained in about a week, I bet the energy difference would favor AlphaGo, depending on what you count you could make AlphaGo or Lee Sedol win the comparison, so really, it's a tossup. And let's compare it to chess. Deep Blue used probably a similar amount of power in it's entire lifetime as Gary Kasparov will need for his lifetime. Maybe a factor 10 difference, but no more. However, a current chess computer uses less than a billionth the amount of power for learning & playing chess than Gary Kasparov needs to live and learn to play chess at his level.
- SubiculumCode 11y agoWhat about the energy it takes to grow one human?
- sguwwochqbib 11y agowhat about the energy it takes to make the chips ?
- deleted 11y ago[deleted]
- benologist 11y agoWhat is the point of only using as much energy as a human to beat a human when we are transitioning to renewable energy sources that are essentially infinite?
- sguwwochqbib 11y agoI love how many of the comments here are along the lines of: "yes Lee Sedol only consumes 20-40W during a game, but you have to sum all the energy consumed to create him, and the power he consumes when he's not playing etc" but then they fail to apply the same logic to computers Sum all the energy required to create computers over the past however many 50+ years then let's see if it's the case that computers made by evolved computers are any more efficient than their evolved creators
- aab0 11y agoThe money spent on CPU/GPU manufacturing improvement is a sunk cost and more than paid for itself along the way in economics benefits; the cost of the cluster that beat him is already around what he was paid for this one set of matches. In contrast, making a Lee Sedol will never be cheaper, and he lacks all of the other myriad advantages of software, like being instantly clonable and continuously improvable. You will never be able to mash up Shusaku and Sedol to see what they do, train them against each other for subjective millennia, examine the internals of their position evaluation, increase or decrease the sizes of their brains, compress them into much smaller neural nets, combine with other algorithms like MCTS, etc. Energy consumption is among the least important aspects of AI at the moment. We are not in a situation where we know how to make a feasible human-level AI but because electricity costs 10 cents/kwh no one can run, but they could if electricity cost 5 cents/kwh...
- sguwwochqbib 11y agosorry, you must have misread my comment I was talking about energy not money, but whatever
- sguwwochqbib 11y agoAlso, did I say that disagreed with AI ? No. I didn't. I'm all in favour of AI and I expect one day that it will be comparable in terms of energy consumption, generality and other measures we could invent. But for now, amazing though it is, this is not such a huge achievement as it's been made-out to be.
- abecedarius 11y agoTake the post's estimate at face value: 1 MW to run AlphaGo. Residential electricity costs on the order of $0.10/kWhr, or $10/hour for 1 MW. If the same machine were a better programmer than you, would you say "pshaw, come back when it beats me at power parity"? OK, the total cost of operation will be substantially higher (though it probably runs on cheaper electricity), maybe even approaching your total cost to your employer. This does bring up a point I've wondered about: why haven't we yet found a productive way to spend lots of computing power on making us better programmers? Usually the most cycles are spent on compiling and on test runs, costing only a tiny fraction of our hourly rates. (Maybe that compute budget for software development looks at least somewhat different at Google and some other places.)
- jacquesm 11y ago$10 will buy you 100KWh, not 1MWh, you're off by a factor of 10. > why haven't we yet found a productive way to spend lots of computing power on making us better programmers? Usually the most cycles are spent on compiling and on test runs, costing only a tiny fraction of our hourly rates. The Go language (ironically) addresses some of this by putting programmer productivity very high up in their feature list by focusing on compiler speed to the point where some of the differences between compiled and interpreted languages disappear.
- DonaldFisk 11y agoTurbo Pascal also had very fast compilation times (no more than a few seconds for large programs) 30 years ago, and it worked well then. But you don't need to compile the whole program, just the functions you have changed. With Lisp incremental compilation is often used, and it is almost instantaneous - just point the mouse at the function you want to recompile in EMACS, hit the load "chord", and it's ready to run in the listener. You're not even aware the compiler is there. Of course, there's a lot more to programmer productivity than compilation speed, as Fred Brooks pointed out.
- abecedarius 11y agoOops, thanks for the correction. OTOH zamalek elsewhere in these comments thinks 1MW is a considerable overestimate.
- nkurz 11y agowe are going to measure who is fastest. In the one corner: Human, all of 175 pounds of extremely well trained runner. And in the other corner, a Formula 1 racecar with a remote control running down a straight track. Over what distance? I think it's generally accepted that a human sprinter is faster than the car if the race is short enough. I can't find an example of human vs F1, but here's a fast sprinter vs a standard car: http://www.roadandtrack.com/motorsports/news/a21802/olympic-sprinter-out-accelerates-a-bmw-320d-34791/ http://www.roadandtrack.com/motorsports/news/a21802/olympic-... And here's a slow sprinter versus a much faster Indy car: https://www.youtube.com/watch?v=seNRu5JjpDM https://www.youtube.com/watch?v=seNRu5JjpDM The obvious comparison would be to a "blitz game" in Go, where I think that humans still have the advantage. Would a shorter (or longer) time be somehow inherently "fairer"?
- fuzzythinker 11y agoAnother way is by weight. Kind of like weight classes in boxing matches. Let Google cram all its hardware it can in the same weight as its opponent, including the batteries that will last for the duration of the match (actually, a longer match since the current match duration seems to be in AlphaGo's favor). Sure, it will probably still win in this setup say in a year or so, but it will provide a much closer match if done within the next few months.
- Houshalter 11y agoI've seen a lot of "AI denial" and moving goalposts in the discussions about AlphGo. Go was, for the longest time, a huge milestone for AI research. And now that it's finally succeeded, there's a lot of discussion about how it's not that significant. But this article takes the cake for moving goalposts. It declares that the Go match wasn't fair because AlphaGo got to use far more energy than Sedol. On the surface this claim is kind of absurd. You could give Sedol an electrical outlet with as much power as he wants. It wouldn't help at all. Humans can't make their brains bigger, or use more power, even if they want to. Which is one advantage the AI has. And that's totally fair. But beyond that, until now no has really cared about the energy usage of AIs. The fact an AI could even do a task on the level of a human was incredible, regardless how much energy was required. The Energy usage isn't important in this domain. It's board game playing. It's shifting goalposts to goals that don't even matter! But there is a valid reason to care about energy usage. It does cost money to run AlphaGo, and in other domains that could matter a lot. But even from the economic perspective, it's not accurate. I bet the food budget of a human is way more than the electricity budget of AlphaGo. For this economic estimate, you need to factor in more than just energy. A human takes 20 years to mature. AlphaGo took less than a week to get good, and a few months to be the best. And over the human's life, they will use lots of energy and food. Humans are extremely expensive, as can be estimated just from wages. If it requires only a rack of GPUs to replace an expert human, it may very well be worth it economically. Additionally, AlphaGo doesn't require a huge distributed network of GPUs. That improves it's performance only slightly. Google claims that a version of AlphaGo that runs on a single machine, was able to beat the distributed version 25% of the time. Which would still give it a higher Elo than Sedol, who only beat AlphaGo 20% of the time. Now besides all that, there is an important point here. Once you get past the accusations of "unfairness", it's still very interesting that human brains are so energy efficient. But the reason for that is that artificial neural networks are run on general purpose hardware. Human brains are highly specialized hardware. General purpose computers always consume vastly more energy than specialized circuits, optimized for energy usage. GPUs compute every possible synapse to 32 bit floating point precision. Most that computation is unnecessary. The majority of synapses are 0, and only a few bits of accuracy are required on active synapses. I know this because there has been a lot of recent research on efficiently running NNs. Especially on low budget hardware, like mobile phones or embedded devices. This is something that is already possible, and is only going to get better over time. In the future AlphaGo could be ported to cheaper hardware, and even to specialized FPGAs or ASICs. Those are incredibly energy efficient. But there was no reason to do this for the match vs Sedol. The fact they didn't do this, tells us nothing about the progress of AI.
- collyw 11y agoInteresting way to look at it, but my guess is that a chess program on your average mobile phone could beat your average person (i.e. no professional) at chess.