4 ms·
Maybe a nitpick, but there's a difference between energy consumption during training and inference. If you want to talk about the energy necessary to train a hu
by pontus 6y ago
Maybe a nitpick, but there's a difference between energy consumption during training and inference. If you want to talk about the energy necessary to train a human brain, it involves years of that 20W power consumption. What is the power consumption for inference time for Dall-E?
- burrows 6y agoNot to mention the billions of years spent in the evolutionary pipeline.
- littlestymaar 6y agoWell, if you want to go that route, you'd need to count all the energy spent to build computers of all kind since the 50s, and also all the energy spent to sustain the lives of people working on AI. And, well, all the millions of years spent in the evolutionary pipeline before these people where born ;)
- littlestymaar 6y agoI did some kick napkin math for Lee Sedol vs AlphaGo a few weeks ago: https://news.ycombinator.com/item?id=25493358 https://news.ycombinator.com/item?id=25493358
- nojvek 6y agoWow. Good to know that entire lifetime energy consumption is 50 MWh. At $0.1/KWh, you’re looking at $5000 in equivalent electric energy consumption over entire lifetime of a human being. The brain uses 20W of power. For a life time of ~80 years, that is 14MWh of energy usage. Suppose we say the brain trains for the first 25 years then that is 4.38 MWh. Equivalent electric energy consumption is only at $438. So yeah, the brain is quite efficient both in hardware and software.
- pontus 6y agoThat's still really surprising I think. I would have imagined that it would have been off by many orders of magnitude. The fact that these models are within a factor of 3-10 of the human consumption is pretty impressive. That being said, these models are training only for very specific tasks whereas obviously the human brain is far more sophisticated in terms of its capabilities.
- littlestymaar 6y agoBear in mind that only a tiny fraction of the energy spent by Sedol's brain during his whole life was dedicated to learning Go. Even while playing Go, a human neural system spends a big part of its energy doing mundane stuff like moving your hand, standing up, deciphering the inputs coming from the eyes and every other sensitive body parts and subconsciously processing it (my back hurts I need to change posture, my opponent smells well, the light in the room is too bright, etc.). Interestingly enough, doing most of these things is also a big challenge for IA today.