3 ms·
What consumes more energy in AI, training or inference? It seems to me that training ought to be a super shift-able workload, a great fit for intermittent gree
by bee_rider 3y ago
What consumes more energy in AI, training or inference?
It seems to me that training ought to be a super shift-able workload, a great fit for intermittent green energy sources.
- tomn 3y agoThe expensive bit is not the electricity, it's the cost of the GPUs, so they will train flat out. There is also a massive amount of competition between the various players, and nobody taking part cares about the energy use (otherwise they wouldn't be doing it), so i don't see this happening. (rough numbers: a H100 is 700W, which is $613/year at $0.1/kWh, and the estimated cost is $25000-40000)
- MacsHeadroom 3y agoTypical datacenter power cost is closer to $0.04/kWh. Datacenters mostly use renewable clean energy as well. Environmental concerns are deeply misplaced. 10 minutes of compute in a datacenter is far more environmentally friendly than 10 hours of work by a human.
- bee_rider 3y agoNice, we can just run the ML model—how do you put humans in suspend mode? Or do we just shut them off for now?
- tomn 3y ago> Datacenters mostly use renewable clean energy as well. Environmental concerns are deeply misplaced. Source? I didn't find much. Google talk about buying "carbon-free energy", which isn't bad, but ultimately just moves the carbon to other uses. Energy is also not the only resource being wasted. I also tend to think that the way people talk about energy use of technology is misguided, but ultimately data center energy use does likely form a significant part of my personal carbon footprint, which i care about, so it's quite reasonable to argue against finding new exciting ways to use energy when, to me, the benefits are so often unclear or negative. > 10 minutes of compute in a datacenter is far more environmentally friendly than 10 hours of work by a human. I don't think this is a good way of looking at it. Lots of things that a computer can do quickly would take a human much longer, but that doesn't mean they are good uses of resources. A better solution is to do neither, and this applies to a lot of what the article talks about.
- MacsHeadroom 3y ago>A better solution is to do neither aka "degrowth" That's a non-starter. Doing more is a given. The best we can do is to do more efficiently. Doing more with datacenters is the reason US economic productivity/output has soared in the past 3 decades while per capita energy usage has been stagnant.