3 ms·
Electricity costs are basically irrelevant because the cards are so expensive. A100 cards consume 250w each, with datacenter overheads we will call it 1000 kil
by Taek 4y ago
Electricity costs are basically irrelevant because the cards are so expensive.
A100 cards consume 250w each, with datacenter overheads we will call it 1000 kilowatts for all 2048 cards. 23 days is 552 hours, or 552,000 kilowatt hours total.
Most dataceneters are between 7 and 10 cents per kilowatt hour for electricity. Some are below 4. At 10 cents, that's $53,000 in electricity costs, which is nothing next to $30 million in capital costs.
- datadeft 4y ago> Electricity costs are basically irrelevant because the cards are so expensive. You mean in terms of money. I think this is exactly the problem that we have in CS, nobody really cares about CO2.
- CGamesPlay 4y agoNo, I'm willing to bet the CO2 cost of the cards is also way higher than the electricity. Those things are built on the global supply chain, with materials potentially making multiple thousands of kms journeys between each step.
- yunwal 4y agoLong term I also imagine it's much cheaper to run these large model trainings on renewables. It's a very centralized process that doesn't necessarily need 100% availability. The manufacturing process, however, is totally decentralized, and NVIDIA mostly manufactures in China where coal is cheap.
- verall 4y agoA100s are manufactured in Taiwan.
- yunwal 4y agoTrue, but previous chips have been manufactured in China, and they’re also developing and manufacturing their successor to A100s (H100s) in China. https://www.cnbc.com/2022/09/01/nvidia-says-us-government-allows-ai-chip-development-in-china.html https://www.cnbc.com/2022/09/01/nvidia-says-us-government-al...
- verall 4y agoThat says "partially developed" in China. H100s will probably be manufactured in Taiwan just like other x100 chips. All consumer SKUs that I know of are manufactured in China. By volume this is certainly the majority of manufacturing.
- Taek 4y agoI believe in terms of climate impact the chips (made in Taiwan) overwhelm everything else.
- Taek 4y agoBecause the cards are so expensive, you really do want them running 24/7. The electricity is not a big deal for these really expensive chips
- SSLy 4y agoI'm on some strong hopium that those DC's run on renewables or nuclear, green energy.
- jedberg 4y agoPretty sure Facebook uses green energy for their datacenter, so the CO2 cost should be nothing.
- TaylorAlexander 4y agoI always feel there is an opportunity cost here though. If that green energy wasn’t being used for compute it could be available to heat someone’s home instead of them using dirty sources.
- echoer_lyrist 4y agoUS grid mix produces about 0.855 pounds of CO2 per kWh[0]. So 552,000 kWh 452,640 pounds of CO2 which is 205.31 metric tons. At a cost of $40 per tonne[1] of CO2 that works out to $8,212.40 which is still small compared to the capital cost of the cards. [0] https://www.eia.gov/tools/faqs/faq.php?id=74&t=11 https://www.eia.gov/tools/faqs/faq.php?id=74&t=11 [1] https://www.pnas.org/doi/full/10.1073/pnas.1609244114 https://www.pnas.org/doi/full/10.1073/pnas.1609244114
- waffletower 4y agoAWS us-west-2 is housed in The Dalles and Prineville, Oregon. Not only are they near a massive wind farm in the Columbia Gorge, but also quite near the Columbia river's many hydro-electric dams. Facebook and Apple also have Prineville data centers. They are built there intentionally. Electricity at many data centers is quite carbon-lean.
- lmeyerov 4y agoThe comparison would be renting, not buying the GPUs I believe capex <> opex is more 1:1 nowadays, so something feels off here...