3 ms·

It's unfortunate that no math is ever done in these stories. If you take the "350,000" H100s that Facebook wants by EOY, each of those can do 700W, which gives

by boulos 3y ago

It's unfortunate that no math is ever done in these stories. If you take the "350,000" H100s that Facebook wants by EOY, each of those can do 700W, which gives you almost 250 MW for just the GPUs. That sounds like a lot, until you realize that a single large power plant is measured in Gigawatts. All of Google's data centers combined are O(10 GW) which are matched with renewable power offsets [1]. Importantly, the world installed >500 Gigawatts of renewable energy in 2023 [2], mostly driven by PV Solar in China. The amount of potential solar and wind and other renewable-ish (hydro) outstrips even a 10x'ing of a lot of these numbers. But even for a single site, dams like Three Gorges are >20 GW. There are real efficiency and scale challenges in doing AI in a single, large site. But existing power generation systems deliver plenty of power. [1] https://www.gstatic.com/gumdrop/sustainability/google-2023-environmental-report-executive-summary.pdf https://www.gstatic.com/gumdrop/sustainability/google-2023-e... [2] https://www.iea.org/reports/renewables-2023/executive-summary https://www.iea.org/reports/renewables-2023/executive-summar...

jacquesm 3y ago
The big point of course is that there is massive asymmetry between training and inference and that even inference at scale is going to require massive amounts of energy and that likely OpenAI's business model isn't viable at scale. It works right now because they have capital to burn but when the music stops it may well turn out that their model isn't sustainable at all. Efficiency gains should come first, long before they start looking at alternative energy sources.
- deleted 3y ago
  [deleted]
nateglims 3y ago
Energy production capacity in the US is relatively flat for the last couple of decades. The renewable installation is offsetting the decommission of coal. The capacity installed in China is not really accessible to Open AI due to recent security competition (both export restrictions on AI and a desire to import less energy). The capital costs of power are also quite high so I think he is pretty accurate considering the expectations of a startup to hockey stick.
- boulos 3y ago
  https://www.publicpower.org/system/files/documents/Americas_Electricity_Generating_Capacity_2023_Update.pdf https://www.publicpower.org/system/files/documents/Americas_... via https://www.publicpower.org/resource/americas-electricity-generating-capacity https://www.publicpower.org/resource/americas-electricity-ge... disagrees with you even in the last decade. And the future seems mostly solar: > This report also analyzes prospective generation capacity in four categories — under construction, permitted, application pending, and proposed. More than 466,000 MW of new generation capacity is under development in the United States — a 13% increase over 2022. Sixty-one percent of capacity most likely to come online, permitted plants and plants that are under construction, are in solar. China's growth in power capacity is non-trivially due to increasing demand. If the US or Europe or wherever suddenly wanted to build XXX GW per year, they could (modulo bureaucracy, which is very real).
  - nateglims 3y ago
    I mispoke about capacity, but adding nameplace capacity hasn't increased the total energy generated, we've been sitting around 4T kWh for a while: https://www.eia.gov/energyexplained/electricity/electricity-in-the-us-generation-capacity-and-sales.php https://www.eia.gov/energyexplained/electricity/electricity-... The US or EU could likely do it, yes. If someone is willing to put up the capital to do it. The point of Sam Altman's statement is that he wants someone to do it.
xnx 3y ago
Definitely the right way to look at things. > If you take the "350,000" H100s that Facebook wants by EOY, each of those can do 700W Plus ~500W for cooling
- pclmulqdq 3y ago
  Less than a 1:1 ratio here is an extremely efficient datacenter. The heat has to get out of the building, not just off the chip.
  - xnx 3y ago
    Oh, wow. I didn't know it was that bad. Google advertises a PUE of 1.1: https://www.google.com/about/datacenters/efficiency/ https://www.google.com/about/datacenters/efficiency/
    - pclmulqdq 3y ago
      Just to clarify, PUE of 1.1 means 10% power waste not spent on computing. However, Google datacenters are extremely efficient, and they design everything end-to-end for efficiency. A normal datacenter with less efficient cooling, less efficient power delivery, less efficient buildings, etc. easily adds up to a lot more wasted power.
BenoitP 3y ago
Facebook is not alone, and there is growth. Also cooling is to be taken account of. And third: renewables need to be associated with its backup like hydro/step or ... batteries which cost a lot. Gas can't be taken in as it's not CO2-free. All that unless training and inference happen when there's the corresponding wind and sun shining. And I'm not seeing that happening right now.

by boulos 3y ago

jacquesm 3y ago

The big point of course is that there is massive asymmetry between training and inference and that even inference at scale is going to require massive amounts of energy and that likely OpenAI's business model isn't viable at scale. It works right now because they have capital to burn but when the music stops it may well turn out that their model isn't sustainable at all. Efficiency gains should come first, long before they start looking at alternative energy sources.

deleted 3y ago
[deleted]

nateglims 3y ago

Energy production capacity in the US is relatively flat for the last couple of decades. The renewable installation is offsetting the decommission of coal. The capacity installed in China is not really accessible to Open AI due to recent security competition (both export restrictions on AI and a desire to import less energy). The capital costs of power are also quite high so I think he is pretty accurate considering the expectations of a startup to hockey stick.

boulos 3y ago
https://www.publicpower.org/system/files/documents/Americas_Electricity_Generating_Capacity_2023_Update.pdf https://www.publicpower.org/system/files/documents/Americas_... via https://www.publicpower.org/resource/americas-electricity-generating-capacity https://www.publicpower.org/resource/americas-electricity-ge... disagrees with you even in the last decade. And the future seems mostly solar: > This report also analyzes prospective generation capacity in four categories — under construction, permitted, application pending, and proposed. More than 466,000 MW of new generation capacity is under development in the United States — a 13% increase over 2022. Sixty-one percent of capacity most likely to come online, permitted plants and plants that are under construction, are in solar. China's growth in power capacity is non-trivially due to increasing demand. If the US or Europe or wherever suddenly wanted to build XXX GW per year, they could (modulo bureaucracy, which is very real).
- nateglims 3y ago
  I mispoke about capacity, but adding nameplace capacity hasn't increased the total energy generated, we've been sitting around 4T kWh for a while: https://www.eia.gov/energyexplained/electricity/electricity-in-the-us-generation-capacity-and-sales.php https://www.eia.gov/energyexplained/electricity/electricity-... The US or EU could likely do it, yes. If someone is willing to put up the capital to do it. The point of Sam Altman's statement is that he wants someone to do it.

xnx 3y ago

Definitely the right way to look at things. > If you take the "350,000" H100s that Facebook wants by EOY, each of those can do 700W Plus ~500W for cooling

pclmulqdq 3y ago
Less than a 1:1 ratio here is an extremely efficient datacenter. The heat has to get out of the building, not just off the chip.
- xnx 3y ago
  Oh, wow. I didn't know it was that bad. Google advertises a PUE of 1.1: https://www.google.com/about/datacenters/efficiency/ https://www.google.com/about/datacenters/efficiency/
  - pclmulqdq 3y ago
    Just to clarify, PUE of 1.1 means 10% power waste not spent on computing. However, Google datacenters are extremely efficient, and they design everything end-to-end for efficiency. A normal datacenter with less efficient cooling, less efficient power delivery, less efficient buildings, etc. easily adds up to a lot more wasted power.

BenoitP 3y ago

Facebook is not alone, and there is growth. Also cooling is to be taken account of. And third: renewables need to be associated with its backup like hydro/step or ... batteries which cost a lot. Gas can't be taken in as it's not CO2-free. All that unless training and inference happen when there's the corresponding wind and sun shining. And I'm not seeing that happening right now.