4 ms·

For hundreds of MW? https://www.visualcapitalist.com/cp/top-data-center-markets/ https://www.visualcapitalist.com/cp/top-data-center-markets/ You can do that

by rlupi 2y ago

For hundreds of MW? https://www.visualcapitalist.com/cp/top-data-center-markets/ https://www.visualcapitalist.com/cp/top-data-center-markets/ You can do that but they are the size of a mountain (literally: https://www.swissinfo.ch/eng/sci-tech/inside-switzerland-s-giant-water-battery/46915530 https://www.swissinfo.ch/eng/sci-tech/inside-switzerland-s-g... this is 900MW)

rapsey 2y ago
I don't know what the power consumption of an individual data center is, but your link talks about the consumption of all datacenters in individual states/countries.
- epistasis 2y ago
  A couple hundred megawatts is what I would expect from a data center, and had planned out before commenting, so that's on the mark. An H100 is very roughly a kW after adding in all the rest of the overhead, so a 200MW data center would have 200,000 H100s. That's massive cluster, but not inconceivable.
epistasis 2y ago
Why not? A typical value for land is 10 acres/MW, so 15-30 sq mi for 1-2 GW, which will average out to hundreds of megawatts over the course of 24 hours, even in shady days. Including land costs, solar is the cheapest source of energy, at <$1/W, which is a tiny fraction of the cost of the rest of the data center equipment, and has a 30-50 year lifetime. For less than $5B you could have hundreds of megawatts of continuous solar power backed by 24+ hours (5+GWh) of batteries. Hydro storage really can't compete with batteries for this sort of application, at least for new storage capacity. Existing hydro certainly is great, just building new stuff is hard. And GW-scale solar installations are fairly commonplace. Far easier to procure than a matching number of H100s.
- rlupi 2y ago
  Maybe it's possible, I haven't seen it done yet. I guess there were better alternatives. Well, I can't share numbers about datacenter MW sizes... the fact that I misread some of those numbers as per datacenter MW is telling :0 In any case, Meta (not my employer) has 24k GPU clusters. In the most dense (and less power hungry) setup, Nvidia superpods have 4x DGX per rack, 8 GPU per DGX (hyperscaler use HGX to build their stuff, but it's the same), and each rack uses ~40kW. That's 750 racks and 30MW of just ML, you need to add some 10-20% for supporting compute&storage, and other DC infrastructure (cooling, etc.). 24k GPU is likely one building, or even just one floor. Meta will likely have multiple clusters like that in the same POP. That's in the ballpark of 100+MW per datacenter, as the starting point.
  - epistasis 2y ago
    Oh I don't dispute your hundreds of MW at all. Other freely available information definitely supports that for the hypescalers. Browsing recent construction projects, I see 250,000 sq ft projects from Apple, and 700,000+ from the hyperscalers, and typically consumption is 150-300W/sqft, with the hyper dense DGX systems at that or above. There are lots of large scale renewable power projects out there waiting to get onto the grid, stuck in long interconnection queues, more than 1TW projects last I heard. There are also lots of data centers wanting to get big power connections, enough that utilities are able to scare their regulatory bodies to make bad short-term decisions to try to support the new load centers. Connecting the builders of these large projects directly to the new demand, and going outside the slow, corrupt, and inept utilities would solve a lot of problems. And you could still eventually get that big interconnection to the grid installed, and in the interim 3-5 years, power the data center mostly off-grid. Because that massive battery plus solar resource would eventually be a massive grid asset that could benefit everyone too, if the utilities weren't so slow.
    - bboygravity 2y ago
      I don't get how this would make sense. Assuming no hydro-electric: at night with no wind you'd need to draw the same 100('s) of MW from batteries (or use gas/coal/nuclear which defeats the purpose of ALSO using renewables on top of that 100 percent backup capacity). Batteries with that capacity are still extremely expensive (and massive in volume) which would essentially mean your energy price is 5 to 10x higher (ballpark rough estimate) than non-renewable continious sources. Not to mention the huge amount of land needed/wasted (costs money), there's a recycling problem (solar panels are not actually sustainable in the sense that they'll last forever or can be recycled) and so on. The only company in the world for which I could see that setup MAYBE make sense business-wise is Tesla/xAI: they could relatively quickly and cheaply roll-out massive battery storage, data center and solar (for example). If only to be slightly faster and bigger at rolling out than their competitors it could make sense from a business perspective. But that's only because they can produce massive battery capacity at the lowest possible cost and quickest turn-around. Maybe I'm missing something.

by rlupi 2y ago

rapsey 2y ago

I don't know what the power consumption of an individual data center is, but your link talks about the consumption of all datacenters in individual states/countries.

epistasis 2y ago
A couple hundred megawatts is what I would expect from a data center, and had planned out before commenting, so that's on the mark. An H100 is very roughly a kW after adding in all the rest of the overhead, so a 200MW data center would have 200,000 H100s. That's massive cluster, but not inconceivable.

epistasis 2y ago

Why not? A typical value for land is 10 acres/MW, so 15-30 sq mi for 1-2 GW, which will average out to hundreds of megawatts over the course of 24 hours, even in shady days. Including land costs, solar is the cheapest source of energy, at <$1/W, which is a tiny fraction of the cost of the rest of the data center equipment, and has a 30-50 year lifetime. For less than $5B you could have hundreds of megawatts of continuous solar power backed by 24+ hours (5+GWh) of batteries. Hydro storage really can't compete with batteries for this sort of application, at least for new storage capacity. Existing hydro certainly is great, just building new stuff is hard. And GW-scale solar installations are fairly commonplace. Far easier to procure than a matching number of H100s.

rlupi 2y ago
Maybe it's possible, I haven't seen it done yet. I guess there were better alternatives. Well, I can't share numbers about datacenter MW sizes... the fact that I misread some of those numbers as per datacenter MW is telling :0 In any case, Meta (not my employer) has 24k GPU clusters. In the most dense (and less power hungry) setup, Nvidia superpods have 4x DGX per rack, 8 GPU per DGX (hyperscaler use HGX to build their stuff, but it's the same), and each rack uses ~40kW. That's 750 racks and 30MW of just ML, you need to add some 10-20% for supporting compute&storage, and other DC infrastructure (cooling, etc.). 24k GPU is likely one building, or even just one floor. Meta will likely have multiple clusters like that in the same POP. That's in the ballpark of 100+MW per datacenter, as the starting point.
- epistasis 2y ago
  Oh I don't dispute your hundreds of MW at all. Other freely available information definitely supports that for the hypescalers. Browsing recent construction projects, I see 250,000 sq ft projects from Apple, and 700,000+ from the hyperscalers, and typically consumption is 150-300W/sqft, with the hyper dense DGX systems at that or above. There are lots of large scale renewable power projects out there waiting to get onto the grid, stuck in long interconnection queues, more than 1TW projects last I heard. There are also lots of data centers wanting to get big power connections, enough that utilities are able to scare their regulatory bodies to make bad short-term decisions to try to support the new load centers. Connecting the builders of these large projects directly to the new demand, and going outside the slow, corrupt, and inept utilities would solve a lot of problems. And you could still eventually get that big interconnection to the grid installed, and in the interim 3-5 years, power the data center mostly off-grid. Because that massive battery plus solar resource would eventually be a massive grid asset that could benefit everyone too, if the utilities weren't so slow.
  - bboygravity 2y ago
    I don't get how this would make sense. Assuming no hydro-electric: at night with no wind you'd need to draw the same 100('s) of MW from batteries (or use gas/coal/nuclear which defeats the purpose of ALSO using renewables on top of that 100 percent backup capacity). Batteries with that capacity are still extremely expensive (and massive in volume) which would essentially mean your energy price is 5 to 10x higher (ballpark rough estimate) than non-renewable continious sources. Not to mention the huge amount of land needed/wasted (costs money), there's a recycling problem (solar panels are not actually sustainable in the sense that they'll last forever or can be recycled) and so on. The only company in the world for which I could see that setup MAYBE make sense business-wise is Tesla/xAI: they could relatively quickly and cheaply roll-out massive battery storage, data center and solar (for example). If only to be slightly faster and bigger at rolling out than their competitors it could make sense from a business perspective. But that's only because they can produce massive battery capacity at the lowest possible cost and quickest turn-around. Maybe I'm missing something.