3 ms·

200 kW/rack is absolutely insane to me. The power consumption of these facilities is just...ridiculous.

by HolyLampshade 7mo ago

Bombthecat 7mo ago
Since you and me and everyone else will foot the elecricy bill. Energy consumption or efficiency is not a concern.
- dylan604 7mo ago
  It's the water use that concerns me
  - speed_spread 7mo ago
    In theory the water stays clean and can be reused. But I assume these cheapskates will go for evaporative cooling everytime? Then yeah, we need laws against that.
    - carlosft 7mo ago
      I keep hearing people claiming that water is just as much as issue as energy for operating these DCs, but that just doesn't make any sense to me. However, I haven't had to step inside a DC for almost two decades.
      - fc417fc802 7mo ago
        Continuously dissipating 1 gigawatt of energy by boiling room temperature water would require approximately 1.38 million liters of water per hour. Seems like the environmentally responsible thing to do be to build the datacenter near the coast and use the waste heat to desalinate water. Or at least dissipate the heat into the ocean rather than boiling off an inland freshwater supply.
        dylan604 7mo ago
        And kill the local aquatic life as you raise the temp beyond their happy place?
        fc417fc802 7mo ago
        Setting aside a small patch of ocean for the task seems like a much better plan than the current practice. Provided you dump it in a place with a decent current any adversely affected area should be exceedingly small. Keep in mind that the sun is constantly dumping energy on us. Absorption averaged across the entire earth is ~200 W/m^2. Assuming I didn't misplace some zeros somewhere then a gigawatt corresponds to ~5 km^2 of ocean surface. That's the daily flux. Penetration falls off exponentially so 75% of that only ever makes it ~10 m down. I think the takeaway here is the utterly incomprehensible scale of the ocean.
        Imustaskforhelp 7mo ago
        This idea is probably more worth it in middle eastern countries given that 90% of their water comes from Desalination Plants. But given the recent war within region, I don't really expect Datacenters to be built within the region for quite a long time.
        blitzar 7mo ago
        run it through a turbine and generate electricity to power the datacenter - infinite energy and infinite ai unlocked.
    - fc417fc802 7mo ago
      I guess when you're dissipating upwards of a gigawatt of power at a single site boiling water starts to look attractive. It's a pretty impressive curveball; I definitely would never have predicted "an evil corporation boils off all the local drinking water" to be a legitimate concern. I'm pretty sure that's too absurd a plot point for even a children's movie.
      - Imustaskforhelp 7mo ago
        > an evil corporation boils off all the local drinking water Nestle jumps into my mind whenever I want to think of an evil corporation and water together.
    - Rapzid 7mo ago
      Yeah because it's cheaper they go evaporative. That's an easy fix by just making it more expensive. People talk about the water usage like it's an intrinsic feature of datacenters; it's not. You just make it more expensive so they are forced to conserve. But you wait till you have to so you don't push them to build elsewhere.
  - keeda 7mo ago
    This may help you feel better: https://andymasley.substack.com/p/the-ai-water-issue-is-fake https://andymasley.substack.com/p/the-ai-water-issue-is-fake
ineedasername 7mo ago
With respect to consumption, it’s pretty efficient vs older traditional servers, though I know workloads like that aren’t completely fungible. Nonetheless it bears keeping in mind that a single GB200 NVL72 rack provides 1.4 ExaFLOPS of AI compute (at FP4 precision, ideal circumstances, but this is envelope math all around). So it’s power efficient, for what it is.
- HolyLampshade 7mo ago
  Oh, I have no doubt it is functionally efficient. I'm just amazed given the system deployments I've been party to, and the tiny amount of per rack energy usage comparatively speaking given the functionality of those systems. Like, what in the good god damn are we using all this energy for?
  - bayarearefugee 7mo ago
    > Like, what in the good god damn are we using all this energy for? Bad AI porn, terrible AI music, AI scams and completely devastating the labor market. And based on the recent Anthropic/Pentagon rift... I guess also creating autonomous kill-bots and doing mass surveillance. Just a bunch of super cool stuff.
    - speed_spread 7mo ago
      You left out overthrowing governments with customized targeted propaganda, jamming citizen discussion with noise, artificially creating and nourishing contrarian cells in democratic societies. The machines will now be programming people.
    - 71bw 7mo ago
      >and completely devastating the labor market. ? lol
Aurornis 7mo ago
The compute power is also ridiculous. Some of the reason for the high density is that you need devices physically close to each other to share such bandwidth. It’s not because we’re limited by the physical building space, because we can construct buildings all day long. Sending bits around at ultra high speed is hard and you need to keep all of the devices physically close to avoid having your interconnect costs explode.
- HolyLampshade 7mo ago
  Interestingly the realm in which I have domain experience has similar constraints, but based primarily on physical transport latency and less on bandwidth. There has been a move in some spaces towards hyper-dense deployments, but it’s a very small amount of the total compute capacity due to other limitations. Still, the world I’m used to operating in is typically 5-10 kVA/rack.
ehnto 7mo ago
There are box trucks with less power consumption.

200 kW/rack is absolutely insane to me. The power consumption of these facilities is just...ridiculous.

Bombthecat 7mo ago

Since you and me and everyone else will foot the elecricy bill. Energy consumption or efficiency is not a concern.

dylan604 7mo ago
It's the water use that concerns me
- speed_spread 7mo ago
  In theory the water stays clean and can be reused. But I assume these cheapskates will go for evaporative cooling everytime? Then yeah, we need laws against that.
  - carlosft 7mo ago
    I keep hearing people claiming that water is just as much as issue as energy for operating these DCs, but that just doesn't make any sense to me. However, I haven't had to step inside a DC for almost two decades.
    - fc417fc802 7mo ago
      Continuously dissipating 1 gigawatt of energy by boiling room temperature water would require approximately 1.38 million liters of water per hour. Seems like the environmentally responsible thing to do be to build the datacenter near the coast and use the waste heat to desalinate water. Or at least dissipate the heat into the ocean rather than boiling off an inland freshwater supply.
      - dylan604 7mo ago
        And kill the local aquatic life as you raise the temp beyond their happy place?
        fc417fc802 7mo ago
        Setting aside a small patch of ocean for the task seems like a much better plan than the current practice. Provided you dump it in a place with a decent current any adversely affected area should be exceedingly small. Keep in mind that the sun is constantly dumping energy on us. Absorption averaged across the entire earth is ~200 W/m^2. Assuming I didn't misplace some zeros somewhere then a gigawatt corresponds to ~5 km^2 of ocean surface. That's the daily flux. Penetration falls off exponentially so 75% of that only ever makes it ~10 m down. I think the takeaway here is the utterly incomprehensible scale of the ocean.
      - Imustaskforhelp 7mo ago
        This idea is probably more worth it in middle eastern countries given that 90% of their water comes from Desalination Plants. But given the recent war within region, I don't really expect Datacenters to be built within the region for quite a long time.
      - blitzar 7mo ago
        run it through a turbine and generate electricity to power the datacenter - infinite energy and infinite ai unlocked.
  - fc417fc802 7mo ago
    I guess when you're dissipating upwards of a gigawatt of power at a single site boiling water starts to look attractive. It's a pretty impressive curveball; I definitely would never have predicted "an evil corporation boils off all the local drinking water" to be a legitimate concern. I'm pretty sure that's too absurd a plot point for even a children's movie.
    - Imustaskforhelp 7mo ago
      > an evil corporation boils off all the local drinking water Nestle jumps into my mind whenever I want to think of an evil corporation and water together.
  - Rapzid 7mo ago
    Yeah because it's cheaper they go evaporative. That's an easy fix by just making it more expensive. People talk about the water usage like it's an intrinsic feature of datacenters; it's not. You just make it more expensive so they are forced to conserve. But you wait till you have to so you don't push them to build elsewhere.
- keeda 7mo ago
  This may help you feel better: https://andymasley.substack.com/p/the-ai-water-issue-is-fake https://andymasley.substack.com/p/the-ai-water-issue-is-fake

ineedasername 7mo ago

With respect to consumption, it’s pretty efficient vs older traditional servers, though I know workloads like that aren’t completely fungible. Nonetheless it bears keeping in mind that a single GB200 NVL72 rack provides 1.4 ExaFLOPS of AI compute (at FP4 precision, ideal circumstances, but this is envelope math all around). So it’s power efficient, for what it is.

HolyLampshade 7mo ago
Oh, I have no doubt it is functionally efficient. I'm just amazed given the system deployments I've been party to, and the tiny amount of per rack energy usage comparatively speaking given the functionality of those systems. Like, what in the good god damn are we using all this energy for?
- bayarearefugee 7mo ago
  > Like, what in the good god damn are we using all this energy for? Bad AI porn, terrible AI music, AI scams and completely devastating the labor market. And based on the recent Anthropic/Pentagon rift... I guess also creating autonomous kill-bots and doing mass surveillance. Just a bunch of super cool stuff.
  - speed_spread 7mo ago
    You left out overthrowing governments with customized targeted propaganda, jamming citizen discussion with noise, artificially creating and nourishing contrarian cells in democratic societies. The machines will now be programming people.
  - 71bw 7mo ago
    >and completely devastating the labor market. ? lol

Aurornis 7mo ago

The compute power is also ridiculous. Some of the reason for the high density is that you need devices physically close to each other to share such bandwidth. It’s not because we’re limited by the physical building space, because we can construct buildings all day long. Sending bits around at ultra high speed is hard and you need to keep all of the devices physically close to avoid having your interconnect costs explode.

HolyLampshade 7mo ago
Interestingly the realm in which I have domain experience has similar constraints, but based primarily on physical transport latency and less on bandwidth. There has been a move in some spaces towards hyper-dense deployments, but it’s a very small amount of the total compute capacity due to other limitations. Still, the world I’m used to operating in is typically 5-10 kVA/rack.

ehnto 7mo ago

There are box trucks with less power consumption.