5 ms·

A DGX B200 costs like ~$0.5 M and uses around 14 kW. If you plan to run it straight for 8 years 100% max usage thats around 1 GWhr. A gigawatt hour is a lot o

A DGX B200 costs like ~$0.5 M and uses around 14 kW. If you plan to run it straight for 8 years 100% max usage thats around 1 GWhr. A gigawatt hour is a lot of energy but its not that much compared to the price of the actual machine. In Germany for example with its expensive energy thats about €100k worth, which spread over 8 years is pretty minor compared to the up front half mill. The real issue with high power consumption is not really the cost of energy but the limited powersupply you can get for a datacenter. A more efficient setup is highly desirable because it means you can fit more in the limited power hookup.

dannyw 3mo ago
It’s more than power supply. Cooling and ventilation becomes a MUCH bigger deal at rack scale, and that costs electricity too.
- thereisnospork 3mo ago
  Cooling demand is only fractional with respect to the load: cooling 1MW of heat will only cost a few 10's to low 100's of kW, depending on the specifics. 10-20% overhead on cooling is probably a close enough estimate for napkin math.
- psychoslave 3mo ago
  And datacenters have impact on everything around them. If at the end of the day to result is a few more yachts and jets and, a lot more of miserable humans starving in ruined ecosystems, maybe that’s not the best go-to direction.
  - butvacuum 3mo ago
    You say they have a large impact, but having lived somewhere with some of the largest data centers- they very much don't. At least not more so then any other structure that paves over greenery. love to debate actual discission points. pull up "datacenter dfw" on google maps for mine.
    - ffsm8 3mo ago
      The people having glass literally break from the vibrations would probably disagree with your opinion https://youtu.be/_bP80DEAbuo?is=sg09k66iutKFIFSo https://youtu.be/_bP80DEAbuo?is=sg09k66iutKFIFSo Yet here we are, discussing "data center" as if they're standardized and of similar (nose) isolation. There are no meaningful regulations in building them, and they can be incredibly polluting. So your experience with a potentially well isolated one is sadly not the norm going forward. And we don't even know how close you lived, if you're eg talking about "within 5km/3miles" then your experience would also have little value in this discussion in general.
      - jml7c5 3mo ago
        >The people having glass literally break from the vibrations would probably disagree with your opinion Can you cite a source for this? It's not in the video, as far as I can tell. I would be wary of Benn Jordan's videos. They are full of mistakes and misrepresentations, as Andy Masley has convincingly demonstrated: https://blog.andymasley.com/p/contra-benn-jordan-data-center-and https://blog.andymasley.com/p/contra-benn-jordan-data-center... I recall seeing Benn Jordan's responses on Bluesky and thinking they were quite poor. He was unwilling to admit to mistakes, and kept trying to grasp at newly searched papers that didn't actually support his arguments.
        hypfer 3mo ago
        Benn unfortunately is one of those people that actually feel stuff, which is a trait that easily gets exploited by bad actors. Indeed, he shot himself in the foot there pretty bad, but I would argue that that was just the result of successful Agitation. I would personally strongly prefer being in the same room with Benn compared with Andy, because one of them is authentic, while the other is calculating. Though, arguably, Benn has been catching up on that lately too. But yeah, taking stuff with a grain of salt should be the default regardless of the person speaking.
        redsocksfan45 3mo ago
        [dead]
- bayindirh 3mo ago
  With liquid cooling technologies (direct or rear door heat exchange), cooling efficiently is easier when compared to a decade ago, and it's pretty efficient when you compare the power consumption numbers (server total vs. cooling total). See PUE (Power Usage Effectiveness) for its scientific form.
jwpapi 3mo ago
Interesting so it’s supply chain and then you need to calculate how long it can be utilized and for how much you can sell it. Would love more calculations on that
heisenbit 3mo ago
Plus the power needed for cooling adding maybe 50%.
bayindirh 3mo ago
> but the limited powersupply you can get for a datacenter. Since many people haven't seen 10MW cabling for a data center or how a big GPU server is cabled, they naturally imagine connecting servers is akin to plugging an appliance to a wall. When the electricity provider says "I neither have the capacity, nor the required cables in that area", thing gets real.
minraws 3mo ago
It's not even about the costs, getting enough power for a large datacenter is impractically hard in most of the world at a single location. If it's efficient and the power costs of not just ongoing costs but the upfront setup is lower that makes a lot different scales of data centers practical, especially for inference which doesn't need massive super clusters. You can't just fire up gas turbines everywhere like US Data centers are doing. I am not even sure if that's legal in US... Note you have to plan for peak usage and a lot of stuff large scale data centers are insane infrastructure projects. Nvidia is both supply and price constrainted, sure if you are willing to pay over 0.5M$ you might get some, but if you try to balance out price to costs by going slightly lower on the pole you realize just how much more expensive Nvidia truly feels like AMD has a lot of margin to under cut them if they want to.
willis936 3mo ago
What they're really asking the authors is "can you not lie about performance cost and do proper accounting?". You can spin any story if you cherry pick your framing sufficiently. Stopping right at the silicon packaging boundary is as meaningless as it seems. The article is highly qualified but the headline is not. If they are not making general statements then they shouldn't open with them.

dannyw 3mo ago

It’s more than power supply. Cooling and ventilation becomes a MUCH bigger deal at rack scale, and that costs electricity too.

thereisnospork 3mo ago
Cooling demand is only fractional with respect to the load: cooling 1MW of heat will only cost a few 10's to low 100's of kW, depending on the specifics. 10-20% overhead on cooling is probably a close enough estimate for napkin math.
psychoslave 3mo ago
And datacenters have impact on everything around them. If at the end of the day to result is a few more yachts and jets and, a lot more of miserable humans starving in ruined ecosystems, maybe that’s not the best go-to direction.
- butvacuum 3mo ago
  You say they have a large impact, but having lived somewhere with some of the largest data centers- they very much don't. At least not more so then any other structure that paves over greenery. love to debate actual discission points. pull up "datacenter dfw" on google maps for mine.
  - ffsm8 3mo ago
    The people having glass literally break from the vibrations would probably disagree with your opinion https://youtu.be/_bP80DEAbuo?is=sg09k66iutKFIFSo https://youtu.be/_bP80DEAbuo?is=sg09k66iutKFIFSo Yet here we are, discussing "data center" as if they're standardized and of similar (nose) isolation. There are no meaningful regulations in building them, and they can be incredibly polluting. So your experience with a potentially well isolated one is sadly not the norm going forward. And we don't even know how close you lived, if you're eg talking about "within 5km/3miles" then your experience would also have little value in this discussion in general.
    - jml7c5 3mo ago
      >The people having glass literally break from the vibrations would probably disagree with your opinion Can you cite a source for this? It's not in the video, as far as I can tell. I would be wary of Benn Jordan's videos. They are full of mistakes and misrepresentations, as Andy Masley has convincingly demonstrated: https://blog.andymasley.com/p/contra-benn-jordan-data-center-and https://blog.andymasley.com/p/contra-benn-jordan-data-center... I recall seeing Benn Jordan's responses on Bluesky and thinking they were quite poor. He was unwilling to admit to mistakes, and kept trying to grasp at newly searched papers that didn't actually support his arguments.
      - hypfer 3mo ago
        Benn unfortunately is one of those people that actually feel stuff, which is a trait that easily gets exploited by bad actors. Indeed, he shot himself in the foot there pretty bad, but I would argue that that was just the result of successful Agitation. I would personally strongly prefer being in the same room with Benn compared with Andy, because one of them is authentic, while the other is calculating. Though, arguably, Benn has been catching up on that lately too. But yeah, taking stuff with a grain of salt should be the default regardless of the person speaking.
      - redsocksfan45 3mo ago
        [dead]
bayindirh 3mo ago
With liquid cooling technologies (direct or rear door heat exchange), cooling efficiently is easier when compared to a decade ago, and it's pretty efficient when you compare the power consumption numbers (server total vs. cooling total). See PUE (Power Usage Effectiveness) for its scientific form.

jwpapi 3mo ago

Interesting so it’s supply chain and then you need to calculate how long it can be utilized and for how much you can sell it. Would love more calculations on that

heisenbit 3mo ago

Plus the power needed for cooling adding maybe 50%.

bayindirh 3mo ago

> but the limited powersupply you can get for a datacenter. Since many people haven't seen 10MW cabling for a data center or how a big GPU server is cabled, they naturally imagine connecting servers is akin to plugging an appliance to a wall. When the electricity provider says "I neither have the capacity, nor the required cables in that area", thing gets real.

minraws 3mo ago

It's not even about the costs, getting enough power for a large datacenter is impractically hard in most of the world at a single location. If it's efficient and the power costs of not just ongoing costs but the upfront setup is lower that makes a lot different scales of data centers practical, especially for inference which doesn't need massive super clusters. You can't just fire up gas turbines everywhere like US Data centers are doing. I am not even sure if that's legal in US... Note you have to plan for peak usage and a lot of stuff large scale data centers are insane infrastructure projects. Nvidia is both supply and price constrainted, sure if you are willing to pay over 0.5M$ you might get some, but if you try to balance out price to costs by going slightly lower on the pole you realize just how much more expensive Nvidia truly feels like AMD has a lot of margin to under cut them if they want to.

willis936 3mo ago

What they're really asking the authors is "can you not lie about performance cost and do proper accounting?". You can spin any story if you cherry pick your framing sufficiently. Stopping right at the silicon packaging boundary is as meaningless as it seems. The article is highly qualified but the headline is not. If they are not making general statements then they shouldn't open with them.