5 ms·
What Nvidia's Blackwell efficiency gains mean for DC operators
- gwbas1c 3y ago"DC" is very confusing here. "DC" normally means "Direct Current," not "Data Center." I recommend updating the title. (And if anyone has connections back to The Register, suggest that they update the original article too.)
- astrodust 3y agoAt what point do you use the waste heat to generate electricity? 120KW is not an insignificant amount of energy.
- 2OEH8eoCRo0 3y agoI'm not sure it falls under cogeneration but I suppose it might if your waste heat can be captured and used to power the GPUs. https://www.energy.gov/eere/iedo/combined-heat-and-power-basics https://www.energy.gov/eere/iedo/combined-heat-and-power-bas...
- aaronblohowiak 3y agoI am not a physicist. In my (mis)understanding of things, the work you can make heat do comes from the difference in temperature, with a bigger difference allowing more work to be done efficiently. Unfortunately, the chips are kept pretty cool in order to stay in their operating range, so you end up with a large volume of warm but not hot stuff. This is hard to turn into electricity. It can be good for things where you want stuff to be warm (like heating homes, etc) but not great for electricity generation.
- namibj 3y agoYeah it's basically not worth it with silicon to recreate electricity. It's good for heat, though.
- ryukoposting 3y agoPerhaps, instead of building them in deserts, we should be building datacenters in cold places where the ambient temperature inside the datacenter should be kept warmer than outside air.
- barryrandall 3y agoI think there's a lot of potential in combining immersion cooling with utility-scale heat pumps. The heat from the coolant isn't high enough to produce power on its own, but it's constant.
- astrodust 3y agoChips these days can run as high as 90-95°C which is awfully close to the boiling point of water, and could even be past the boiling point of other liquids, at least when under pressure. It's not something you could do with a typical closed-loop water cooling system, but if you used different fluids, converted the loop into a heat exchanger, and had some kind of Sterling engine to recover energy...
- acchow 3y agoThey cannot do anything at all with the heat. The temperature differential is so small, they would get nearly zero useful work out of it. Such a low temperature differential in fact that the heat dissipating passively is not enough to run the chips - they need to use active cooling. See also second law of thermodynamics.
- deleted 3y ago[deleted]
- kristianp 3y ago> saying it is 30x faster than the Hopper generation when inferencing a 1.8 trillion parameter mixture-of-experts model. Is that the size of gpt4? The upcoming systems seem designed to fit gpt4 on one system with 4bit quantisation, making gpt4 cheaper to deliver.
- sigmoid10 3y agoIt's the alleged size of GPT4. There's no official info anymore about architectures from OpenAI since they turned for-profit. But if the number is true, the new generation of cards still won't be able to run GPT4 on a single instance, because even with 4bit, it would need 900GB of VRAM just for the parameters. You'd need at least four B200s for that. And we're not even talking about activations yet.