4 ms·

Intel, AMD just created a headache for datacenters

34 points by signa11 4y ago

PreInternet01 4y ago
> Server silos didn't see today's watt-gobbling, space-heater chips coming Well, data centers definitely had a trial run over a decade ago when 'blade servers' were all the rage. Since then, you can definitely get 'high density power' racks in most places (with 32A 240V feeds, or better, instead of the standard 8A or 12A, plus appropriate hot-air removal options), but this tends to be expensive. Compared to a blade server chassis, your typical GPU server isn't that bad, by the way, but stuffing a rack full of 'em isn't going to work (but, hey, you need space to put your general compute and storage nodes anyway, so diversify a bit!)
rektide 4y ago
I've long thought of the GPU servers with 6x or whatever top tier GPUs & cackle a bit that a regular 15A circuit in a house wouldnt have enough power, that it's creating considerably more heat than a plug in space heater. It's great to see massively multicore really ramping up in CPU land. Watts burned to push MHz way up are usually ill spent, but spending watts to scale out, to power huge caches, can often be quite efficient. We're just doing more work. By compare, buying a ton of amaller blades feels like there's much more redundant parts purchases, with lots of motherboards lots of sleds lots od nics, and those blades have nowhere near the core fabric you get from massive multicore. Putting more computing closer together decreases latency among the systems & decreases joules/bit, & we're making incredible gains getting more together. I dont think it'll have a massive total system impact, but also notable that transcievers are also >10W, and SSD's probably near there as well, especially with bigger & faster CXL designs we'll be seeing more & more of. It seems like watercooling is upcoming king, but theres a lot of componemts that still need some cooling & unless you go immersion that still has a big parts & budget cost. I feel like we could or should be using convection more, maybe turning servers off the horizontal & making them verticle, but that's obviously way harder to install & handle.
c_o_n_v_e_x 4y ago
>In pursuit of ever-higher compute density, chipmakers are juicing their chips with more and more power, and according to the Uptime Institute... Because that's what the market wants/wanted. The Silicon people aren't working in a vacuum, they're talking to customers. >this could spell trouble for many legacy datacenters ill equipped to handle new, higher wattage systems. The limits of air cooling - forced or natural convection - has been on everyone's radar for a long time. Any hyperscaler or DC operator worth their salt knows you aren't going to load up a rack in a legacy air cooled data hall with CPUs at these TDPs. These chips require new DCs with new cooling systems or legacy DCs to be retrofitted with the appropriate equipment. Precision / direct to chip cooling - good but has lots of connectors. You get to use water as your coolant which has great thermal properties and it's cheap. The risk is that you can get water in your equipment. Immersion in another option which comes in 2 flavors: single phase or 2 phase (the coolant boils). Both options require a high volume of coolant which is expensive. 2 phase immersion has the added challenge of needing a sealed enclosure, otherwise the coolant will leak while it is a vapor. You also have concerns about pressure relief that need to be addressed. On top of all that, there's material compatibility issues between the coolants and the PCBs. You're also not meant to immerse your fiber connections. Spray cooling eliminates the need for high volume of coolant but comes with it's own set of problems. It's still very exotic, used in defense applications.
Out_of_Characte 4y ago
Its weird to say that datacenters 'didn't see it coming' when every chip produced in the last few decades has relied on smaller process nodes to increase performance/watt/silicon-area at the cost of increased power density. Theres no good way around the power density so the first company that can integrate advanced active cooling features in chips themselves will have an advantage as cpu's are already hitting their tdp limits with your standard behemoth cooler unable to outscale the power density.
dogma1138 4y ago
Datacenters handled GPUs and mining rigs which are as power dense if not more no?
- simplotek 4y ago
  The article refers explicitly to "legacy datacenters ill equipped to handle new, higher wattage systems". I'm not convinced this article makes a good case. It sounds like it's complaining that old data centers need to invest in their infrastructure.
Kon-Peki 4y ago
I know next to nothing about this kind of stuff, but… If you’ve got an existing data center that’s going to be rendered obsolete, what do you do with it? I’m betting that we see some creativity. > "This is a key problem: removing greater volumes of lower-temperature heat is thermodynamically challenging," the analysts wrote. Ok, so you have the land, you have the building, you have customers. With the improved performance of the newer chips, can you pack them in less densely to get the same processing power, but maybe now the density is so low you can rip out some of your cooling infrastructure and lower your costs? Presumably you’ve already amortized out a ton of the building costs. It could be pretty profitable!

34 points by signa11 4y ago

> Server silos didn't see today's watt-gobbling, space-heater chips coming Well, data centers definitely had a trial run over a decade ago when 'blade servers' were all the rage. Since then, you can definitely get 'high density power' racks in most places (with 32A 240V feeds, or better, instead of the standard 8A or 12A, plus appropriate hot-air removal options), but this tends to be expensive. Compared to a blade server chassis, your typical GPU server isn't that bad, by the way, but stuffing a rack full of 'em isn't going to work (but, hey, you need space to put your general compute and storage nodes anyway, so diversify a bit!)

rektide 4y ago

I've long thought of the GPU servers with 6x or whatever top tier GPUs & cackle a bit that a regular 15A circuit in a house wouldnt have enough power, that it's creating considerably more heat than a plug in space heater. It's great to see massively multicore really ramping up in CPU land. Watts burned to push MHz way up are usually ill spent, but spending watts to scale out, to power huge caches, can often be quite efficient. We're just doing more work. By compare, buying a ton of amaller blades feels like there's much more redundant parts purchases, with lots of motherboards lots of sleds lots od nics, and those blades have nowhere near the core fabric you get from massive multicore. Putting more computing closer together decreases latency among the systems & decreases joules/bit, & we're making incredible gains getting more together. I dont think it'll have a massive total system impact, but also notable that transcievers are also >10W, and SSD's probably near there as well, especially with bigger & faster CXL designs we'll be seeing more & more of. It seems like watercooling is upcoming king, but theres a lot of componemts that still need some cooling & unless you go immersion that still has a big parts & budget cost. I feel like we could or should be using convection more, maybe turning servers off the horizontal & making them verticle, but that's obviously way harder to install & handle.

c_o_n_v_e_x 4y ago

>In pursuit of ever-higher compute density, chipmakers are juicing their chips with more and more power, and according to the Uptime Institute... Because that's what the market wants/wanted. The Silicon people aren't working in a vacuum, they're talking to customers. >this could spell trouble for many legacy datacenters ill equipped to handle new, higher wattage systems. The limits of air cooling - forced or natural convection - has been on everyone's radar for a long time. Any hyperscaler or DC operator worth their salt knows you aren't going to load up a rack in a legacy air cooled data hall with CPUs at these TDPs. These chips require new DCs with new cooling systems or legacy DCs to be retrofitted with the appropriate equipment. Precision / direct to chip cooling - good but has lots of connectors. You get to use water as your coolant which has great thermal properties and it's cheap. The risk is that you can get water in your equipment. Immersion in another option which comes in 2 flavors: single phase or 2 phase (the coolant boils). Both options require a high volume of coolant which is expensive. 2 phase immersion has the added challenge of needing a sealed enclosure, otherwise the coolant will leak while it is a vapor. You also have concerns about pressure relief that need to be addressed. On top of all that, there's material compatibility issues between the coolants and the PCBs. You're also not meant to immerse your fiber connections. Spray cooling eliminates the need for high volume of coolant but comes with it's own set of problems. It's still very exotic, used in defense applications.

Out_of_Characte 4y ago

Its weird to say that datacenters 'didn't see it coming' when every chip produced in the last few decades has relied on smaller process nodes to increase performance/watt/silicon-area at the cost of increased power density. Theres no good way around the power density so the first company that can integrate advanced active cooling features in chips themselves will have an advantage as cpu's are already hitting their tdp limits with your standard behemoth cooler unable to outscale the power density.

dogma1138 4y ago

Datacenters handled GPUs and mining rigs which are as power dense if not more no?

simplotek 4y ago
The article refers explicitly to "legacy datacenters ill equipped to handle new, higher wattage systems". I'm not convinced this article makes a good case. It sounds like it's complaining that old data centers need to invest in their infrastructure.

Kon-Peki 4y ago

I know next to nothing about this kind of stuff, but… If you’ve got an existing data center that’s going to be rendered obsolete, what do you do with it? I’m betting that we see some creativity. > "This is a key problem: removing greater volumes of lower-temperature heat is thermodynamically challenging," the analysts wrote. Ok, so you have the land, you have the building, you have customers. With the improved performance of the newer chips, can you pack them in less densely to get the same processing power, but maybe now the density is so low you can rip out some of your cooling infrastructure and lower your costs? Presumably you’ve already amortized out a ton of the building costs. It could be pretty profitable!