3 ms·

None of this is particularly new. Horizontal submerging has been an area of research for at least 20 years. Spinny disks were a pain to do, because they need a

None of this is particularly new. Horizontal submerging has been an area of research for at least 20 years. Spinny disks were a pain to do, because they need a certain amount of air to not have head crashes. (Sealed HDDs existed I'm sure. ) The problem is that its not that practical. Firstly it much less dense than upright racks, secondly its almost certainly more heavy. What is happening now is direct watercooling to the rack. Currently in widespread use is effectively a back of rack "car" radiator (ie it looks like a rack sized car radiator), where the coolant is piped to the back of the rack. Another, newer, but needed for high density GPU, is coolant block direct to component cooling. Where you have a coolant manifold on the back of the rack with each server plumbed direct into the coolant loop. However, that doesn't solve the problem of what to do with the heat once you have it.

szvsw 2y ago
> However, that doesn't solve the problem of what to do with the heat once you have it. Unfortunately they are not usually in great locations for this since you need other buildings, but if you have an ambient temperature district heating and cooling system for buildings nearby (sometimes called a 5th gen DHC system [1] - mainly characterized by the notion of prosumers ie producer/consumers of heat connected to a single loop system, rather than a more traditional supply loop/return loop topology) then datacenters can very easily fit in to the mesh as producers. Temps are typically 10-20c for ambient loops. The main insight is that that operating range is one that can be used quite well for both extracting heat and dumping heat simultaneously at different locations in the network. You can also of course just add the heat directly to a hot water supply loop, eg [2] [1] https://en.m.wikipedia.org/wiki/District_heating https://en.m.wikipedia.org/wiki/District_heating (see the section in history on 5th gen) [2] https://www.nrel.gov/computational-science/waste-heat-energy-reuse.html https://www.nrel.gov/computational-science/waste-heat-energy... https://www.nrel.gov/docs/fy19osti/73644.pdf https://www.nrel.gov/docs/fy19osti/73644.pdf
blackeyeblitzar 2y ago
> However, that doesn't solve the problem of what to do with the heat once you have it. Wouldn’t we next see a change to the design of data centers themselves (buildings, cooling systems, etc) that house these heat dense racks, to provide more capacity in removing heat?
- KaiserPro 2y ago
  We've migrated away from using aircon/heatpumps to move heat from inside a building to outside, to basically just swamp coolers: https://engineering.fb.com/2018/06/05/data-center-engineering/statepoint-liquid-cooling/ https://engineering.fb.com/2018/06/05/data-center-engineerin... they are effectively glorified wet cardboard and a fan
the_sleaze_ 2y ago
Less that it's not practical, more that it's too revolutionary to be accepted. Don't want to take too many risks with your new billion-dollar datacenter contruction. These make much more sense when you look at the energy contracts available where the weather is too hot for "traditional" cooling. Whether that will overcome the natural hesitancy is anyones guess, but probably a nice little gap for someone to disrupt.

szvsw 2y ago

> However, that doesn't solve the problem of what to do with the heat once you have it. Unfortunately they are not usually in great locations for this since you need other buildings, but if you have an ambient temperature district heating and cooling system for buildings nearby (sometimes called a 5th gen DHC system [1] - mainly characterized by the notion of prosumers ie producer/consumers of heat connected to a single loop system, rather than a more traditional supply loop/return loop topology) then datacenters can very easily fit in to the mesh as producers. Temps are typically 10-20c for ambient loops. The main insight is that that operating range is one that can be used quite well for both extracting heat and dumping heat simultaneously at different locations in the network. You can also of course just add the heat directly to a hot water supply loop, eg [2] [1] https://en.m.wikipedia.org/wiki/District_heating https://en.m.wikipedia.org/wiki/District_heating (see the section in history on 5th gen) [2] https://www.nrel.gov/computational-science/waste-heat-energy-reuse.html https://www.nrel.gov/computational-science/waste-heat-energy... https://www.nrel.gov/docs/fy19osti/73644.pdf https://www.nrel.gov/docs/fy19osti/73644.pdf

blackeyeblitzar 2y ago

> However, that doesn't solve the problem of what to do with the heat once you have it. Wouldn’t we next see a change to the design of data centers themselves (buildings, cooling systems, etc) that house these heat dense racks, to provide more capacity in removing heat?

KaiserPro 2y ago
We've migrated away from using aircon/heatpumps to move heat from inside a building to outside, to basically just swamp coolers: https://engineering.fb.com/2018/06/05/data-center-engineering/statepoint-liquid-cooling/ https://engineering.fb.com/2018/06/05/data-center-engineerin... they are effectively glorified wet cardboard and a fan

the_sleaze_ 2y ago

Less that it's not practical, more that it's too revolutionary to be accepted. Don't want to take too many risks with your new billion-dollar datacenter contruction. These make much more sense when you look at the energy contracts available where the weather is too hot for "traditional" cooling. Whether that will overcome the natural hesitancy is anyones guess, but probably a nice little gap for someone to disrupt.