5 ms·

So this saves energy because cooling with water is more efficient than with air. The Data center also needs to add water cooling for all power supplies and chi

So this saves energy because cooling with water is more efficient than with air. The Data center also needs to add water cooling for all power supplies and chips for maximum efficiency.

zozbot234 4y ago
Water cooling is hugely problematic because the cooling system will eventually spring a leak, and having water spraying/dripping all over your expensive server hardware is not anyone's idea of fun. That's why data centers use air cooling and a separate HVAC.
- newsclues 4y ago
  Data centre water cooling is just like car water cooling and it’s absolutely a thing
  - BeefWellington 4y ago
    Sure but notably car water cooling, as robust as it is, does eventually spring leaks. I think it's a question of whether the efficiency tradeoff is worth the maintenance tradeoff (which we probably won't know since there's not a ton of these kinds of solutions in use). With air cooling it's easier since the entire medium around the servers is the fluid. With liquid cooling, short of doing a bath setup (which doesn't appear to be what's proposed here) I'm not convinced it's going to work well; Just imagine 2-4 pipes running to every 2U server on a rack. IMO, we shouldn't be against trying an idea again if it seems like on paper it could make a lot of sense. There's a lot to like about the potential of this solution; improved efficiency, density, etc. Heck, you might even make a fun side project of trying to reclaim that waste heat and turn it into electrical generation. I just think the plumbing alone is likely to make it cost ineffective compared to existing air systems and then add quite a bit of potential maintenance headache.
    - smileysteve 4y ago
      > notably car water cooling, as robust as it is, does eventually spring leaks. Internal combustion engine cooling is not the same application as industrial computing. Differences include - ICE Cooling relies on pressure (boiling if released to atmospheric pressure - Maintenance (Schedules and Leak Detection); Most modern cars go a lifetime, if not 100k miles with no cooling system maintenance; if preventative maintenance is performed (replacement of radiator, pressure tank and/or rubber hosing regularly)and the system is monitored for level, repairs can be pre-empted. (very rarely does the leak happen in the block)) - Vibration and Environment (including cost or effort of maintenance)
- willis936 4y ago
  Water cooling will spring a leak when you're bad at plumbing. Similarly, I could spill my soda all over my keyboard. Is that my keyboard's fault?
  - moffkalast 4y ago
    Well it's your keyboard's fault for not being waterproof.
  - JohnJamesRambo 4y ago
    Do you have any data to back up your assertion that there is no age related leak issue with water cooling?
    - willis936 4y ago
      Water distribution is absolutely ubiquitous on the planet with leak rates that are quite low. When there are leaks it is because of an incident of damage, decades without maintenance, or poor implementation. Adding heat changes nothing. Many residential and commercial use water for heat exchange. Humans are good at this. Do you have any data to the contrary?
      - JohnJamesRambo 4y ago
        My data is the entirety of human existence haha. Have you never worked on an old radiator, boiler, car, hot water heater...
      - kergonath 4y ago
        > Many residential and commercial use water for heat exchange. Stupendous amounts of water are lost to leaks all over the place. The difference is that it does most of the time no damage, as it just goes to the ground. It still ends up in residential buildings every now and then, and it usually isn’t pretty.
- tristor 4y ago
  “Water” cooling in datacenter environments is almost always done with specialized liquids which are definitely not water, so a leak isn’t really an issue. Fluorinert is one of the more commonly used options.
  - zozbot234 4y ago
    Fluorocarbons have huge GHG potential so any leak is most definitely an issue.
    - jabl 4y ago
      Yes, I'm personally not very happy about another large scale usage of flourocarbons. Seems the history of these compounds is basically "look, we found this new compound with some very exciting and useful properties" followed a few decades later by "Oh shit". Rinse and repeat. That being said, there are alternatives. Like polyalphaolefin (PAO), which is apparently used for electronics cooling in aerospace applications and for some immersion cooling datacenter products.
    - tristor 4y ago
      I mean a leak isn't an issue as far as equipment damage, which is the general concern with "water cooling" leaking. These specialized fluids are non-conductive. In most of the facilities I've been in that used these special fluids they had specially designed floors that acted as drains and capture in case any leaks occurred or for incidental spillage as part of replacing components in immersion cooled systems. All of the technicians were properly trained for safe handling, and of course these fluids are not something you want to be exposed to without the proper PPE. How they were plumbed and contained, a leak would not result in direct contact of the fluid to a person, nor would it result in it escaping to the environment outside the containment area.
  - jabl 4y ago
    It's water. Or, if you want to be pedantic, a mixture of water and glycol as a fungicide and antibacterial agent.
- CoastalCoder 4y ago
  I wonder if your basing that on consumer-grade AIO coolers. I'm guessing that commercial-grade systems could use thicker hoses or actual pipes, and better seals / valves / clamps.
wongarsu 4y ago
That depends on what your water cooling system does. I can totally believe that circulating water through cooling pipes from the hvac to the GPUs is more energy efficient than moving huge amounts of air, and gives you better heat differentials. But if the datacenter doesn't have that and you build a more consumer-typical setup with a small pump and a radiator you're no more efficient than air cooling, you just moved the fans by a couple centimeters from the GPU to the radiator.
- jabl 4y ago
  Exactly this. Makes no sense for a single desktop computer, but when you have a huge room filled with servers, it's a lot more efficient to pump cooling water around than blowing a huge amount of air around. Also air cooling a rack full of equipment is getting more and more difficult as power densities increase, which is one of the main motivations of going to liquid cooling.

So this saves energy because cooling with water is more efficient than with air. The Data center also needs to add water cooling for all power supplies and chips for maximum efficiency.

zozbot234 4y ago

Water cooling is hugely problematic because the cooling system will eventually spring a leak, and having water spraying/dripping all over your expensive server hardware is not anyone's idea of fun. That's why data centers use air cooling and a separate HVAC.

newsclues 4y ago
Data centre water cooling is just like car water cooling and it’s absolutely a thing
- BeefWellington 4y ago
  Sure but notably car water cooling, as robust as it is, does eventually spring leaks. I think it's a question of whether the efficiency tradeoff is worth the maintenance tradeoff (which we probably won't know since there's not a ton of these kinds of solutions in use). With air cooling it's easier since the entire medium around the servers is the fluid. With liquid cooling, short of doing a bath setup (which doesn't appear to be what's proposed here) I'm not convinced it's going to work well; Just imagine 2-4 pipes running to every 2U server on a rack. IMO, we shouldn't be against trying an idea again if it seems like on paper it could make a lot of sense. There's a lot to like about the potential of this solution; improved efficiency, density, etc. Heck, you might even make a fun side project of trying to reclaim that waste heat and turn it into electrical generation. I just think the plumbing alone is likely to make it cost ineffective compared to existing air systems and then add quite a bit of potential maintenance headache.
  - smileysteve 4y ago
    > notably car water cooling, as robust as it is, does eventually spring leaks. Internal combustion engine cooling is not the same application as industrial computing. Differences include - ICE Cooling relies on pressure (boiling if released to atmospheric pressure - Maintenance (Schedules and Leak Detection); Most modern cars go a lifetime, if not 100k miles with no cooling system maintenance; if preventative maintenance is performed (replacement of radiator, pressure tank and/or rubber hosing regularly)and the system is monitored for level, repairs can be pre-empted. (very rarely does the leak happen in the block)) - Vibration and Environment (including cost or effort of maintenance)
willis936 4y ago
Water cooling will spring a leak when you're bad at plumbing. Similarly, I could spill my soda all over my keyboard. Is that my keyboard's fault?
- moffkalast 4y ago
  Well it's your keyboard's fault for not being waterproof.
- JohnJamesRambo 4y ago
  Do you have any data to back up your assertion that there is no age related leak issue with water cooling?
  - willis936 4y ago
    Water distribution is absolutely ubiquitous on the planet with leak rates that are quite low. When there are leaks it is because of an incident of damage, decades without maintenance, or poor implementation. Adding heat changes nothing. Many residential and commercial use water for heat exchange. Humans are good at this. Do you have any data to the contrary?
    - JohnJamesRambo 4y ago
      My data is the entirety of human existence haha. Have you never worked on an old radiator, boiler, car, hot water heater...
    - kergonath 4y ago
      > Many residential and commercial use water for heat exchange. Stupendous amounts of water are lost to leaks all over the place. The difference is that it does most of the time no damage, as it just goes to the ground. It still ends up in residential buildings every now and then, and it usually isn’t pretty.
tristor 4y ago
“Water” cooling in datacenter environments is almost always done with specialized liquids which are definitely not water, so a leak isn’t really an issue. Fluorinert is one of the more commonly used options.
- zozbot234 4y ago
  Fluorocarbons have huge GHG potential so any leak is most definitely an issue.
  - jabl 4y ago
    Yes, I'm personally not very happy about another large scale usage of flourocarbons. Seems the history of these compounds is basically "look, we found this new compound with some very exciting and useful properties" followed a few decades later by "Oh shit". Rinse and repeat. That being said, there are alternatives. Like polyalphaolefin (PAO), which is apparently used for electronics cooling in aerospace applications and for some immersion cooling datacenter products.
  - tristor 4y ago
    I mean a leak isn't an issue as far as equipment damage, which is the general concern with "water cooling" leaking. These specialized fluids are non-conductive. In most of the facilities I've been in that used these special fluids they had specially designed floors that acted as drains and capture in case any leaks occurred or for incidental spillage as part of replacing components in immersion cooled systems. All of the technicians were properly trained for safe handling, and of course these fluids are not something you want to be exposed to without the proper PPE. How they were plumbed and contained, a leak would not result in direct contact of the fluid to a person, nor would it result in it escaping to the environment outside the containment area.
- jabl 4y ago
  It's water. Or, if you want to be pedantic, a mixture of water and glycol as a fungicide and antibacterial agent.
CoastalCoder 4y ago
I wonder if your basing that on consumer-grade AIO coolers. I'm guessing that commercial-grade systems could use thicker hoses or actual pipes, and better seals / valves / clamps.

wongarsu 4y ago

That depends on what your water cooling system does. I can totally believe that circulating water through cooling pipes from the hvac to the GPUs is more energy efficient than moving huge amounts of air, and gives you better heat differentials. But if the datacenter doesn't have that and you build a more consumer-typical setup with a small pump and a radiator you're no more efficient than air cooling, you just moved the fans by a couple centimeters from the GPU to the radiator.

jabl 4y ago
Exactly this. Makes no sense for a single desktop computer, but when you have a huge room filled with servers, it's a lot more efficient to pump cooling water around than blowing a huge amount of air around. Also air cooling a rack full of equipment is getting more and more difficult as power densities increase, which is one of the main motivations of going to liquid cooling.