4 ms·
Data center mech eng here - from our perspective it's higher rack densities typically due to GPUs. It's certainly possible to have high densities due to CPUs as
by tjmc 4mo ago
Data center mech eng here - from our perspective it's higher rack densities typically due to GPUs. It's certainly possible to have high densities due to CPUs as well but I've seen a significant spike in rack densities in the last couple of years which has caused a switch from air cooling to liquid to chip.
One side effect of higher density is less footprint on the building to exhaust the heat, which is one reason (the main one being efficiency) that cooling towers and indirect evaporative cooling are favoured over air cooled condensers which leads to large amounts of water consumption.
Cooling towers are also much quieter than air cooled condensers which is a significant factor near any residential areas. It would be great to see more use of data center waste heat for process or district heating to save on water consumption.
Another issue with AI training in particular is huge (multi-MW) swings in power consumption at the start and end of each training run which must be a nightmare for the sparkies.
- mike_hearn 4mo agoIt is a nightmare for sparkies! See Meta's creative solution: https://github.com/pytorch/pytorch/pull/132936/changes#diff-98e548dc46290c54900149a6fcf818fe33f775196ace51405978a709976c39e9R41 https://github.com/pytorch/pytorch/pull/132936/changes#diff-...
- zozbot234 4mo agoIsn't that just HPC?