4 ms·

I am mildly surprised they can dissipate enough heat from the pods--those drives are packed so close together. I can't tell from the photos: Is there any airfl

I am mildly surprised they can dissipate enough heat from the pods--those drives are packed so close together. I can't tell from the photos: Is there any airflow between them? (For that matter, do higher temperatures decrease MTTF for hard drives?)

brianwski 15y ago
Backblaze employee here. Yes, the 6 fans move a lot of air through the drives. We monitor every drive in every pod in our datacenter with the smartmontools (/usr/sbin/smartctl) which includes the temperature of the drives. The drives stay well within their recommended operating temperatures as long as the fans are running. In the past, we have detected that several fans failed in one single pod when alarms went off saying some drives were running hot, so the fans are working (normally) and the air is flowing. Side note: personally I think it's really awesome that consumer grade drives come with such detailed internal monitoring for free. I also think it is disappointing that desktop consumer OS's don't monitor these things BY DEFAULT and popup dialogs explaining to you when your drive is exhibiting high temperatures or high data loss!
mturmon 15y ago
Temp does not matter as much as has been thought. See: http://labs.google.com/papers/disk_failures.html http://labs.google.com/papers/disk_failures.html IIRC, up to about 40 C, failure rates are pretty flat as a function of temperature. The paper has some nice graphs. There is one other large-scale study in circulation that showed the same thing, but I don't recall the source.
drv 15y ago
A Google study about hard drive failure indicated that they couldn't find any conclusive relationship between temperature and failure rate. However, they did mention it had some effect at the extreme high end of the temperature spectrum (based on the graph, above 45 °C or so). http://static.googleusercontent.com/external_content/untrusted_dlcp/labs.google.com/en/us/papers/disk_failures.pdf http://static.googleusercontent.com/external_content/untrust...
- barrkel 15y ago
  It doesn't take a huge amount of work to drive a disk over 45 degrees. Using 5K drives helps a lot, since many 7K drives will idle at 35 degrees in a room temperature environment and passive airflow. I've seen actively cooled 7K drives in my ZFS NAS start giving checksum errors while silvering (i.e. recalculating parity etc.) after replacing a bad disk; smartmon reported temperatures of about 60, IIRC. Using an LSI 8-port SATA controller, I was silvering a 4-disk raidz at a rate of perhaps 300MB/sec, and it was making the drives too hot. I had to fall back to the motherboard SATA connectors (and ~150MB/sec array throughput) to keep things cool enough to complete.
  - derobert 15y ago
    If your active-cooled drives are getting that hot, then something is very wrong with either your ambient room temperature (is it 40°C?) or with your active cooling. Active cooling in a reasonable-temperature room keeps even 15K drives well under 40°C at full duty (closer to 30°C, really, according to our monitoring data I just looked at, the room is around 20°C). Keeping drives within 15°C of ambient shouldn't be a huge deal (though it can be noisy).
mrb 15y ago
It's only 400W in 4U (see my other comment in the thread). 100W/U is tiny, easy to cool. That's about two 120V-20A circuits per cabinet, which is very common.

brianwski 15y ago

Backblaze employee here. Yes, the 6 fans move a lot of air through the drives. We monitor every drive in every pod in our datacenter with the smartmontools (/usr/sbin/smartctl) which includes the temperature of the drives. The drives stay well within their recommended operating temperatures as long as the fans are running. In the past, we have detected that several fans failed in one single pod when alarms went off saying some drives were running hot, so the fans are working (normally) and the air is flowing. Side note: personally I think it's really awesome that consumer grade drives come with such detailed internal monitoring for free. I also think it is disappointing that desktop consumer OS's don't monitor these things BY DEFAULT and popup dialogs explaining to you when your drive is exhibiting high temperatures or high data loss!

mturmon 15y ago

Temp does not matter as much as has been thought. See: http://labs.google.com/papers/disk_failures.html http://labs.google.com/papers/disk_failures.html IIRC, up to about 40 C, failure rates are pretty flat as a function of temperature. The paper has some nice graphs. There is one other large-scale study in circulation that showed the same thing, but I don't recall the source.

drv 15y ago

A Google study about hard drive failure indicated that they couldn't find any conclusive relationship between temperature and failure rate. However, they did mention it had some effect at the extreme high end of the temperature spectrum (based on the graph, above 45 °C or so). http://static.googleusercontent.com/external_content/untrusted_dlcp/labs.google.com/en/us/papers/disk_failures.pdf http://static.googleusercontent.com/external_content/untrust...

barrkel 15y ago
It doesn't take a huge amount of work to drive a disk over 45 degrees. Using 5K drives helps a lot, since many 7K drives will idle at 35 degrees in a room temperature environment and passive airflow. I've seen actively cooled 7K drives in my ZFS NAS start giving checksum errors while silvering (i.e. recalculating parity etc.) after replacing a bad disk; smartmon reported temperatures of about 60, IIRC. Using an LSI 8-port SATA controller, I was silvering a 4-disk raidz at a rate of perhaps 300MB/sec, and it was making the drives too hot. I had to fall back to the motherboard SATA connectors (and ~150MB/sec array throughput) to keep things cool enough to complete.
- derobert 15y ago
  If your active-cooled drives are getting that hot, then something is very wrong with either your ambient room temperature (is it 40°C?) or with your active cooling. Active cooling in a reasonable-temperature room keeps even 15K drives well under 40°C at full duty (closer to 30°C, really, according to our monitoring data I just looked at, the room is around 20°C). Keeping drives within 15°C of ambient shouldn't be a huge deal (though it can be noisy).

mrb 15y ago

It's only 400W in 4U (see my other comment in the thread). 100W/U is tiny, easy to cool. That's about two 120V-20A circuits per cabinet, which is very common.