4 ms·

If you just built your array, you wouldn't encounter this issue yet. The problem would come (assuming you're using ZFS) when your NAS is full of data and one of

If you just built your array, you wouldn't encounter this issue yet. The problem would come (assuming you're using ZFS) when your NAS is full of data and one of your disks fails, and you try to replace it with another SMR drive. The drive would be rejected before it could be added to the array.

dannyw 6y ago
As benchmarked in the article, SMR drives rebuild Linux RAID fine because it's just a sequential data overwrite. The problem occurs when you are dealing with sequential files with small sector sizes, and non-sequential files, where performance is abysmal: 93% slower than CMR as benchmarked by servethehome and Ars.
- zozbot234 6y ago
  The problem is not just the slow performance, but the fact that these drives can be dropped from the array when rebuilding - RAID implementations including the one in ZFS treat abysmal performance as damage and route around it. The performance problem could be addressed simply by raising the number of drives in the array (RAID5 to RAID6 or more).
  - ezoe 6y ago
    Isn't that a problem of ZFS? I don't mind SMR HDD if it was labeled as such and warn about it's weird performance traits.
    - ubercow13 6y ago
      This whole issue was originally that Western Digital was trying to hide which HDDs were SMR. ZFS can't be changed to have special behaviour for SMR drives if the vendors try to hide the fact they are SMR, even from software. It wouldn't be able to identify them.
      - ezoe 6y ago
        Well, that's bad.
- fomine3 6y ago
  Rebuilding ZFS array is normally faster than traditional array because it won't sync whole disk but sync only used blocks. This PR looks like adding support for traditional sequential rebuild way like Linux MD that suitable for SMR disks. https://github.com/openzfs/zfs/pull/10349 https://github.com/openzfs/zfs/pull/10349
- StillBored 6y ago
  Their test is completely invalid, they only used a small portion of the disk in a sequential manner. Likely the "cache" portion of the array is a significant portion of the partition they created. Further, I'm not aware of a single raid system built in the last 20 years that doesn't do online rebuild. This means its only sequential IO as long as the RAID controller isn't servicing other IO requests. Futher it also assumes that your running a dead simple RAID system that is RAIDing the entire array rather than doing the RAID on top of a LVM/thin provisioning like setup (this is basically ZFS's problem). Frankly, I would be willing to bet that its possible to tweak a benchmark like iometer to use a sufficiently large part of the disk with a 50% random RW workload and cause quite a number of RAID controllers to kick the drives for write timeouts. Some percentage of them might even kick multiple disks at the same time resulting in complete array loss.
exikyut 6y ago
> The drive would be rejected before it could be added to the array. Wait, what? Why? Because it's too slow?
- ubercow13 6y ago
  Yes something like that. There were reports of disks being kicked out of arrays (at least for ZFS I think) due to them taking so long to respond that the filesystem decided they were dead.
  - blackflame7000 6y ago
    If the array goes into the faulted state, you can just reboot and if the drive that was kicked out responds it will be brought back into the array and a scrub/resilvering will begin. Now if you have a failing drive, you can coax it back to online status by powering off your computer and letting it rest and a certain percentage of the scrub has completed. I've used ZFS with seagate green drives that were definitely not meant for that purpose but since they would be 99% read-only drives, once I got the data copied (which involved drives being kicked for poor performance) they perform just fine. They could not cope with continuous I/O however and would start to error out during heavy writes or long scrubs.

dannyw 6y ago

As benchmarked in the article, SMR drives rebuild Linux RAID fine because it's just a sequential data overwrite. The problem occurs when you are dealing with sequential files with small sector sizes, and non-sequential files, where performance is abysmal: 93% slower than CMR as benchmarked by servethehome and Ars.

zozbot234 6y ago
The problem is not just the slow performance, but the fact that these drives can be dropped from the array when rebuilding - RAID implementations including the one in ZFS treat abysmal performance as damage and route around it. The performance problem could be addressed simply by raising the number of drives in the array (RAID5 to RAID6 or more).
- ezoe 6y ago
  Isn't that a problem of ZFS? I don't mind SMR HDD if it was labeled as such and warn about it's weird performance traits.
  - ubercow13 6y ago
    This whole issue was originally that Western Digital was trying to hide which HDDs were SMR. ZFS can't be changed to have special behaviour for SMR drives if the vendors try to hide the fact they are SMR, even from software. It wouldn't be able to identify them.
    - ezoe 6y ago
      Well, that's bad.
fomine3 6y ago
Rebuilding ZFS array is normally faster than traditional array because it won't sync whole disk but sync only used blocks. This PR looks like adding support for traditional sequential rebuild way like Linux MD that suitable for SMR disks. https://github.com/openzfs/zfs/pull/10349 https://github.com/openzfs/zfs/pull/10349
StillBored 6y ago
Their test is completely invalid, they only used a small portion of the disk in a sequential manner. Likely the "cache" portion of the array is a significant portion of the partition they created. Further, I'm not aware of a single raid system built in the last 20 years that doesn't do online rebuild. This means its only sequential IO as long as the RAID controller isn't servicing other IO requests. Futher it also assumes that your running a dead simple RAID system that is RAIDing the entire array rather than doing the RAID on top of a LVM/thin provisioning like setup (this is basically ZFS's problem). Frankly, I would be willing to bet that its possible to tweak a benchmark like iometer to use a sufficiently large part of the disk with a 50% random RW workload and cause quite a number of RAID controllers to kick the drives for write timeouts. Some percentage of them might even kick multiple disks at the same time resulting in complete array loss.

exikyut 6y ago

> The drive would be rejected before it could be added to the array. Wait, what? Why? Because it's too slow?

ubercow13 6y ago
Yes something like that. There were reports of disks being kicked out of arrays (at least for ZFS I think) due to them taking so long to respond that the filesystem decided they were dead.
- blackflame7000 6y ago
  If the array goes into the faulted state, you can just reboot and if the drive that was kicked out responds it will be brought back into the array and a scrub/resilvering will begin. Now if you have a failing drive, you can coax it back to online status by powering off your computer and letting it rest and a certain percentage of the scrub has completed. I've used ZFS with seagate green drives that were definitely not meant for that purpose but since they would be 99% read-only drives, once I got the data copied (which involved drives being kicked for poor performance) they perform just fine. They could not cope with continuous I/O however and would start to error out during heavy writes or long scrubs.