3 ms·

The problem is that once the buffers are full the drive effectively stops accepting commands while it rewrites a SMR region. When that happens depending on wha

The problem is that once the buffers are full the drive effectively stops accepting commands while it rewrites a SMR region. When that happens depending on what your doing, the drive might just get kicked from an array, or your whole system might freeze if some critical page needs to be read back in and it can't get sufficient priority and a command slot. I guess the vast number of people won't ever write more than a buffers full of data, or the drive will never get fragmented sufficiently that even small write operations amplify into entire drive rewrites.....

londons_explore 6y ago
Smarter firmware would detect the case you are discussing, and re-designate a part of the SMR drive to be regular(non-SMR space). For that to work, you need the host OS to be able to see that the drive has just "shrunk". Obviously, you still have data on it while it's shrinking, so the reality is the OS needs to give the drive a list of sectors which can be "shrunk away". One simple approach would be for a daemon to create a massive file filled with "MAGICBYTESMAGICBYTESMAGICBYTES...". As that data is 'written' to the drive, the drive sees that it's the magic bytes, and rather than storing the data, simply marks those sectors as no longer needed. As soon as enough sectors in a row have that designation, reformat them as non-smr, and use as regular (non-smr) disk space. Then, a few hours or days later, the drive can rewrite the data back to be SMR, then tell the daemon, which can remove the magic bytes and delete the files, and your free disk space increases again. The failure mode of this is that your "6TB" SMR disk ends up with a 3TB file filled with MAGICBYTES, and 3TB of your data. But for the same money, you'd only get to store 3TB of data on a non-SMR drive anyway...
- StillBored 6y ago
  That is sorta the reverse of what the QLC drives are doing, in that they treat the cells as a lower density cell which is faster/etc until the drive gets to the point where it needs to use QLC to get the stated capacity. That works though because individual page writes are at least a couple orders of magnitude faster on a QLC than the SMR region on a disk. Shrinking the disk is _MUCH_ harder, there are various enterprise storage arrays which are basically thin provisioned dedupe/etc arrays, and they overwhelmingly just lie about the capacity and throw up big warnings if the physical capacity is being approached. Then depending on which filesystem your running in linux, if they abort writes, there is a good chance the filesystem is damaged (some handle it better than others, and its getting better).
- rlpb 6y ago
  > One simple approach would be for a daemon to create a massive file filled with "MAGICBYTESMAGICBYTESMAGICBYTES...". As that data is 'written' to the drive, the drive sees that it's the magic bytes, and rather than storing the data, simply marks those sectors as no longer needed. As soon as enough sectors in a row have that designation, reformat them as non-smr, and use as regular (non-smr) disk space. You could nearly get there today with drive firmware only, and no special OS support, using TRIM. The OS can already tell the firmware what space it doesn't need, so the firmware could do what you suggest with this information. The only catch (and difference from your proposal) is that there's no way for the firmware to tell the OS that the space is currently not available, so if the OS puts pressure on that space the firmware would still need to block writes while it SMR-izes and frees the space it previously borrowed. In other words, drive firmware could use space freed by TRIM for SMR-ization caching today, with no OS modifications.

londons_explore 6y ago

Smarter firmware would detect the case you are discussing, and re-designate a part of the SMR drive to be regular(non-SMR space). For that to work, you need the host OS to be able to see that the drive has just "shrunk". Obviously, you still have data on it while it's shrinking, so the reality is the OS needs to give the drive a list of sectors which can be "shrunk away". One simple approach would be for a daemon to create a massive file filled with "MAGICBYTESMAGICBYTESMAGICBYTES...". As that data is 'written' to the drive, the drive sees that it's the magic bytes, and rather than storing the data, simply marks those sectors as no longer needed. As soon as enough sectors in a row have that designation, reformat them as non-smr, and use as regular (non-smr) disk space. Then, a few hours or days later, the drive can rewrite the data back to be SMR, then tell the daemon, which can remove the magic bytes and delete the files, and your free disk space increases again. The failure mode of this is that your "6TB" SMR disk ends up with a 3TB file filled with MAGICBYTES, and 3TB of your data. But for the same money, you'd only get to store 3TB of data on a non-SMR drive anyway...

StillBored 6y ago
That is sorta the reverse of what the QLC drives are doing, in that they treat the cells as a lower density cell which is faster/etc until the drive gets to the point where it needs to use QLC to get the stated capacity. That works though because individual page writes are at least a couple orders of magnitude faster on a QLC than the SMR region on a disk. Shrinking the disk is _MUCH_ harder, there are various enterprise storage arrays which are basically thin provisioned dedupe/etc arrays, and they overwhelmingly just lie about the capacity and throw up big warnings if the physical capacity is being approached. Then depending on which filesystem your running in linux, if they abort writes, there is a good chance the filesystem is damaged (some handle it better than others, and its getting better).
rlpb 6y ago
> One simple approach would be for a daemon to create a massive file filled with "MAGICBYTESMAGICBYTESMAGICBYTES...". As that data is 'written' to the drive, the drive sees that it's the magic bytes, and rather than storing the data, simply marks those sectors as no longer needed. As soon as enough sectors in a row have that designation, reformat them as non-smr, and use as regular (non-smr) disk space. You could nearly get there today with drive firmware only, and no special OS support, using TRIM. The OS can already tell the firmware what space it doesn't need, so the firmware could do what you suggest with this information. The only catch (and difference from your proposal) is that there's no way for the firmware to tell the OS that the space is currently not available, so if the OS puts pressure on that space the firmware would still need to block writes while it SMR-izes and frees the space it previously borrowed. In other words, drive firmware could use space freed by TRIM for SMR-ization caching today, with no OS modifications.