3 ms·

File system journaling typically refers to a log that helps ensure filesystem consistency by recording attempted operations before they happen, but the system s

by tveita 7y ago

File system journaling typically refers to a log that helps ensure filesystem consistency by recording attempted operations before they happen, but the system still has to do all the same random writes to the disk afterwards. Some file systems (notably ZFS and Btrfs) use a copy-on-write approach that should in theory not require many random writes, but I don't think any of them are adapted for the large blocks you need to write to for SMR. (256MB?) It should be possible to make a filesystem with drastically better performance on SMR drives given the right low-level access, probably based on something more like a log-structured merge-tree, like e.g. LevelDB. One interesting example I found was Dropbox which seems to store data on SMR in big immutable chunks using raw Zoned Block Command access. https://dropbox.tech/infrastructure/smr-what-we-learned-in-our-first-year https://dropbox.tech/infrastructure/smr-what-we-learned-in-o...

klodolph 7y ago
> It should be possible to make a filesystem with drastically better performance on SMR drives given the right low-level access, probably based on something more like a log-structured merge-tree, like e.g. LevelDB. The filesystem would need to do periodic compaction. You would necessarily encounter situations where a write to “allocated” data would fail because the disk is out of space, which is a new, surprising error that would undoubtedly confuse some user-space programs.
- eru 7y ago
  You could go around that error by claiming a lower capacity. All file systems take up a bit of space to store their own metadata etc. So users are already used to 'lose' a bit of space on formatting.
  - klodolph 7y ago
    That wouldn’t fix it. As long as programs can write faster than the compaction algorithm frees space, you can run out of space. The only solutions I see here are to block IO until the compaction algorithm frees space, or return ENOSPC. Both options violate assumptions that real-world programs make.

by tveita 7y ago

klodolph 7y ago

> It should be possible to make a filesystem with drastically better performance on SMR drives given the right low-level access, probably based on something more like a log-structured merge-tree, like e.g. LevelDB. The filesystem would need to do periodic compaction. You would necessarily encounter situations where a write to “allocated” data would fail because the disk is out of space, which is a new, surprising error that would undoubtedly confuse some user-space programs.

eru 7y ago
You could go around that error by claiming a lower capacity. All file systems take up a bit of space to store their own metadata etc. So users are already used to 'lose' a bit of space on formatting.
- klodolph 7y ago
  That wouldn’t fix it. As long as programs can write faster than the compaction algorithm frees space, you can run out of space. The only solutions I see here are to block IO until the compaction algorithm frees space, or return ENOSPC. Both options violate assumptions that real-world programs make.