4 ms·

A few things that can make walking HDDs by physical layout even faster: * also do it when walking directories! on ext4 you can FIEMAP the directories while wal

A few things that can make walking HDDs by physical layout even faster: * also do it when walking directories! on ext4 you can FIEMAP the directories while walking them. * keep a readahead buffer that spans multiple files, that way you can keep X megabytes in flight even when the files are small * do readaheads on directories (can only be done as root for ext4) * do drop-behind to prevent page cache thrashing * even when writing synchronous code you can use io_uring to batch syscalls such as statx(), open() and fadvise() I wrote crates for some of those points: https://crates.io/crates/platter-walk https://crates.io/crates/platter-walk https://crates.io/crates/reapfrog https://crates.io/crates/reapfrog

rbanffy 6y ago
If you keep many accesses in flight simultaneously, you give the OS the chance to reorder your operations to minimize seek times. This is one of the reasons why async disk IO is cool. Even on an SSD, if you group your accesses, you’ll end up reading a flash block only once for all files that live on it, and the same will happen with writing.
- dataflow 6y ago
  Is this one of those things that hypothetically could happen in the future or are there kernels that actually do this right now?
  - wiml 6y ago
    Reordering disk io requests to minimize seek time and rotational latency has been a standard OS behavior since the 1970s, at least. Whether it's still effective in an SSD+io_uring world I don't know but I'm curious.
    - astrange 6y ago
      If you can issue multiple requests at once then they can go into the SSD controller's queues where it can reorder them, so it is beneficial. As usual with parallelism, it's hard to find if you look for it bottom-up, so running multiple processes is a better way to take advantage of it.
  - doctor_eval 6y ago
    I learned about “elevator algorithms” in CS in the early 90s, well before Linux was a thing... I think it was common even then.
- tinus_hn 6y ago
  Doesn’t the kernel read a whole flash block regardless of how small a part you request? Seems like the smart thing to do.
  - fulafel 6y ago
    Flash block size generally affects only writing (and things related to that happen on the on-device FTL, not in the kernel, except for embedded raw flash devices that require specialized filesystems). There is a general readahead in Linux though, see eg blkdev(8). For many use cases the small randomish iops rate is the bottleneck (think database index), and because of the small latencies in flash storage you often want to keep the readahead modest absent some detected pattern of sequential io.

rbanffy 6y ago

If you keep many accesses in flight simultaneously, you give the OS the chance to reorder your operations to minimize seek times. This is one of the reasons why async disk IO is cool. Even on an SSD, if you group your accesses, you’ll end up reading a flash block only once for all files that live on it, and the same will happen with writing.

dataflow 6y ago
Is this one of those things that hypothetically could happen in the future or are there kernels that actually do this right now?
- wiml 6y ago
  Reordering disk io requests to minimize seek time and rotational latency has been a standard OS behavior since the 1970s, at least. Whether it's still effective in an SSD+io_uring world I don't know but I'm curious.
  - astrange 6y ago
    If you can issue multiple requests at once then they can go into the SSD controller's queues where it can reorder them, so it is beneficial. As usual with parallelism, it's hard to find if you look for it bottom-up, so running multiple processes is a better way to take advantage of it.
- doctor_eval 6y ago
  I learned about “elevator algorithms” in CS in the early 90s, well before Linux was a thing... I think it was common even then.
tinus_hn 6y ago
Doesn’t the kernel read a whole flash block regardless of how small a part you request? Seems like the smart thing to do.
- fulafel 6y ago
  Flash block size generally affects only writing (and things related to that happen on the on-device FTL, not in the kernel, except for embedded raw flash devices that require specialized filesystems). There is a general readahead in Linux though, see eg blkdev(8). For many use cases the small randomish iops rate is the bottleneck (think database index), and because of the small latencies in flash storage you often want to keep the readahead modest absent some detected pattern of sequential io.