4 ms·

Local disk I/O is no longer the bottleneck on modern systems: https://benhoyt.com/writings/io-is-no-longer-the-bottleneck/ https://benhoyt.com/writings/io-is-no

Local disk I/O is no longer the bottleneck on modern systems: https://benhoyt.com/writings/io-is-no-longer-the-bottleneck/ https://benhoyt.com/writings/io-is-no-longer-the-bottleneck/ In addition, the official 1BRC explicitly evaluated results on a RAM disk to avoid I/O speed entirely: https://github.com/gunnarmorling/1brc?tab=readme-ov-file#evaluating-results https://github.com/gunnarmorling/1brc?tab=readme-ov-file#eva... "Programs are run from a RAM disk (i.o. the IO overhead for loading the file from disk is not relevant)"

davidmurdoch 3y ago
I was surprised by this recently when optimizing a build process that uses an intermediate write-to-disk step. I replaced the intermediate filesystem API with an in-memory one and it was not measurably faster. Not even by a single millisecond.
- 10000truths 3y ago
  How much data were you writing? If you don't fill the OS's page cache and the SSD controller's DRAM cache, and you're not blocking on fsync() or O_DIRECT or some other explicit flushing mechanism, then you're not going to see much of a difference in throughput.
  - davidmurdoch 3y ago
    Only a few MB written then read, so that is a likely explanation.
deleted 3y ago
[deleted]
stavros 3y ago
Ahh, thanks, I didn't know about the ramdisk. Very interesting about I/O not being the bottleneck, though.
mike_hearn 3y ago
Unfortunately there are still cases where local disk I/O can be a serious bottleneck that you do have to be aware of: 1. Cloud VMs sometimes have very slow access even to devices advertised as local disks, depending on which cloud 2. Windows The latter often catches people out because they develop intuitions about file I/O speed on Linux, which is extremely fast, and then their app runs 10x slower on Windows due to opening file handles being much slower (and sometimes they don't sign their software and virus scanners make things much slower again).
- NikkiA 3y ago
  Apparently, the cause of the long standing windows disk IO problem was discovered a month or so ago, and MS were said to be working on a fix. Whether it'll be constrained to Win 11 is yet to be seen.
  - dr_zoidberg 3y ago
    That's interesting. A project at work is affected by Windows slow open() calls (wrt to Linux/Mac) but we haven't found a strong solution rather than "avoid open() as much as you can".
    - tomnipotent 3y ago
      It's likely Windows Defender, which blocks on read I/O to scan files. Verify by adding a folder to its exclusion list, though this isn't helpful if it's a desktop app. The difference is most noticeable when you're reading many small files.
  - mike_hearn 3y ago
    Really? Where did you hear that? Conventional wisdom is based on a post by an MS employee years ago that described the performance issues as architectural. MS does have something called "dev drive" in Win11. Dev Drive is what ReFS turned into and is an entirely different filesystem that isn't NTFS, which is better optimized for UNIX-style access patterns. The idea is that core Windows remains slow, but developers (the only people who care about file IO performance apparently) can format another partition and store their source/builds there.

davidmurdoch 3y ago

I was surprised by this recently when optimizing a build process that uses an intermediate write-to-disk step. I replaced the intermediate filesystem API with an in-memory one and it was not measurably faster. Not even by a single millisecond.

10000truths 3y ago
How much data were you writing? If you don't fill the OS's page cache and the SSD controller's DRAM cache, and you're not blocking on fsync() or O_DIRECT or some other explicit flushing mechanism, then you're not going to see much of a difference in throughput.
- davidmurdoch 3y ago
  Only a few MB written then read, so that is a likely explanation.

deleted 3y ago

[deleted]

stavros 3y ago

Ahh, thanks, I didn't know about the ramdisk. Very interesting about I/O not being the bottleneck, though.

mike_hearn 3y ago

Unfortunately there are still cases where local disk I/O can be a serious bottleneck that you do have to be aware of: 1. Cloud VMs sometimes have very slow access even to devices advertised as local disks, depending on which cloud 2. Windows The latter often catches people out because they develop intuitions about file I/O speed on Linux, which is extremely fast, and then their app runs 10x slower on Windows due to opening file handles being much slower (and sometimes they don't sign their software and virus scanners make things much slower again).

NikkiA 3y ago
Apparently, the cause of the long standing windows disk IO problem was discovered a month or so ago, and MS were said to be working on a fix. Whether it'll be constrained to Win 11 is yet to be seen.
- dr_zoidberg 3y ago
  That's interesting. A project at work is affected by Windows slow open() calls (wrt to Linux/Mac) but we haven't found a strong solution rather than "avoid open() as much as you can".
  - tomnipotent 3y ago
    It's likely Windows Defender, which blocks on read I/O to scan files. Verify by adding a folder to its exclusion list, though this isn't helpful if it's a desktop app. The difference is most noticeable when you're reading many small files.
- mike_hearn 3y ago
  Really? Where did you hear that? Conventional wisdom is based on a post by an MS employee years ago that described the performance issues as architectural. MS does have something called "dev drive" in Win11. Dev Drive is what ReFS turned into and is an entirely different filesystem that isn't NTFS, which is better optimized for UNIX-style access patterns. The idea is that core Windows remains slow, but developers (the only people who care about file IO performance apparently) can format another partition and store their source/builds there.