5 ms·

I'm still of the opinion that even with SSD/NVMe, the random reads/writes are still very slow.

marginalia_nu 3y ago
With SSDs it's more like a discount on sequential/adjacent reads within a physical block that is of a size that varies with model and manufacturer.
exabrial 3y ago
In my experience, depends on the SSD. Consumer grade ones appear really fast because they have a DRAM cache, and as soon as you start having a bunch of cache-misses you realize how slow they actually are. We got a bunch of enterprise grade Samsung SSDs and there'a large difference in sustained I/O. It's not "instant" by any means, but it there are other things slowing I/O.
- zamadatix 3y ago
  Only the very best drives hit >200 MB/s in 4k write (even with DRAM as cache) and for 4k mixed not even Optane drives get that high.
  - wtallis 3y ago
    Did you forget to stipulate that you're only talking about workloads that issue IO requests serially with zero parallelism?
    - zamadatix 3y ago
      No, I just didn't pick QD32T8 to pump the numbers either. Even in such workloads the gap to sequential speed is a decent ratio, just not painfully so as in most other mid to singular workloads.
willis936 3y ago
That's why all of my minecraft servers are on RAM drives.
wmf 3y ago
Relative to what?
95014_refugee 3y ago
This obviously depends substantially on the implementation, but (especially for standalone devices) there tends to be metadata access associated with translating a given LBA to a physical page. This brings cache locality of reference into the picture, and any workload that is "random enough" such that the metadata working set exceeds the cache will suffer...
thelastparadise 3y ago
I don't understand why so much emphasis is put on optimizing for sequential reads and writes. Sequential IO is not something that happens in the real world except in rare circumstances. In most applications IO is random, even if it's a log file.
- acscott 3y ago
  When writing to a log file, isn't that sequential write?
- dmitrygr 3y ago
  > In most applications IO is random, even if it's a log file. I am all ears... how are log writes random?
  - JdeBP 3y ago
    I think that I see what thelastparadise is getting at. At base, most logfile output is a read-modify-write of usually no more than the final block of the file, whereas the sequential writes that are generally optimized for by filesystem and disc drivers are multiple-block writes sent as single output requests. The twain are similar, but aren't exactly the same. Optimize for the cases where you are writing multiple blocks in a chunk, with contiguous free block allocation policies and write behind and whatnot, and you haven't in reality quite optimized for the case where what you're actually doing is overwriting the tail block in the file repeatedly, oftentimes changing as few as mere 10s of bytes in that block with each read-modify-write request. (And yes, logging mechanisms often do like to ensure that logs are synchronously flushed to disc line by line, effectively and intentionally defeating write caching and write behind optimizations.)
    - justsomehnguy 3y ago
      I would add what it is very rare to have 100MiB+ writes to a log file and anything less would be concurrent with the other operations, which would make 'sequential' here a moot point. > At base, most logfile output is a read-modify-write It's worse, actually, as you never can tell if this would be RMW to the same block or to the other free block. In the former you waste the writes for the whole blocks. In the latter you can hit a write amplification despite never have been writing any meaningful amount of data.

With SSDs it's more like a discount on sequential/adjacent reads within a physical block that is of a size that varies with model and manufacturer.

exabrial 3y ago

In my experience, depends on the SSD. Consumer grade ones appear really fast because they have a DRAM cache, and as soon as you start having a bunch of cache-misses you realize how slow they actually are. We got a bunch of enterprise grade Samsung SSDs and there'a large difference in sustained I/O. It's not "instant" by any means, but it there are other things slowing I/O.

zamadatix 3y ago
Only the very best drives hit >200 MB/s in 4k write (even with DRAM as cache) and for 4k mixed not even Optane drives get that high.
- wtallis 3y ago
  Did you forget to stipulate that you're only talking about workloads that issue IO requests serially with zero parallelism?
  - zamadatix 3y ago
    No, I just didn't pick QD32T8 to pump the numbers either. Even in such workloads the gap to sequential speed is a decent ratio, just not painfully so as in most other mid to singular workloads.

willis936 3y ago

That's why all of my minecraft servers are on RAM drives.

wmf 3y ago

Relative to what?

95014_refugee 3y ago

This obviously depends substantially on the implementation, but (especially for standalone devices) there tends to be metadata access associated with translating a given LBA to a physical page. This brings cache locality of reference into the picture, and any workload that is "random enough" such that the metadata working set exceeds the cache will suffer...

thelastparadise 3y ago

I don't understand why so much emphasis is put on optimizing for sequential reads and writes. Sequential IO is not something that happens in the real world except in rare circumstances. In most applications IO is random, even if it's a log file.

acscott 3y ago
When writing to a log file, isn't that sequential write?
dmitrygr 3y ago
> In most applications IO is random, even if it's a log file. I am all ears... how are log writes random?
- JdeBP 3y ago
  I think that I see what thelastparadise is getting at. At base, most logfile output is a read-modify-write of usually no more than the final block of the file, whereas the sequential writes that are generally optimized for by filesystem and disc drivers are multiple-block writes sent as single output requests. The twain are similar, but aren't exactly the same. Optimize for the cases where you are writing multiple blocks in a chunk, with contiguous free block allocation policies and write behind and whatnot, and you haven't in reality quite optimized for the case where what you're actually doing is overwriting the tail block in the file repeatedly, oftentimes changing as few as mere 10s of bytes in that block with each read-modify-write request. (And yes, logging mechanisms often do like to ensure that logs are synchronously flushed to disc line by line, effectively and intentionally defeating write caching and write behind optimizations.)
  - justsomehnguy 3y ago
    I would add what it is very rare to have 100MiB+ writes to a log file and anything less would be concurrent with the other operations, which would make 'sequential' here a moot point. > At base, most logfile output is a read-modify-write It's worse, actually, as you never can tell if this would be RMW to the same block or to the other free block. In the former you waste the writes for the whole blocks. In the latter you can hit a write amplification despite never have been writing any meaningful amount of data.