3 ms·

Oh yeah, that I can't deny. The peak throughput numbers used in marketing are definitely questionable at times. Kinda makes sense considering SSDs and HDDs are

by jchw 3y ago

Oh yeah, that I can't deny. The peak throughput numbers used in marketing are definitely questionable at times. Kinda makes sense considering SSDs and HDDs are basically computers of their own that give you some abstraction over the actual raw storage media. You could compare memcpy to sequential write speed or something, but it'd definitely not tell the full story.

marginalia_nu 3y ago
It can be a pretty real problem. I encountered a problem where I needed to construct a file with out of order writes 64 bits long. This works fine for small files, but this file was dozens of gigabytes. Turns out if you do this the naive way on a consumer disk, it will take weeks and it will wear out the disk incredibly quickly, because all the buffers will fill up in the worst possible way and the disk ends up with worst case write amplification for every write. I ended up creating a bunch of smaller files containing essentially assembly instructions, of the form "put X at offset Y", and grouping adjacent write instructions in ~100Mb chunks. This is just sequential writes, which the OS will gladly help buffer and the disk likes. Then I go over the files one by one and evaluate the instructions. Now since the "random" writes are concentrated to a relatively small area on disk, the buffering works well. Despite writing 2.5x as much data (and reading the data an additional time), this makes the write operation take like an hour instead of several weeks.
- gavinray 3y ago
  > ...but this file was dozens of gigabytes. Turns out if you do this the naive way on a consumer disk, it will take weeks and it will wear out the disk incredibly quickly This is what C's "fallocate(2)" with "FALLOC_FL_ZERO_RANGE" or "FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE" flags are for, along with "pwrite()"
  - marginalia_nu 3y ago
    It's mostly a SSD problem. They really do not like sustained loads of random small IO, since their write cycles have much larger granularity.

by jchw 3y ago

marginalia_nu 3y ago

It can be a pretty real problem. I encountered a problem where I needed to construct a file with out of order writes 64 bits long. This works fine for small files, but this file was dozens of gigabytes. Turns out if you do this the naive way on a consumer disk, it will take weeks and it will wear out the disk incredibly quickly, because all the buffers will fill up in the worst possible way and the disk ends up with worst case write amplification for every write. I ended up creating a bunch of smaller files containing essentially assembly instructions, of the form "put X at offset Y", and grouping adjacent write instructions in ~100Mb chunks. This is just sequential writes, which the OS will gladly help buffer and the disk likes. Then I go over the files one by one and evaluate the instructions. Now since the "random" writes are concentrated to a relatively small area on disk, the buffering works well. Despite writing 2.5x as much data (and reading the data an additional time), this makes the write operation take like an hour instead of several weeks.

gavinray 3y ago
> ...but this file was dozens of gigabytes. Turns out if you do this the naive way on a consumer disk, it will take weeks and it will wear out the disk incredibly quickly This is what C's "fallocate(2)" with "FALLOC_FL_ZERO_RANGE" or "FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE" flags are for, along with "pwrite()"
- marginalia_nu 3y ago
  It's mostly a SSD problem. They really do not like sustained loads of random small IO, since their write cycles have much larger granularity.