4 ms·

"Writes to the WAL are purely sequential and writes to long-term storage are purely sequential." Is that simply a typo or am I missing something? Shouldn't it

• jasonwatkinspdx 3y ago
  No. In WAL based systems new writes are appended to the write ahead log, and the dirty database pages held in ram. Periodically, a checkpoint process writes out all those dirty pages to long term storage, then truncates the write ahead log. So both these writes can be sequential. This is one of the reasons databases have the performance they do vs just using lots of little files like a database naively.
  • eatonphil 3y ago
    > So both these writes can be sequential. In the worst case though, the long-term storage can still have random writes though no (for btrees anyway)? I figure LSM tree writes can always be sequential, even during compaction. But btrees can only do solely sequential writes if the entire tree needs to be rewritten. And you don't want to always do that if only parts of the tree need to be updated?
    • saltcured 3y ago
      I would take it as a simplification to really mean "sequential enough" when writing out the reorganized copy to the long-term storage. With all the different layers in modern storage, I think you can get asymptotically close to the sequential rate once you start hitting some of these other multi-block granularities, i.e. around the block size for flash erasure, encryption, redundancy coding, etc. These don't have to be that big, i.e. several megabytes.
    • jasonwatkinspdx 3y ago
      The dirty pages can be written out in physical address order no matter what indexing structure is used. What real databases do involves a bunch of complicated choices, but basically there's ways you can make checkpoints incremental such that it's nearly always large sequential writes.
      • eatonphil 3y ago
        Sure it can be written in order but it still can involve non-consecutive writes which I assumed is closer to random writes from the disk's perspective.