3 ms·

Flash controllers provide an abstraction layer that should turn any write workload into a sequential one, freeing software developers from having to care about

Flash controllers provide an abstraction layer that should turn any write workload into a sequential one, freeing software developers from having to care about it.

wtallis 3y ago
That abstraction layer cannot always avoid creating write amplification when you send it writes that are too small.
- dsr_ 3y ago
  Which is why having a nice chunk of RAM on the disk itself is useful. 2x write queue size x block size is nice.
- xenadu02 3y ago
  It can but it involves additional complexity in the controller. Some definitely do a much better job than others. Most controllers handle the block to page mapping problem well. They just mark the old block in the map as dead and write a new block on a new page. GC will later erase pages, hopefully prioritizing pages with the most dead blocks (subject to wear-leveling concerns). But that same concept can be extended to partial block writes. It complicates the read process but there is no reason the controller can't coalesce multiple partial block writes into special partial update blocks and update the map accordingly. Basically write a record saying byte range (X,Y) was overwritten with just those bytes - no need to read the old block, merge the change, and write the whole block back out. Obviously you need to handle things like read chain thresholds (too many partially overlapping updates). Let the GC handle merging the partial updates into the full block during idle periods. Again prioritize doing this for blocks on pages that are going to be erased - you had to rewrite those blocks anyway, turning what would be write amplification into a "free" write. Like I said a lot of controller firmware doesn't bother but that doesn't mean it is impossible or unprofitable to do.
  - wtallis 3y ago
    I didn't say it was impossible, I said it is not always possible. Even the most advanced SSD firmware will cause write amplification for some pathological workloads.
    - xenadu02 3y ago
      That's fair. For many systems you can come up with a pathological workload that defies any existing optimizations.

Flash controllers provide an abstraction layer that should turn any write workload into a sequential one, freeing software developers from having to care about it.

wtallis 3y ago

That abstraction layer cannot always avoid creating write amplification when you send it writes that are too small.

dsr_ 3y ago
Which is why having a nice chunk of RAM on the disk itself is useful. 2x write queue size x block size is nice.
xenadu02 3y ago
It can but it involves additional complexity in the controller. Some definitely do a much better job than others. Most controllers handle the block to page mapping problem well. They just mark the old block in the map as dead and write a new block on a new page. GC will later erase pages, hopefully prioritizing pages with the most dead blocks (subject to wear-leveling concerns). But that same concept can be extended to partial block writes. It complicates the read process but there is no reason the controller can't coalesce multiple partial block writes into special partial update blocks and update the map accordingly. Basically write a record saying byte range (X,Y) was overwritten with just those bytes - no need to read the old block, merge the change, and write the whole block back out. Obviously you need to handle things like read chain thresholds (too many partially overlapping updates). Let the GC handle merging the partial updates into the full block during idle periods. Again prioritize doing this for blocks on pages that are going to be erased - you had to rewrite those blocks anyway, turning what would be write amplification into a "free" write. Like I said a lot of controller firmware doesn't bother but that doesn't mean it is impossible or unprofitable to do.
- wtallis 3y ago
  I didn't say it was impossible, I said it is not always possible. Even the most advanced SSD firmware will cause write amplification for some pathological workloads.
  - xenadu02 3y ago
    That's fair. For many systems you can come up with a pathological workload that defies any existing optimizations.