3 ms·

I can see how that'd be very effective for a cache, but I'm not at all seeing how it'd work for a database. You'd need to touch all live data to copy it forward

I can see how that'd be very effective for a cache, but I'm not at all seeing how it'd work for a database. You'd need to touch all live data to copy it forward and keep track of the low water mark before you truncate the prefix of the log. That'd have horrific write amplification.

bob1029 5y ago
> You'd need to touch all live data to copy it forward This is already happening as a natural part of the utilization patterns. Just looking at something in a splay tree will cause it to be relocated to the root (the most live part of the log). That is the dynamic part of the data structure. You are doing a huge part of the GC effort simply by reading the database. There are clearly applications where this would not be a good fit (i.e. data warehousing where you want to keep old shit on purpose). Even with the natural cleanup mechanism, you may still need to save stuff you haven't accessed in a while. Linked list nodes built on top being an example of an area that would need careful attention to avoid chopping up data structures unexpectedly. >That'd have horrific write amplification. This is precisely why I recommend batching as part of the approach in my original post above. With batching, you can take a total rewrite of all nodes in a linked list and squeeze them into a single I/O operation that flows into exactly the # of storage blocks required. In implementations without batching, you are talking about block I/O per linked list node that needs updating. Clearly this is a total non-starter for drive lifetime & performance.