3 ms·
I was expecting this to say “S3 append is buggy” or something. But really it just is saying “my specific application cannot replace making new files with append
by jitl 2y ago
I was expecting this to say “S3 append is buggy” or something. But really it just is saying “my specific application cannot replace making new files with append for a very specific reason”
The issue cited is that using append instead of writing a new file means losing put-if-absent consistency guarantee.
So… just don’t do that? Make the rule that a leader can append to a file it already wrote, but new process or newly elected leader must always start the next file in the sequence, and don’t design your system so that the append use-case needs at-most-once delivery or something. You can put a causal clock in each append chunk header or something if you want to stay as close as possible to the original system design.
Like, I will happily write pages of a WAL using append with no issues as long as my system isn’t multi-leader. ¯\_(ツ)_/¯