2 ms·

I’m not talking about data loss if the host crashes. I’m talking about a much broader sense of correctness. Imagine an RPC server that uses persistent connecti

by amluto 1y ago

I’m not talking about data loss if the host crashes. I’m talking about a much broader sense of correctness. Imagine an RPC server that uses persistent connections. If you reply using a buffered writer and forget to flush, then your tail latency blows up, possibly to infinity. It’s very easy to imagine situations involving multiple threads or processes that simply don’t work if buffers aren’t flushed on time. Imagine a program that is intended to crash on error but writes a log message first. If it buffers and doesn’t flush, then the log message will almost always get lost, whereas if the writer is unbuffered or is flushed, then the message will usually not get lost.

josephg 1y ago
> I’m not talking about data loss if the host crashes. I’m talking about a much broader sense of correctness Then why aren’t you talking about data loss if the host crashes? I consider that in the same bucket as your other examples. If you write a log and then the system crashes before it’s been written out to disk, the log message is lost. If a database issues two writes then the host crashes, what has been written to disk? It could be one write, both, neither, or a skewed write. If the database isn’t extremely careful, that crash could totally corrupt the database files. Correctness matters. Writes are sent to disk (or sent over the network) some time between when you call write and when your call to flush returns. Using a buffered writer API doesn’t (shouldn’t) change that.