4 ms·
Fsync is not synonymous with transactions. Not using fsync anywhere means there's a wide window that can be over 10 minutes long when data can be lost when a se
by codexon 7y ago
Fsync is not synonymous with transactions. Not using fsync anywhere means there's a wide window that can be over 10 minutes long when data can be lost when a server gets an unplanned shutdown.
- valyala 7y agoBtw, it should be easy adding fsync to ClickHouse. For instance VictoriaMetrics uses similar file format as ClickHouse, and it issues proper fsyncs at least every second, so it may lose only the last second of data on unclean shutdown such as OOM, hardware reset or `kill -9`. [1], [2]. [1] https://medium.com/@valyala/wal-usage-looks-broken-in-modern-time-series-databases-b62a627ab704 https://medium.com/@valyala/wal-usage-looks-broken-in-modern... [2] https://medium.com/@valyala/how-victoriametrics-makes-instant-snapshots-for-multi-terabyte-time-series-data-e1f3fb0e0282 https://medium.com/@valyala/how-victoriametrics-makes-instan...