4 ms·

> DbOptions::durable() > Appended to the WAL without a per-write sync So… it’s not durable? Durable doesn’t mean “survives a process restart”, it means “durab

> DbOptions::durable() > Appended to the WAL without a per-write sync So… it’s not durable? Durable doesn’t mean “survives a process restart”, it means “durably saved to persistent storage”. For example, this “durable” mode wouldn’t survive power loss.

stingraycharles 1mo ago
Yeah this should be benchmarked against other systems that have flush() disabled. mmap is nice but it doesn’t support durable semantics in the way that we usually mean with databases. if a write is acknowledged it should not be forgotten, which is not what this is.
- rgbimbochamp 1mo ago
  You're right, that mode provides process crash recovery, not power-loss durability. The benchmark compares it against fjall’s equivalent buffered-WAL mode.
  - a2ff6eeb0 1mo ago
    If that's your design constraint, couldn't you speed it up by getting rid of the WAL?
    - oneshadab 1mo ago
      You'd lose durability against process crashes. If your system has a reasonable tolerance for power failure (multi-az multi-cloud), this can provide much better throughput
      - t098i3 1mo ago
        Indeed, a common enough pattern for etcd is to run it backed by a RAMdisk and have multi-az availability + periodic backups + tolerance at a business level to be OK losing some recent data.
  - deathanatos 1mo ago
    Word choice matters. Defaults matter. People will go "well it says durable right here" and while arguably, yes, they should RTFM, it would still be great if tool-builders did not set the shotgun's default state to State::AT_FOOT. It would be nice if every paragraph of technical writing that I have to do need not be burdened by a thousand asterisks of "durable in this context means something other than durable".
    - carlmr 1mo ago
      >arguably, yes, they should RTFM Agreed. Good design is when the things do what you expect them to do without reading the manual, don't reuse wording with other meaning in the wrong way. That way if you do encounter nee wording, you know you should read the manual.
      - gryfft 1mo ago
        https://en.wikipedia.org/wiki/Principle_of_least_astonishment https://en.wikipedia.org/wiki/Principle_of_least_astonishmen...
- muvlon 1mo ago
  As of a couple years ago, mmap actually has a MAP_SYNC flag that makes it durable in the DB sense. The caveat is that it requires DAX on the file and so comes with a whole bunch of restrictions w.r.t. filesystem, storage media and even CPU architecture.
deepsun 1mo ago
Are we back to MongoDB -- no fsync() but webscale speed?
- rgbimbochamp 1mo ago
  Nope - https://github.com/kingroryg/turbokv#durability-presets https://github.com/kingroryg/turbokv#durability-presets
- HatchedLake721 1mo ago
  webscale… https://youtu.be/b2F-DItXtZs?is=wFhGujRHma3b7VLp https://youtu.be/b2F-DItXtZs?is=wFhGujRHma3b7VLp
  - whirlwin 1mo ago
    This brings back memories!
- re-thc 1mo ago
  Agent scale!
procaryote 1mo ago
Pretty much... paranoid() seems to be the real durable() which isn't a great look for a database project. Being able to recover a db without corruption beyound losing the last few writes is a pretty useful feature, and buys a lot of performance, but it would be better to label that clearly, as a reasonable expectation on the durable() preset would be for it to be Durable.
insanitybit 1mo ago
This is surprisingly common, from what I can tell.
nijave 1mo ago
I give a little leeway to distributed systems that replicate and don't flush since there's a bit of middle ground assuming they're in different fault domains. Garage object storage defaults to that However, this doesn't appear to be the case here... Unsurprisingly, performance goes to crap when sync is enabled. This is pretty old now but has some useful fsync/sec numbers which can be completely divorced from other I/O performance https://www.percona.com/blog/fsync-performance-storage-devices/ https://www.percona.com/blog/fsync-performance-storage-devic...
rdtsc 1mo ago
Absolutely. If they just dirty some pages in memory and return back to the client the benchmarks will look "insanely fast". I have nothing against this being a non-default option in a db/kv engine but anything advertising to be durable and not fsyncing by default is something I would stay away from. To me it's like a litmus test of how well the author knows/cares data durability and not destroying users data.
- rgbimbochamp 1mo ago
  durable() syncs periodically on flush, WAL rotation, and clean close; paranoid() is the sync-before-ack mode. This is clarified in the README, and the benchmarks report all three modes separately. Other KV-stores that you see on the market, do this too. It's a performance tradeoff most applications make. Sync on every write kills every optimization. See the benchmark table for example.
Betelbuddy 1mo ago
10 years after MongoDB is back.... https://youtu.be/b2F-DItXtZs https://youtu.be/b2F-DItXtZs
random3 1mo ago
Every few years someone pulls this. If you search HN for fsync you can see the trail :)
railka 1mo ago
Is fsync needed for every write in all projects? I am willing to trade database speed for the loss of the last written data within a 1-10ms window once or twice a year for non-financial and other non-critical data. So, power loss shouldn't be such a frequent event when there are active transactions at that moment, right?

stingraycharles 1mo ago

Yeah this should be benchmarked against other systems that have flush() disabled. mmap is nice but it doesn’t support durable semantics in the way that we usually mean with databases. if a write is acknowledged it should not be forgotten, which is not what this is.

rgbimbochamp 1mo ago
You're right, that mode provides process crash recovery, not power-loss durability. The benchmark compares it against fjall’s equivalent buffered-WAL mode.
- a2ff6eeb0 1mo ago
  If that's your design constraint, couldn't you speed it up by getting rid of the WAL?
  - oneshadab 1mo ago
    You'd lose durability against process crashes. If your system has a reasonable tolerance for power failure (multi-az multi-cloud), this can provide much better throughput
    - t098i3 1mo ago
      Indeed, a common enough pattern for etcd is to run it backed by a RAMdisk and have multi-az availability + periodic backups + tolerance at a business level to be OK losing some recent data.
- deathanatos 1mo ago
  Word choice matters. Defaults matter. People will go "well it says durable right here" and while arguably, yes, they should RTFM, it would still be great if tool-builders did not set the shotgun's default state to State::AT_FOOT. It would be nice if every paragraph of technical writing that I have to do need not be burdened by a thousand asterisks of "durable in this context means something other than durable".
  - carlmr 1mo ago
    >arguably, yes, they should RTFM Agreed. Good design is when the things do what you expect them to do without reading the manual, don't reuse wording with other meaning in the wrong way. That way if you do encounter nee wording, you know you should read the manual.
    - gryfft 1mo ago
      https://en.wikipedia.org/wiki/Principle_of_least_astonishment https://en.wikipedia.org/wiki/Principle_of_least_astonishmen...
muvlon 1mo ago
As of a couple years ago, mmap actually has a MAP_SYNC flag that makes it durable in the DB sense. The caveat is that it requires DAX on the file and so comes with a whole bunch of restrictions w.r.t. filesystem, storage media and even CPU architecture.

deepsun 1mo ago

Are we back to MongoDB -- no fsync() but webscale speed?

rgbimbochamp 1mo ago
Nope - https://github.com/kingroryg/turbokv#durability-presets https://github.com/kingroryg/turbokv#durability-presets
HatchedLake721 1mo ago
webscale… https://youtu.be/b2F-DItXtZs?is=wFhGujRHma3b7VLp https://youtu.be/b2F-DItXtZs?is=wFhGujRHma3b7VLp
- whirlwin 1mo ago
  This brings back memories!
re-thc 1mo ago
Agent scale!

procaryote 1mo ago

Pretty much... paranoid() seems to be the real durable() which isn't a great look for a database project. Being able to recover a db without corruption beyound losing the last few writes is a pretty useful feature, and buys a lot of performance, but it would be better to label that clearly, as a reasonable expectation on the durable() preset would be for it to be Durable.

insanitybit 1mo ago

This is surprisingly common, from what I can tell.

nijave 1mo ago

I give a little leeway to distributed systems that replicate and don't flush since there's a bit of middle ground assuming they're in different fault domains. Garage object storage defaults to that However, this doesn't appear to be the case here... Unsurprisingly, performance goes to crap when sync is enabled. This is pretty old now but has some useful fsync/sec numbers which can be completely divorced from other I/O performance https://www.percona.com/blog/fsync-performance-storage-devices/ https://www.percona.com/blog/fsync-performance-storage-devic...

rdtsc 1mo ago

Absolutely. If they just dirty some pages in memory and return back to the client the benchmarks will look "insanely fast". I have nothing against this being a non-default option in a db/kv engine but anything advertising to be durable and not fsyncing by default is something I would stay away from. To me it's like a litmus test of how well the author knows/cares data durability and not destroying users data.

rgbimbochamp 1mo ago
durable() syncs periodically on flush, WAL rotation, and clean close; paranoid() is the sync-before-ack mode. This is clarified in the README, and the benchmarks report all three modes separately. Other KV-stores that you see on the market, do this too. It's a performance tradeoff most applications make. Sync on every write kills every optimization. See the benchmark table for example.

Betelbuddy 1mo ago

10 years after MongoDB is back.... https://youtu.be/b2F-DItXtZs https://youtu.be/b2F-DItXtZs

random3 1mo ago

Every few years someone pulls this. If you search HN for fsync you can see the trail :)

railka 1mo ago

Is fsync needed for every write in all projects? I am willing to trade database speed for the loss of the last written data within a 1-10ms window once or twice a year for non-financial and other non-critical data. So, power loss shouldn't be such a frequent event when there are active transactions at that moment, right?