2 ms·
To add a little backstory, TigerBeetle's clock fault failure detector described here is only for the financial domain of the state machine, not for the correctn
by jorangreef 5y ago
To add a little backstory, TigerBeetle's clock fault failure detector described here is only for the financial domain of the state machine, not for the correctness of the consensus protocol. This has to do with financial regulation around auditing in some jurisdictions and not with total order in a distributed systems context.
We simply need a mechanism to know when PTP or NTP is broken so we can shut down as an extra safety mechanism, that’s all. Detecting when the clock sync service is broken (specifically, an unaligned partition so that the hierarchical clock sync doesn’t work, but the TigerBeetle cluster is still up and running) is in no way required by TigerBeetle for strict serializability, it’s pure defense-in-depth for the financial domain to avoid running into bad financial timestamps.
We tried to make clear in the talk itself that leader leases are “dangerous, something we would never recommend”. And on the home page, right up front (because it’s important to us) we have: “No stale reads”.
We also want a fault-tolerant clock with upper and lower bounds for timing out financial two-phase commit payments that need to be rolled back if another bank’s payments system fails. We can’t lock people’s liquidity for years on end because of a clock fault that we could have detected otherwise.
You can imagine TigerBeetle’s state machine (not the consensus protocol itself) as processing two-phase payments that look alot like a credit card transaction that has a two-phase auth/accept flow, so you want the leader to timestamp roughly close enough to true time, so that the financial transaction either goes through or ultimately gets rolled back after roughly N seconds.
Hope that clarifies the post (and gets you excited about TigerBeetle’s safety)!
P.S. We’re launching a $20K consensus challenge in early September. All the early invite details are over here [1]. Hope to see you at the live event, where we’ll have back-to-back interviews with Brian Oki and James Cowling, who authored and revised the pioneering Viewstamped Replication consensus protocol respectively.
[1] https://www.tigerbeetle.com/20k-challenge https://www.tigerbeetle.com/20k-challenge