9 ms·

Real time, if accurate, allows you to establish relative ordering without coordination. With logical clocks, you can only establish partial order.

londons_explore 5y ago
But a real clock has some uncertainty bounds, and within those bounds, there is only partial or no ordering guarantee.
- sudhirj 5y ago
  Exactly, and Google claims to have made those bounds so small they can be assumed to be zero for all practical applications. They mention the bound size somewhere, and if you have enough leading zeroes for your use case you can just round it down.
  - judofyr 5y ago
    Do you have a source for this? From my understanding these bounds are actively being used during the commit protocol: The leader will wait (i.e. just run sleep()) for a period of time. This period is determined from the bounds and ensures that the commit is definitely in the past [across all replicas].
saurik 5y ago
But the whole point is that "real time" doesn't exist in some comparable sense, due to (if nothing else) relativity (which can't be ignored at the speed of light level, but is also relevant at an analogous, slower, speed of packets level). You need something like vector clocks, interval trees, or (at least) lamport clocks to understand the causal impacts of effects, as if something supposedly happened somewhere but there was no way for you to have yet known it happened due to being too far away to have noticed the impact yet by the time you did something else, you simply can't claim anything you did happened "after" the other thing: the ordering of two events disconnected by a-causality is fundamentally subjective and you can't sweep that under the rug.
- bob1029 5y ago
  I feel like the "sweeping under the rug" is being done here by way of engineering compromises (assumptions & bounds). The compromise in this case happens to be something along the lines of "If the GPS constellation fails, TrueTime fails (or gets really slow)". For better or worse, all of these TrueTime-style products (AWS has one now too), are effectively dependent on GPS to provide a global synchronous context. I don't think there is any feasible way to bootstrap or maintain a distributed clock with higher precision than its terrestrial network links would otherwise allow. The way I look at this, the GPS constellation contributes to the core of these database engines. It provides the most essential identity for all transactions (precise & unique timestamps). The actual database servers are just storing the data and ensuring transactions are interleaved properly. It is probably reasonable from a practical perspective to align your failure modes with those of other telecom and military organizations.
  - vvern 5y ago
    > “If the GPS constellation fails, TrueTime fails (or gets really slow)". To deal with this threat model, spanner has atomic clocks in each data center, named “Armageddon time masters”, the most badass name of a component in any distributed system I know of.
- judofyr 5y ago
  I’m not quite sure what point you’re making here. Spanner ensures causality by waiting until TrueTime tells you that the commit time is considered “in the past” for all replicas. There’s nothing swept under the rug here and this is a theoretically sound way of ensuring consistency (as long as you’re willing to sacrifice correctness if you’re not able to keep clocks in sync).
- seoaeu 5y ago
  The point of a database is to compute on. Nobody cares about the nanosecond precision time of when each event happened or the exact real-world ordering between them. What they care about is that all replicas process events in the same order so that they get deterministic results. The strong consistency spanner provides gives that guarantee, whereas using causal consistency would not.

londons_explore 5y ago

But a real clock has some uncertainty bounds, and within those bounds, there is only partial or no ordering guarantee.

sudhirj 5y ago
Exactly, and Google claims to have made those bounds so small they can be assumed to be zero for all practical applications. They mention the bound size somewhere, and if you have enough leading zeroes for your use case you can just round it down.
- judofyr 5y ago
  Do you have a source for this? From my understanding these bounds are actively being used during the commit protocol: The leader will wait (i.e. just run sleep()) for a period of time. This period is determined from the bounds and ensures that the commit is definitely in the past [across all replicas].

saurik 5y ago

But the whole point is that "real time" doesn't exist in some comparable sense, due to (if nothing else) relativity (which can't be ignored at the speed of light level, but is also relevant at an analogous, slower, speed of packets level). You need something like vector clocks, interval trees, or (at least) lamport clocks to understand the causal impacts of effects, as if something supposedly happened somewhere but there was no way for you to have yet known it happened due to being too far away to have noticed the impact yet by the time you did something else, you simply can't claim anything you did happened "after" the other thing: the ordering of two events disconnected by a-causality is fundamentally subjective and you can't sweep that under the rug.

bob1029 5y ago
I feel like the "sweeping under the rug" is being done here by way of engineering compromises (assumptions & bounds). The compromise in this case happens to be something along the lines of "If the GPS constellation fails, TrueTime fails (or gets really slow)". For better or worse, all of these TrueTime-style products (AWS has one now too), are effectively dependent on GPS to provide a global synchronous context. I don't think there is any feasible way to bootstrap or maintain a distributed clock with higher precision than its terrestrial network links would otherwise allow. The way I look at this, the GPS constellation contributes to the core of these database engines. It provides the most essential identity for all transactions (precise & unique timestamps). The actual database servers are just storing the data and ensuring transactions are interleaved properly. It is probably reasonable from a practical perspective to align your failure modes with those of other telecom and military organizations.
- vvern 5y ago
  > “If the GPS constellation fails, TrueTime fails (or gets really slow)". To deal with this threat model, spanner has atomic clocks in each data center, named “Armageddon time masters”, the most badass name of a component in any distributed system I know of.
judofyr 5y ago
I’m not quite sure what point you’re making here. Spanner ensures causality by waiting until TrueTime tells you that the commit time is considered “in the past” for all replicas. There’s nothing swept under the rug here and this is a theoretically sound way of ensuring consistency (as long as you’re willing to sacrifice correctness if you’re not able to keep clocks in sync).
seoaeu 5y ago
The point of a database is to compute on. Nobody cares about the nanosecond precision time of when each event happened or the exact real-world ordering between them. What they care about is that all replicas process events in the same order so that they get deterministic results. The strong consistency spanner provides gives that guarantee, whereas using causal consistency would not.