4 ms·

The post talks about Spanner using "a specialized hardware solution that uses both GPS and atomic clocks to ensure a minimal clock skew across servers." But w

by tytso 8y ago

The post talks about Spanner using "a specialized hardware solution that uses both GPS and atomic clocks to ensure a minimal clock skew across servers." But what it fails to mention is that is that this time is distributed using a software solution --- NTP, in fact. Google makes its "leap-smeared NTP network" available via Google's Public NTP service. And it's not expensive for someone to buy their own GPS clock and run their own leap-smared NTP service. Yes, it means that someone who installs a NewSQL database will have to set up their own time infrastructure. But that's not hard! There are lots of things about hardware setup which are at a similar level of trickiness, such as UPS with notification so that servers can shutdown gracefully when the batteries are exhausted, locating your servers in on-prem data centers so that if a flood takes out one data center, you have continuity of service elsewhere, etc., etc. Or of course, you can pay a cloud infrastructure provider (which Google happens to provide, but Amazon and Azure also provides similar services) to take care all of these details for you. Heck, if you use Google Compute Platform, you can use the original Spanner service (accept no substitutes :-)

jrockway 8y ago
I agree. Attempting to remove the GPS and atomic clocks from the system seems like a premature optimization to me. They are cheap and the ability to order events according to an actual wall clock is a huge benefit. Some basics for less-critical applications: https://github.com/jrockway/beaglebone-gps-clock/blob/master/README.md https://github.com/jrockway/beaglebone-gps-clock/blob/master...
jakemoshenko 8y ago
I feel like the author is making a huge omission by not talking about the properties of multi-raft sharded DBs coupled with the TrueTime-like APIs from cloud providers in 2018. He links to a post from the CEO of CockroachDB written in early 2016, and Amazon launched their TimeSync service in 2017. It's likely that Spencer would have something different to say about the matter today.
- Quekid5 8y ago
  > I feel like the author is making a huge omission by not talking about the properties of multi-raft sharded DBs coupled with the TrueTime-like APIs from cloud providers in 2018. I mean this is all fine and dandy if it works (plus, how do know that it actually works in all the edge cases?), but that's a HUGE amount of complexity. IMO you really need to have a very good reason to even consider anything this complex.
ryandrake 8y ago
> And it's not expensive for someone to buy their own GPS clock and run their own leap-smared NTP service This is a fun weekend hobby project that anyone can do at home if they are curious. Get a GPS chip with a PPS output signal, a plain old “hockey puck” antenna, and a computer to hook it up to (i have a rpi in my attic) and you can deliver fairly accurate time to your home network. Not a setup I’d rely on for a commercial, globally distributed deployment, but accurate enough for home use, and probably better than your current time source.
- deepsun 8y ago
  The problem with production usage is not clock accuracy, but accuracy guarantees. If a clock is within whopping 1ms, it is ok as long as I can prove it is definitely within the 1ms. Which, as far as I see, boils down to hardware+software reliability.
adrianmonk 8y ago
It makes me wonder if, as an industry, we shouldn't just include hardware clocks in our basic expectations for a data center. That is, in order to be considered a satisfactory, non bare bones data center, you'd need to supply on-site, hardware, redundant, maintained and monitored clocks, verifiably meeting certain standards of accuracy and precision. Just like right now people expect backup power, cooling, network connectivity, physical security, access, etc. It doesn't seem like the burden would be that large for data center operators. The hardware costs don't sound large, and they already have people being paid to monitor and maintain things.

by tytso 8y ago

jrockway 8y ago

I agree. Attempting to remove the GPS and atomic clocks from the system seems like a premature optimization to me. They are cheap and the ability to order events according to an actual wall clock is a huge benefit. Some basics for less-critical applications: https://github.com/jrockway/beaglebone-gps-clock/blob/master/README.md https://github.com/jrockway/beaglebone-gps-clock/blob/master...

jakemoshenko 8y ago

I feel like the author is making a huge omission by not talking about the properties of multi-raft sharded DBs coupled with the TrueTime-like APIs from cloud providers in 2018. He links to a post from the CEO of CockroachDB written in early 2016, and Amazon launched their TimeSync service in 2017. It's likely that Spencer would have something different to say about the matter today.

Quekid5 8y ago
> I feel like the author is making a huge omission by not talking about the properties of multi-raft sharded DBs coupled with the TrueTime-like APIs from cloud providers in 2018. I mean this is all fine and dandy if it works (plus, how do know that it actually works in all the edge cases?), but that's a HUGE amount of complexity. IMO you really need to have a very good reason to even consider anything this complex.

ryandrake 8y ago

> And it's not expensive for someone to buy their own GPS clock and run their own leap-smared NTP service This is a fun weekend hobby project that anyone can do at home if they are curious. Get a GPS chip with a PPS output signal, a plain old “hockey puck” antenna, and a computer to hook it up to (i have a rpi in my attic) and you can deliver fairly accurate time to your home network. Not a setup I’d rely on for a commercial, globally distributed deployment, but accurate enough for home use, and probably better than your current time source.

deepsun 8y ago
The problem with production usage is not clock accuracy, but accuracy guarantees. If a clock is within whopping 1ms, it is ok as long as I can prove it is definitely within the 1ms. Which, as far as I see, boils down to hardware+software reliability.

adrianmonk 8y ago

It makes me wonder if, as an industry, we shouldn't just include hardware clocks in our basic expectations for a data center. That is, in order to be considered a satisfactory, non bare bones data center, you'd need to supply on-site, hardware, redundant, maintained and monitored clocks, verifiably meeting certain standards of accuracy and precision. Just like right now people expect backup power, cooling, network connectivity, physical security, access, etc. It doesn't seem like the burden would be that large for data center operators. The hardware costs don't sound large, and they already have people being paid to monitor and maintain things.