8 ms·
Keeping Time with Amazon Time Sync Service
- CaliforniaKarl 9y agoArgh! I wish the NTP servers were usable from outside of AWS. Outside of North America and Europe, there aren't as many public NTP servers as there should be. For example, according to http://www.pool.ntp.org/zone/south-america http://www.pool.ntp.org/zone/south-america, South America has less than 50 NTP servers. It would be awesome if the large, widely-distributed providers (not just AWS with CloudFront, but also Akami, Cloudflare, etc.) provided public, Stratum-1 NTP servers at their POPs.
- ehPReth 9y agoGoogle hosts some: https://developers.google.com/time/ https://developers.google.com/time/
- jlgaddis 9y agoNote that they smear time, though (WRT leap seconds), as do Amazon. You generally do not want to mix "smearing" and "non-smearing" time servers.
- mark-r 9y agoYou probably don't want to mix time servers that use a different algorithm for smearing, either.
- aunty_helen 9y agoInteresting point, we're looking at implementing this new ntp server. Wanted to know a bit more about the risks of mixing leap second algos. Is there any more info / links you can give on the subject?
- LeoPanthera 9y agotime.windows.com, time.apple.com, and time1.google.com are all publicly accessible. Apple's one is also available at time.asia.apple.com and time.euro.apple.com
- deleted 9y ago[deleted]
- 1010011010 9y agowindows and apple tend to keep terribly inconsistent time, though, and windows time goes nutso at leap smears.
- LeoPanthera 9y agoDo you have a citation for this? I've been monitoring NTP servers for years and Apple's are among the closest to my GPS reference.
- dboreham 9y agoHmm...our data center (bare iron, not cloud) has always provided NTP servers. I assumed this was standard.
- tedivm 9y agoAnyone can provide an NTP server, but that's much different from having an atomic clock powered NTP server. The servers your datacenter provide are probably clients of other NTP servers, so the quality and availability of NTP servers near your datacenter is still really important.
- daurnimator 9y agoI would assume a datacentre would put a GPS somewhere to provide time to an NTP server. It's quite cheap, easy to do, and a useful value-add for their customers; and it's something that can only can be done by someone with physical (and likely roof) access.
- wpietri 9y agoAnd I would add for the crowd that one can easily check what sort of NTP server is on offer. NTP tracks the "stratum" of a server, which is how far the server is from an authoritative time source: https://en.wikipedia.org/wiki/Network_Time_Protocol#Clock_strata https://en.wikipedia.org/wiki/Network_Time_Protocol#Clock_st... If you're using ntpd, you can check it like this: $ ntpq ntpq> peer remote refid st t when poll reach delay offset jitter ============================================================================== *y.ns.gin.ntt.ne 249.224.99.213 2 u 815 1024 377 1.230 -4.148 1.825 +hadb2.smatwebde 209.51.161.238 2 u 809 1024 377 49.672 -2.073 3.869 +six0.ntp3.mattn 128.227.205.3 2 u 409 1024 377 63.721 -4.328 2.669 -hadb1.smatwebde 45.79.187.10 3 u 808 1024 377 49.767 -6.600 2.927 These are 4 servers out of the NTP pool; 3 are stratum 2 and one is stratum 3. The * means it has selected the first, the + means it likes the next 2, and the - means it has discarded the last one.
- Veratyr 9y ago
- BillinghamJ 9y agoI have been hoping for this for a long time, with Google’s TrueTime system being so unique. Quite a lot more work would have to go into guaranteeing the results all the way into your application, but this is still a great start.
- eccstartup 9y ago"sudo sudo yum erase ntp*"
- anubhavmishra 9y agoWhen is Google announcing TrueTime as a service on GCP? I see them doing this really soon since aws has it now
- NelsonMinar 9y agoTrueTime is a lot more than just the NTP service Amazon just announced.
- elvinyung 9y agoCan you expand on what you meant by that? If you take it at face value, the TrueTime API is implementable on top of any clock synchronization protocol. It's just coincidentally true that if you're close to a stratum 0-equivalent clock, your skew will be low. In terms of implementation, TrueTime daemons essentially also use Marzullo's algorithm to estimate timestamps, just like NTP. What am I missing?
- tqh 9y agoAmazon TimeSync is probably a lot more than NTP as well. It must be part of XEN/hw as it is not yet available on the Nitro based instances. That it provide's an NTP endpoint is just the top of the cake...
- NelsonMinar 9y agoDetails worth highlighting.. They recommend chrony over ntpd, which makes particular sense given that chrony is designed to work well on virtual machine. They are doing leap second smearing like Google does internally. They don't say anything about what sort of accuracy you can expect but if the clock source is truly local it could be very good. It's kinda neat they're using a link local address.
- ams6110 9y agoWhat is leap second smearing? Spreading it out over some other period, vs. a sudden jump of one second?
- irhd 9y ago> Spreading it out over some other period Yes. They make a second last slightly long for some time period.
- deleted 9y ago[deleted]
- zaxomi 9y agoOr shorter... Leap seconds can be negative.
- duskwuff 9y agoTheoretically, yes, but it's never happened since leap seconds were established in 1972, and it seems very unlikely to happen for quite some time. Chances are that a lot of software doesn't actually support negative leap seconds at all.
- sharth 9y agohttps://developers.google.com/time/smear https://developers.google.com/time/smear
- carterschonwald 9y agoI’m amidst digging into getting some gps clock hardware to make sure an open source db we're developing at work can work in some interesting situations. (Hit me up if you wanna join the team ;) ) High accuracy time is a fascinating domain. It’s this weird mix of general relativity, hardware, and anthropology. Cesium atoms and satellites vs calendars vs astronomy: who wins? :) Point being: more organizations making high accuracy time available to more folks is a good thing!
- deleted 9y ago[deleted]
- ac2u 9y agoCool, something like what happens in Google's datacentres to provide Spanner?
- carterschonwald 9y agoYes! The system we're working on is one that benefits from the exact same sort of idea : having time info of better accuracy should allow better performance of the same system. (Logical time works too, but will often require much more coordination to resolve ordering or conflicts )
- dfc 9y agoWhat trouble are you having with the GPSs? Getting them hooked up to ntpd is fairly easy after you get used to some of the common hiccups. As far as who wins for a long time now it has been astronomers. The fact that Google and Amazon are doing leap smearing is a nice sign that the tide may be turning. It is very rare that you see organizations supporting leap seconds during big standards discussions that are not astronomers.
- carterschonwald 9y agoGetting a rig for some servers is easy. Figuring out a range of hardware to test and support for a range of different types of possible users with different budgets, from some tiny hobby user at home to the entire global economy... have very different needs and hardware budgets. :)
- jacquesm 9y agoIs it going to be monotonically increasing?
- etskinner 9y agoI see no reason that it would ever go backward, so yes. Especially with leap second smearing, meaning it's not only monotonically increasing, but also never stops (never a slope of zero).
- benwilber0 9y agoSystem clock synchronization is a really hard problem but especially important for things like live video stream encoding where the frame timestamps need to correspond to the real wall-clock (within milliseconds) for things like sub-second play-by-play synchronization.
- SomeStupidPoint 9y agoHow accurate does that need to be...? 100ms? 10ms? 1ms? 100us?
- benwilber0 9y agoIt varies by sport and how accurate you need your timers to be. NBA needs a timer within "seconds" -- Formula 1 and NASCAR need timers within milliseconds.
- Daviey 9y agoFinance centric: Finra (Financial Industry Regulatory Authority) requires <50ms MiFID II (EU centric) HFT requirement can be as low as 100 microseconds accuracy, or 0.1ms.
- elvinyung 9y agoAre there SLAs for this out? Can I get TrueTime-style assumptions about my clock skew?
- falcolas 9y agoSomewhat meta, but this is a bit ironic of an offering considering the time on their display boards at re:Invent are noticeably out of sync.
- ajessup 9y agoI'd be curious to know how this compares to Google's (internal ) TrueTime service[1], and if it allows for development of globally consistent but highly available storage systems like Spanner. [1] https://research.google.com/pubs/pub45855.html https://research.google.com/pubs/pub45855.html
- wahnfrieden 9y agoConsidering this coincided with announcements of multi-region multi-master DBs (Aurora, DynamoDB) it seems likely.
- decebalus1 9y agoInteresting to know if they have an SLA for this and how it's formulated. Time is important and hard. I doubt that customers who really need precise time would walk away from their ghetto solutions with no SLA because when something like this craps out, it's almost always a circus. But hey, when time precision and accuracy is a bonus not a requirement for your AWS instance then 'free local NTP server!'
- dfc 9y ago> uses a fleet of redundant satellite-connected and atomic clocks in each region I am really curious to know what atomic frequency standards they are using. There is something a little suspicious about the wording. I have a feeling they mean "we use GPSDOs" and there are no atomic frequency standards physically located in each regional datacenter. I'm also a little surprised to see that the remote clock is stratum 4. It would be interesting to see how they are distributing the time between their clocks. If someone from amazon reads this it would be neat if you posted some more details about your setup.
- ninjakeyboard 9y agoDoes this actually work? We had horrible issues with time drifting between services in AWS at scale in distributed systems. Something like eg Google Spanner relies on atomic clocks - it's a combination of hardware and software that enable the use of wallclock time.
- sudhirj 9y agoThe amount of dog fooding that happens at AWS is impressive - looks like they needed and built this for DynamoDB global tables (assumes clock sync across all regions) and made it available to all instances.
- mwarkentin 9y agoCurious if I’ll be able to replace the NTP servers inside our PCI-compliant environment with this.