4 ms·

> In the event of a leap second, the Amazon Time Sync service automatically handles this for you by smoothing out the addition, or removal, of the leap second w

by amluto 4y ago

> In the event of a leap second, the Amazon Time Sync service automatically handles this for you by smoothing out the addition, or removal, of the leap second with a 24-hour linear smear from noon to noon UTC. No, wrong! Setting your system clock to use smeared time is legitimate. Doing this across a large fleet is even reasonable. But this is a policy decision, and it is not UTC. If a fleet wants shaded time, it should be explicit in the protocol or current configuration. Smearing time by having a server that claims to be NTP doing the smearing is simply a lie, and will confuse clients who expect the server to tell the actual time. Even chrony’s leapsecmode=slew option expects the upstream server to report the leap second. Last time there was a leap second, some smeared servers got into the NTP pool, and the result was a mess.

vbezhenar 4y ago
NTP protocol contains leap indicator. So I'd expect that chrony would not do anything if AWS NTP server would do smoothing out and not set that leap indicator.
- amluto 4y ago
  Chrony would inherit AWS’s skew, which may or may not match chrony’s skew. (But maybe incorrectly because chrony does not expect the length of a second to suddenly change by about 11 ppm.) Chrony configured to not skew will obviously skew anyway.
eternityforest 4y ago
I wish UTC would just standardize smears. Computers should use perfect unsmeared clocks, translation for humans can just smear with a polynomial predictor that updates every few years. If you miss an update you get gracefully degraded accuracy, no jumps anywhere m
the_mitsuhiko 4y ago
These servers are not in the NTP pool so that concern does not apply. I'm pretty sure most cloud providers use smearing at the NTP level.
- donavanm 4y ago
  Yes. Everyones been doing smearing (of slightly different flavors) since 2014ish when a kernel bug caused a bunch of havoc across linux and cisco boxes. It was a Bad Time. For everyone that gets leap seconds right theres 10 who muck it up and have incorrect telemtry at best or data loss and crashes at worst. And this happens every time. So folks like GP who want or need to be Technically Correct are free to do so. But the smeared time from the big providers saves tons of hassle for regular developers and customers. Disclaimer: principal at AWS, opinions are my own. I have no inside knowledge of this AWS NTP release.
  - ignoramous 4y ago
    > ...saves tons of hassle for regular developers and customers. True. I mean, regulars would even be okay even with roughtime: https://roughtime.googlesource.com/ https://roughtime.googlesource.com/ While BigTech needs even more accurate sources: https://research.google/pubs/pub49716/ https://research.google/pubs/pub49716/
  - brookst 4y ago
    > Bad Time Nice.
  - amluto 4y ago
    The Bad Time was bad. But it was also bad when, a while later, at least one smearing server got into the public NTP pool, causing regrettable loss of time sync. (How do I know this? One of my servers caught the contagion and disagreed with my other servers by about a second. This can happen regardless of whether I’m trying to use smeared time myself.) I find it quite regrettable that, 8 years later, large fleets and their unwitting customers use various, not necessarily compatible smears, and that they kludge it in via NTP. IMO a much better solution would be to make all this explicit: Define UTCS as smeared UTC. Get everyone to agree at to exactly how it is smeared. Publish an RFC or similar. (Getting general purpose software to be aware of UTC vs UTCS is not necessarily required.) Get all the major software vendors to implement UTCS by default, chrony-style. Have the major cloud NTP servers continue to report UTC. Make the major NTP software implementations understand this all well enough so a server can run UTCS on its system clock but still report UTC to its own NTP clients.
cesarb 4y ago
This might be fixed by NTPv5; the proposed draft RFC I looked at (https://datatracker.ietf.org/doc/draft-mlichvar-ntp-ntpv5/05/ https://datatracker.ietf.org/doc/draft-mlichvar-ntp-ntpv5/05...) has a flag to specify whether or not the server is using leap smear.
- toomuchtodo 4y ago
  Can the server provide both a smeared and non smeared response simultaneously, with the client electing which correction to apply? This would make the process more deterministic, versus the uncertainty of what time you’re being provided actually is.
- amluto 4y ago
  Oof, I feel like I should send a comment. This is bad: 1. There is no definition of what the leap smeared timescale means. This may prevent a client from usefully synchronizing with multiple servers using that timescale. 2. There’s enough information in the messages to compute leap smeared time. (The leap second indicator along with UTC is sufficient.). Chrony does this. As a result, it seems like it would never actually make sense for servers to synchronize to each other in smeared mode.

by amluto 4y ago

vbezhenar 4y ago

NTP protocol contains leap indicator. So I'd expect that chrony would not do anything if AWS NTP server would do smoothing out and not set that leap indicator.

amluto 4y ago
Chrony would inherit AWS’s skew, which may or may not match chrony’s skew. (But maybe incorrectly because chrony does not expect the length of a second to suddenly change by about 11 ppm.) Chrony configured to not skew will obviously skew anyway.

eternityforest 4y ago

I wish UTC would just standardize smears. Computers should use perfect unsmeared clocks, translation for humans can just smear with a polynomial predictor that updates every few years. If you miss an update you get gracefully degraded accuracy, no jumps anywhere m

the_mitsuhiko 4y ago

These servers are not in the NTP pool so that concern does not apply. I'm pretty sure most cloud providers use smearing at the NTP level.

donavanm 4y ago
Yes. Everyones been doing smearing (of slightly different flavors) since 2014ish when a kernel bug caused a bunch of havoc across linux and cisco boxes. It was a Bad Time. For everyone that gets leap seconds right theres 10 who muck it up and have incorrect telemtry at best or data loss and crashes at worst. And this happens every time. So folks like GP who want or need to be Technically Correct are free to do so. But the smeared time from the big providers saves tons of hassle for regular developers and customers. Disclaimer: principal at AWS, opinions are my own. I have no inside knowledge of this AWS NTP release.
- ignoramous 4y ago
  > ...saves tons of hassle for regular developers and customers. True. I mean, regulars would even be okay even with roughtime: https://roughtime.googlesource.com/ https://roughtime.googlesource.com/ While BigTech needs even more accurate sources: https://research.google/pubs/pub49716/ https://research.google/pubs/pub49716/
- brookst 4y ago
  > Bad Time Nice.
- amluto 4y ago
  The Bad Time was bad. But it was also bad when, a while later, at least one smearing server got into the public NTP pool, causing regrettable loss of time sync. (How do I know this? One of my servers caught the contagion and disagreed with my other servers by about a second. This can happen regardless of whether I’m trying to use smeared time myself.) I find it quite regrettable that, 8 years later, large fleets and their unwitting customers use various, not necessarily compatible smears, and that they kludge it in via NTP. IMO a much better solution would be to make all this explicit: Define UTCS as smeared UTC. Get everyone to agree at to exactly how it is smeared. Publish an RFC or similar. (Getting general purpose software to be aware of UTC vs UTCS is not necessarily required.) Get all the major software vendors to implement UTCS by default, chrony-style. Have the major cloud NTP servers continue to report UTC. Make the major NTP software implementations understand this all well enough so a server can run UTCS on its system clock but still report UTC to its own NTP clients.

cesarb 4y ago

This might be fixed by NTPv5; the proposed draft RFC I looked at (https://datatracker.ietf.org/doc/draft-mlichvar-ntp-ntpv5/05/ https://datatracker.ietf.org/doc/draft-mlichvar-ntp-ntpv5/05...) has a flag to specify whether or not the server is using leap smear.

toomuchtodo 4y ago
Can the server provide both a smeared and non smeared response simultaneously, with the client electing which correction to apply? This would make the process more deterministic, versus the uncertainty of what time you’re being provided actually is.
amluto 4y ago
Oof, I feel like I should send a comment. This is bad: 1. There is no definition of what the leap smeared timescale means. This may prevent a client from usefully synchronizing with multiple servers using that timescale. 2. There’s enough information in the messages to compute leap smeared time. (The leap second indicator along with UTC is sufficient.). Chrony does this. As a result, it seems like it would never actually make sense for servers to synchronize to each other in smeared mode.