5 ms·

ELI5: How does this impact UNIX timestamps? Particularly for things that are in maintenance mode or otherwise minimally maintained. Nothing I do requires this

ELI5: How does this impact UNIX timestamps? Particularly for things that are in maintenance mode or otherwise minimally maintained. Nothing I do requires this level of precision, but certainly there are things that do.

eqvinox 3mo ago
UNIX timestamps are fully ignorant of leap seconds, i.e. pretends they don't exist. That means there can be physical seconds of time that cannot be referenced with a UNIX timestamp (when a leap second is inserted) as well as UNIX timestamps for seconds that don't exist (when a leap second is deleted).
- aidenn0 3mo ago
  It also means that if you subtract two timestamps, you might not get the actual time between them. Though this is also true of most ways of representing time (TAI being a notable exception).
  - eqvinox 3mo ago
    The really annoying part is that "leap smearing" (where people decided to just mush the leap second across about day) has made CLOCK_MONOTONIC unclear in this regard, since some leap smearing approaches affect that as well. Which destroyed any assumption a developer could make about CLOCK_MONOTONIC, since you won't know if leap smearing is in use :(. (And depending on the leap smearing implementation, it also smears CLOCK_TAI, jumps it to opposite polarity at the actual time of leap second, and then smears it again. The leap smearing people really made a mess of this.) From a correctness perspective, the only good choice is to go all-in on TAI. [Ed./P.S.:] "just ignore leap seconds" - that's going all-in on TAI. At this point it's probably easier to redefine UNIX timestamps as TAI based after 2035 ("abolishing leap seconds"), and introduce a new CLOCK_SOLAR_EARTH that accumulates leap seconds and can be used if/where necessary. The main issue is to create a proper delineation between the two clocks, which we just don't have at this point. Way too many systems where it's just not clear what they use. And note that leap seconds are earth specific too. You'll have entirely different requirements on e.g. Mars.
    - mtoner23 3mo ago
      After reading all that, maybe we should just have some bugs in our code and stop making up new things
    - alright2565 3mo ago
      > made CLOCK_MONOTONIC unclear in this regard I don't understand what you mean. Doesn't smearing still mean the clock only goes forward? It's still steadily incrementing. The only difference is that a second is slightly longer/shorter than you expect, but you already have to account for that if you're doing the kind of physics experiment where it would matter.
      - hdgvhicv 3mo ago
        If you have one event per millisecond then you get 86,401,000 events on a day you smear, and that’s inconsistent with the 86,400,000 you get on days either side.
        oconnore 3mo ago
        Sure but a system that breaks if it doesn't get exactly 86,400,000 millisecond scale scheduled events in a day has already been driving your SRE team insane.
    - Dansvidania 3mo ago
      I don't think I understand.. can you please dumb it down even more? I'd figured that UNIX time just counts actual seconds and that leap seconds and similar calendar shenanigans would be a problem of mapping epoch to the correct date, so that if normally epoch X maps to date D then both epoch X and X+1 map to date D. Am i to understand that leap seconds "stretch" a epoch unit ? so that some epoch second "lasted longer".
      - nemomarx 3mo ago
        I think for the stretch of time where a leap second is smeared, the epoch second would drift by a tiny fraction for each second or etc? So you'd have some chunk of UTC time where each second lasts something like 1.0005 epoch seconds, maybe? This would make converting from unix time to UTC need extra arbitrary handling for display, at least.
- eqvinox 3mo ago
  "fully ignorant" might not have been the best wording there... - clarification: "fully ignorant" from a human perspective, using dates and times. UNIX time lines up with those.
rzzzt 3mo ago
Whenever leap seconds were added, Google was running the clocks on their servers slower/faster over a longer period of time (hours) so they would slowly drift back in sync with the solid platinum, perfectly spherical grandfather's clock sitting in NIST or whatever: https://developers.google.com/time/smear https://developers.google.com/time/smear
mizmar 3mo ago
Yes, time() and clock_gettime(CLOCK_REALTIME) results are affected by leap seconds. New leap second will get to your system through NTP. Sadly NTP only distributes indicator flag that leap second is going to be introduced, but not the offset itself. But the distributed time itself is already affected by leap second, so NTP client doesn't really need to know. (In contrast the other time sync mechanisms - GPS and PTP - use time scale unaffected by leap seconds and distribute it as an additional information with UTC offset. And it's left on client to modify received time on its end. Kernel has a parameters in clock_adjtime() for leap seconds.) So if you have a passive system that has NTP client then it's time will change for new leap second on runtime. Linux treats UTC time as the dominant one, so that's the one saved to RTC device and will survive reboot. There is CLOCK_TAI that sounds like it should return TAI time, but it is such a second class citizen to the point that nothing on regular Linux desktop and server distros even set the offset and it returns the same time as CLOCK_REALTIME. There is a file in /etc with list of leap seconds that is part of some package, so you need to update the system to update this file. I don't believe traditional NTP software updates this package dynamically. But not many software uses it. If some init service script parsed and set the kernel UTC offset then your system's CLOCK_TAI would be one second late from rest of the world until update. But it doesn't affect UTC time on Linux in any way I know.

eqvinox 3mo ago

UNIX timestamps are fully ignorant of leap seconds, i.e. pretends they don't exist. That means there can be physical seconds of time that cannot be referenced with a UNIX timestamp (when a leap second is inserted) as well as UNIX timestamps for seconds that don't exist (when a leap second is deleted).

aidenn0 3mo ago
It also means that if you subtract two timestamps, you might not get the actual time between them. Though this is also true of most ways of representing time (TAI being a notable exception).
- eqvinox 3mo ago
  The really annoying part is that "leap smearing" (where people decided to just mush the leap second across about day) has made CLOCK_MONOTONIC unclear in this regard, since some leap smearing approaches affect that as well. Which destroyed any assumption a developer could make about CLOCK_MONOTONIC, since you won't know if leap smearing is in use :(. (And depending on the leap smearing implementation, it also smears CLOCK_TAI, jumps it to opposite polarity at the actual time of leap second, and then smears it again. The leap smearing people really made a mess of this.) From a correctness perspective, the only good choice is to go all-in on TAI. [Ed./P.S.:] "just ignore leap seconds" - that's going all-in on TAI. At this point it's probably easier to redefine UNIX timestamps as TAI based after 2035 ("abolishing leap seconds"), and introduce a new CLOCK_SOLAR_EARTH that accumulates leap seconds and can be used if/where necessary. The main issue is to create a proper delineation between the two clocks, which we just don't have at this point. Way too many systems where it's just not clear what they use. And note that leap seconds are earth specific too. You'll have entirely different requirements on e.g. Mars.
  - mtoner23 3mo ago
    After reading all that, maybe we should just have some bugs in our code and stop making up new things
  - alright2565 3mo ago
    > made CLOCK_MONOTONIC unclear in this regard I don't understand what you mean. Doesn't smearing still mean the clock only goes forward? It's still steadily incrementing. The only difference is that a second is slightly longer/shorter than you expect, but you already have to account for that if you're doing the kind of physics experiment where it would matter.
    - hdgvhicv 3mo ago
      If you have one event per millisecond then you get 86,401,000 events on a day you smear, and that’s inconsistent with the 86,400,000 you get on days either side.
      - oconnore 3mo ago
        Sure but a system that breaks if it doesn't get exactly 86,400,000 millisecond scale scheduled events in a day has already been driving your SRE team insane.
  - Dansvidania 3mo ago
    I don't think I understand.. can you please dumb it down even more? I'd figured that UNIX time just counts actual seconds and that leap seconds and similar calendar shenanigans would be a problem of mapping epoch to the correct date, so that if normally epoch X maps to date D then both epoch X and X+1 map to date D. Am i to understand that leap seconds "stretch" a epoch unit ? so that some epoch second "lasted longer".
    - nemomarx 3mo ago
      I think for the stretch of time where a leap second is smeared, the epoch second would drift by a tiny fraction for each second or etc? So you'd have some chunk of UTC time where each second lasts something like 1.0005 epoch seconds, maybe? This would make converting from unix time to UTC need extra arbitrary handling for display, at least.
eqvinox 3mo ago
"fully ignorant" might not have been the best wording there... - clarification: "fully ignorant" from a human perspective, using dates and times. UNIX time lines up with those.

rzzzt 3mo ago

Whenever leap seconds were added, Google was running the clocks on their servers slower/faster over a longer period of time (hours) so they would slowly drift back in sync with the solid platinum, perfectly spherical grandfather's clock sitting in NIST or whatever: https://developers.google.com/time/smear https://developers.google.com/time/smear

mizmar 3mo ago

Yes, time() and clock_gettime(CLOCK_REALTIME) results are affected by leap seconds. New leap second will get to your system through NTP. Sadly NTP only distributes indicator flag that leap second is going to be introduced, but not the offset itself. But the distributed time itself is already affected by leap second, so NTP client doesn't really need to know. (In contrast the other time sync mechanisms - GPS and PTP - use time scale unaffected by leap seconds and distribute it as an additional information with UTC offset. And it's left on client to modify received time on its end. Kernel has a parameters in clock_adjtime() for leap seconds.) So if you have a passive system that has NTP client then it's time will change for new leap second on runtime. Linux treats UTC time as the dominant one, so that's the one saved to RTC device and will survive reboot. There is CLOCK_TAI that sounds like it should return TAI time, but it is such a second class citizen to the point that nothing on regular Linux desktop and server distros even set the offset and it returns the same time as CLOCK_REALTIME. There is a file in /etc with list of leap seconds that is part of some package, so you need to update the system to update this file. I don't believe traditional NTP software updates this package dynamically. But not many software uses it. If some init service script parsed and set the kernel UTC offset then your system's CLOCK_TAI would be one second late from rest of the world until update. But it doesn't affect UTC time on Linux in any way I know.