11 ms·

Unix time is bad and needs replacement, not UTC

133 points by zowie_vd 4y ago

rapsey 4y ago
Is it not irrelevant for most use cases? Leap seconds or whatever. Think of the time of this post. It is stored but how important is the accuracy?
- fhd2 4y ago
  Timestamps are used for a lot of scheduling tasks that need to be reasonably precise. So an event happening one second earlier than expected, or even twice, could be a problem I suppose. Not exactly impossible to deal with, but I guess lots of people screw it up? Haven't really worked on systems where this _truly_ matters.
  - rapsey 4y ago
    Scheduling tasks on a system is done using UTC/local time which the system keeps accurate. Unix time is not used for that.
    - butlerm 4y ago
      I am not aware of any operating system kernel that schedules tasks or timer expiration using a strict definition of UTC. Unix time is the most common, and local monotonic (non-decreasing) clocks are becoming common for that as well. Of course there are application level schedulers that use wall clock time in UTC format, and if you set the clock back your task may run twice, which is a more serious real world problem than running twice around a leap second. That is why no one who is not attached to civil time should use UTC or Unix time for short duration timer expiration in particular.
  - spookie 4y ago
    Synchronization between computers is hard. https://www.gnu.org/software/coreutils/manual/coreutils.html#Date-input-formats https://www.gnu.org/software/coreutils/manual/coreutils.html...
  - sohkamyung 4y ago
    Could it matter for high-frequency trading? Since these trades occur in milliseconds, I presume trading being off by a second would be considered very damaging.
    - gpderetta 4y ago
      It is not just very damaging, it also has legal implications. For example MIFID II regulations require at least 100us resolution for trade reporting. In practice PTP and GPS clocks are a requirement.
- globular-toast 4y ago
  That was my thought when considering the difficult and impossible tasks of converting to/from UTC-based timestamps. If we accept that UTC is only for human use then we should also accept that humans can't tell the time of day down to the second just based on their circadian rhythm and the Sun. Given a TAI-based timestamp a hundred years in the future we can be pretty certain in saying "yeah it will be around lunch time" (assuming there is no major changes in the solar system that would probably have such an impact on the lives of every human that nobody would care about this event anyway).
  - account42 4y ago
    But if you follow that argument then you might as well drop leap secnds from UTC (which also drops them from Unix time). Which is what was done. So computers can have nice math and conversions to UTC are not needlessly complicated for little benefit to humans.
- zowie_vd 4y ago
  I agree that for many use cases the exact second is irrelevant, like submission times and things likes that. That being said, programmers do seem to expect, for example, that code like `(int)time(NULL) - previous_time` works and accurately gives the difference in time between now and whenever `previous_time` was given its value. This does work, but only 99.99% of the time. Then it breaks 0.01% of the time. For bug-free code it's important that the expectations of the programmer are in line with reality, and the problem with Unix time is that programmers' expectations of the mathematical properties of Unix time and the real mathematical properties of Unix time do not line up. This is what leads to the countless bugs involving leap seconds. The fact that accuracy is often not important is also crucial to my proposal of legacy Unix time, because legacy Unix time would eventually go out of sync with the rotation of the Earth just like TAI. In almost all cases this would practically be an aesthetic bug where dates in old user interfaces would be a few seconds off from the real date, and therefore not particularly harmful.
- rgmerk 4y ago
  For things like mission-critical logging, having time jump around, or even skew, is a real PITA.
Tenoke 4y ago
>The exact way Unix time handles leap seconds* is complex and an unnecessary detail for this article, but it will suffice to say that (i) whenever a positive leap second occurs, a timestamp is used twice, and (ii) if a negative leap second were to occur, a timestamp is skipped. Huh, I always assumed that Unix time just ignores them and ticks forward. I get why e.g. UTC needs them but why would it make sense for Unix time to account for them given how it's defined.
- Cu3PO42 4y ago
  If Unix time ignored them, the conversion between timestamps and UTC would no longer be possible with just arithmetic operations. You'd need a list of leap seconds and take them into account. Now one can argue if this is better or worse than the current solution, but it's not without its tradeoffs.
  - jesprenj 4y ago
    Edit: My original comment is incorrect. Please read child comments. Original comment: Yes, to convert from UNIX time to UTC, you do need a list of leap seconds.
    - masklinn 4y ago
      No. You need a list of leap seconds to convert from Unix time to TAI.
    - valleyer 4y ago
      Others have already corrected you, but in case it makes it more concrete, here is the implementation of timegm() in musl (the libc used in Alpine Linux and some other Linux distributions). timegm() takes a time in UTC and converts it to Unix time. http://git.musl-libc.org/cgit/musl/tree/src/time/timegm.c http://git.musl-libc.org/cgit/musl/tree/src/time/timegm.c http://git.musl-libc.org/cgit/musl/tree/src/time/__tm_to_secs.c http://git.musl-libc.org/cgit/musl/tree/src/time/__tm_to_sec... You'll notice there is no handling or knowledge of leap seconds. (Leap years are handled, of course.)
  - 4y ago
dunno7456 4y ago
Everything is bad and needs replacement :)
PeterisP 4y ago
Can someone comment on the practical usefulness of having UTC stay strictly synchronized with the rotation of the Earth? Intuitively it should be synchronized, but giving it more thought it is not entirely obvious why; e.g. my current local time zone is more than half an hour off of local solar time, and that isn't really a practical problem, so if UTC would be, say, five minutes off of actual solar time in Greenwich, what issues does it cause for people? Astronomy would be using specific time systems and/or adjustments anyway.
- gerikson 4y ago
  I see leap seconds as regulatory capture by astronomers :D Basically it's convenient for them to have civil time (UTC) never more than 0.5s from UT1.
  - masklinn 4y ago
    Astronomers don’t care, they already need like 5 different clocks. Also the allowable variance between UTC and ut1 is 0.9.
    - PeterisP 4y ago
      Exactly - and if astronomers don't care, what other reasons are there to require accurate synchronization of UTC (and civil time) with earth's rotation?
      - lostmsu 4y ago
        Considering the rate of progress, timestamps might need to be replaced with reference-invariant coordinates.
    - moffkalast 4y ago
      Or one clock that can switch between 5 different times.
      - 4y ago
ZiiS 4y ago
Large cloud providers smear time. A day containing a leap second has the normal number of Unixtime "seconds" but each is 1/86400 too long. The inaccuracy this introduces is relevant to such a tiny tiny percentage of use cases; who already have to deal with the extreme complexities of time at this accuracy that everyone wins.
- zowie_vd 4y ago
  Large cloud providers are actually exactly the ones who argued that leap seconds should be abolished, so I presume there were enough use cases where this was an issue. I faintly remember one article about the announcement of the voting results where they talked about the issues that smearing could cause but I don't remember exactly what it was, or where I read it.
  - sllabres 4y ago
    Voting running after midnight local time (where leap seconds get inserted) and dependent on sub-seconds, with only five leap seconds since 2000 introduced reads like a really interesting case.
- Ferret7446 4y ago
  I find it difficult to believe there are use cases affected by this; given that NTP normally has an error window of some milliseconds, those use cases would never have worked.
  - goodpoint 4y ago
    Wrong. The smearing cumulates during a day...
newaccount74 4y ago
Clocks are out of sync all the time, so any code that relies on precise timestamps needs some way to account for out-of-sync clocks anyway. So if you store two timestamps, you can't rely that the actual time that has passed between storing the two timestamps is equal to their difference, since the clock might have been slow, fast, or might have been synchronised with another clock between the recording of the two timestamps. I think dealing with leap seconds should work in a similar way as working with other clock synchronisation issues, so I really don't see the point of adding new types of clocks in unix.
DemocracyFTW2 4y ago
I think the article should not have left out the observation that in case you want to get to local time from UTC, you can only do so with certainty for the past, the present and the near future, because there could always be a change in daylight savings time (DST) regulations for a given locale. In my eyes that makes UTC a tiny little less useful for some purposes. OTOH human schedules should not be affected. Even for a country like [Egypt] which apparently has no issues with changing DST regulations with a mere few days of notice, meeting at 1:00pm in Cairo on some given day in the future will always mean whatever the local time stipulates it to be, come the day. OTOH I believe that for computers, adopting something like TAI for all timestamps would probably a good thing. Re-adjusting display dates for the past where needed is much less complex wrt leap seconds than it is for time zones and DST. The conceptual simplicity of TAI and the invariants afforded by it come with the slight disadvantage that "the future can not be predicted", but we already cannot predict the future of countries changing time zones and DST dates, so we'd not be any worse off in that regard. [Egypt](https://en.wikipedia.org/wiki/Daylight_saving_time_in_Egypt https://en.wikipedia.org/wiki/Daylight_saving_time_in_Egypt)
jesprenj 4y ago
EDIT: This comment was corrected by numerous replies below and the original comment contained incorrect information. Thanks to all posters for the corrections. Original comment: > The simplified definition of Unix time is that Unix time counts the amount of seconds that have passed since the first of January 1970 (which is referred to as “the epoch”, similar to the first of January of 1 AD being the epoch of the Gregorian calendar). This is of course not the complete definition, since it does not take leap seconds into consideration. I disagree with the statement that UNIX time is not the amount of seconds since the Epoch. Leap seconds never happened, they were added to UTC and UTC only in order to correct it for sunrise and sunset. If we started an experiment on 23:00 and ended it on 01:00 the next day, but a leap second occured in the mean time, the experiment wasn't running for 7200 seconds, but for 7199 seconds. The leap second had nothing to do with time counting, a stopwatch would not add a second to the reading. If we started an experiment on Epoch, the UNIX time tells us the amount of seconds this experiment has been running. If we added 86400 leap seconds in between, the experiment would count real time, not our made up time. One day wouldn't be added to the count. As far as I understand it, that's what UNIX time is, and there's nothing wrong with that. It's a feature. Please add your opinion.
- account42 4y ago
  Your understanding of Unix time is wrong. In your experiment the difference between the end and start Unix timestamps would be 7200 even though thats off from the experiment duration. That is exactly the problem: Unix time gets adjusted for leap seconds to stay in sync with UTC instead of real time.
- masklinn 4y ago
  > As far as I understand it, that's what UNIX time is Then you don’t understand Unix time. In the first case it would return 7200, and in the second case it would count “made up time”. UNIX time is 86400*days since epoch + seconds since midnight. It does not measure “real time”, whatever that is. > Leap seconds never happened, they were added to UTC and UTC only in order to correct it for sunrise and sunset. You’re apparently confusing leap seconds and DST. Leap seconds are used to compensate for minute variations in the speed at which the earth rotates (irregularities and an overall slowdown). So they serve to resynchronize atomic time (TAI) and solar time (UT1). This resynchronized version (UT1 ~ 1s) is UTC.
globular-toast 4y ago
The author is right. I couldn't believe it when I first learnt the difference between UTC and TAI that Unix time based on UTC. It makes no sense. And now they are talking about "correcting" UTC to make Unix time work? Thus defeating the entire point of UTC? There's something about time that people seem to find very difficult to reason with. It reminds me of the arguments between whether we should use standard time all year or daylight savings time all year.
atemerev 4y ago
Sorry, no. UNIX time is easy, works fine for most of cases and can be extended easily to the rest of the cases, and a single well-defined compromise approach. Replacing it with three different time representations is dangerous overengineering. We should all re-read "the rise of worse is better" until enlightenment (https://dreamsongs.com/RiseOfWorseIsBetter.html https://dreamsongs.com/RiseOfWorseIsBetter.html).
samwillis 4y ago
This is a good article, it's worth reading in full. It touches on the different use cases for different types of time/date storage but doesn't make it explicit enough that the is no "one size fits all" solution. It's true that internally a TIA timestamp is cleaner for storing a moment in time, but that is only half the issue. I like to think of it as a difference between physical time and human time. For engineering, science, precise measurement, simulation and such using TAI is clearly the cleaner, most robust solution. But as soon as you have a humans "perception" of time as part of the system a timezone aware timestamp is required. But even then, the most common version of that ISO 8601 is flawed. ISO 8601 only has a fixed offset, it has no concept of timezones or geolocation. Borders between timezones move, DST windows change or disappear, it has no way to account for that. What we need is an extended ISO 8601 that includes a location rather than offset. Human time is tied to a geographic location in time. I think we need something like this: 2022-11-23T08:52:02!Europe/London using the names from the TZ database: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones https://en.wikipedia.org/wiki/List_of_tz_database_time_zones or even coordinates: 2022-11-23T08:52:02!51.509865/-0.118092 we probably even want planetary body: 1969-07-20T20:17:00!Moon/0.67416/23.47314 This is the only way to store "human time" in a future proof way. Physical moment in time = TAI timestamp Human perception of a moment in time = Geolocated timestamp
- Someone 4y ago
  I expect you’ll find that a single spot on earth can have multiple values of the current time, depending on who you ask. For example, India, Pakistan, and China are each are in their own time zone, and (https://en.wikipedia.org/wiki/Jammu_and_Kashmir_(union_territory) https://en.wikipedia.org/wiki/Jammu_and_Kashmir_(union_terri...) “Jammu and Kashmir is a region administered by India as a union territory and consists of the southern portion of the larger Kashmir region, which has been the subject of a dispute between India and Pakistan since 1947, and between India and China since 1962” ⇒ I expect there are events that are recorded to have happened at three different times, depending on who you ask. https://en.wikipedia.org/wiki/List_of_territorial_disputes https://en.wikipedia.org/wiki/List_of_territorial_disputes has many other territorial disputes, including ones between Canada and the USA (https://en.wikipedia.org/wiki/List_of_areas_disputed_by_Canada_and_the_United_States https://en.wikipedia.org/wiki/List_of_areas_disputed_by_Cana...), but most of them won’t be across time zones. Also, there are other calendars in use than the Gregorian one. Most confusingly, some groups still use the Julian Calender.
beardyw 4y ago
Please don't bite, but why not move the UTC merdian?
- DemocracyFTW2 4y ago
  why not move the UTC merdian? You certainly mean to say the UTC nerdian, right? It's defined to cut through the median of all discussions around UTC, TAI, Unix timestamps and leap seconds that occur on HN and I see no reason to move it to another forum.
mytailorisrich 4y ago
There is an obvious need for time synchronized with the rotation of the Earth so leap seconds, or an equivalent mechanism, is also needed. The question is where to convert between linear time, more suited for computer systems, and Earth-synchronised time. In terms of computer system design I'd say that the best is usually to convert as late as possible in a presentation layer and to use linear time internally everywhere.
deleted 4y ago
[deleted]
deleted 4y ago
[deleted]
tommiegannert 4y ago
As much as I appreciate the topic, I fear this will be one of those 14 to 15 standards transitions. The proposal is complex in that it proposes three time units. I see a lower-bound case for two: sun/Earth time and universal time. Farmers must be able to express "I will plant seeds at noon on August 29, 2329". They care about how the sun affects Earth. The worker trying to schedule a meeting is the same, because they'll want to guarantee the meeting happens during daylight. As such, I find the obsession over "always use Unix timestamps" a bit weird. Yeah, they're simple, but semantically wonky for some practical things. Everyone else, dealing with durations and time deltas unrelated to geological conditions want TAI. For the farmer, the problem is the Gregorian calendar is deeply flawed/inaccurate, not that UTC has/had a leap second. Sun and Earth rotations are not aligned, so we cannot expect these to line up. We need year, day of year and time of day. The year is just the "number of solar turns" integer. The first and last day of the year would have to be cut up in unsavoury ways to account for misalignment. The time of day only needs to line up with Earth's rotation, but this means the speed relative TAI will need to shift as Earth is affected/slowed by the universe. (Now, I would suggest making the clock zenith based on something around the poles or leaning of the Earth axis, to end the Greenwich bias: that's pretty damn arbitrary.)
denton-scratch 4y ago
I hope that, when they redefine UTC, they define a new name for it. UTC has been the same timescale since it started; the history of redefinitions of GMT is a horror-story. I guess what I'm really saying is that UTC-without-LS won't be UTC; rather, we'd declare that the new universal civil-time timescale is to be UTC-without-LS. You could carry on computing old-fashioned UTC after the transition; but one wouldn't generally use UTC to render civil-time for display.
- sgtnoodle 4y ago
  Leap seconds are determined manually, though, so it seems like there's no practical difference other than UTC's error from solar time going out of spec?
  - denton-scratch 4y ago
    > other than UTC's error from solar time going out of spec? Well, yeah. Except that if it's not in-spec, it's not UTC; it's something else, and it shouldn't be called UTC. If you don't change the name, you would end up unable to compare two UTC times without also knowing which version of UTC the two times were given in.
    - sgtnoodle 4y ago
      I don't follow. UTC with no more leap second updates isn't a new version. It's the same timebase, just without any future updates to the leap second table. If the folk that maintain UTC decide to stop adding or subtracting leap seconds, it's not a fork in the timebase. That fork already happened with TAI. TAI was the same as universal time in 1958, and has progressed without any leap seconds. The current offset between TAI and UTC is 37 seconds. We could all agree to switch unix time to TAI, but there would be a 37 second discontinuity. I currently work on autonomous aircraft, and we use GPS time internally for avionics. The monotonicity is very nice for correctness when developing real time code. Outside of avionics, it's very confusing for people that tend to work in UTC. I've written table based timebase converters a few times now.
- 4y ago
shpx 4y ago
If we’re going to change the timestamp let’s do it correctly this time and start counting from 0001-01-01 (like “ticks” in C#) or 0000-01-01 because adding another number (1970) for other people to have to learn about and memorize is bad design and unnecessary overhead and one subgroup of one profession naming a whole time epoch was kind of pretentious. It would also have the added benefit of being hard to confuse a value in this timestamp with a Unix timestamp.
- gpderetta 4y ago
  > let’s do it correctly this time and start counting from 0001-01-01 [...] > one subgroup of one profession naming a whole time epoch was kind of pretentious. I think that the UNIX epoch is bless contentious that the other epoch, as is not tied to a religious event. OS wars not withstanding. Also the UNIX time was specifically designed for UNIX.
- sllabres 4y ago
  You know, that they used a different calendar back then, right?
hoseja 4y ago
Not to mention all these are measured inside the gravity well of Earth and Sun, introducing relativistic dilation (of, apparently, about 600 picoseconds per second [this value would have resulted in the first relativistic leap second in Unix Time this October]).
ilyt 4y ago
Anyone find it weird that they just abolished it without picking up replacement first ? Feels very "let have future us worry about that problem". It does feel very unscientific but ultimately the UTC won't get out of sync from rotation for more than a minute in my lifetime (unless desync rate increases I guess) so I don't really care
beeforpork 4y ago
These are 'interesting', but ultimately naive ideas, and the headline is presumptuous. Instead, the world needs agreement. And, wow!, an agreement among countries to simplify time keeping is an achievement! We do not need a more complex POSIX time spec, but we need to cope with billions of written lines of code from the past that use the existing API. We need that API to be as stable as possible to reduce complexity. We need to see that no changes are needed to existing code. We need to reduce complexity, e.g., by removing an irregular, unpredictable drift between TAI and UTC. We do not need to rewrite legacy code to use a shiny new API, but we need to make sure that legacy code will not break, because we do not know, unfortunately, were all that code hides.
- zowie_vd 4y ago
  Thank you for your opinions. The point of my proposal was to keep the API entirely backwards compatible, so rewrites would not be necessary. The important part is the "Legacy Unix Time" part. The other two definitions I provided simply serve that definition. I hope that you will agree that this is not that much more complicated. Notably I don't disagree with the idea of abolishing leap seconds, but I also don't think it's entirely agreeable to have the world's timekeeping standards changed solely because of engineering needs with regards to Unix time. In my opinion it's somewhat silly to have the timekeeping standard of the world changed because someone at Bell labs made an unfortunate design decision regarding Unix time, rather than changing Unix time itself.
  - beeforpork 4y ago
    > I hope that you will agree that this is not that much more complicated. Sorry, but I do not agree. The current Unix time API is good enough. Especially with the leap seconds gone. We need no more complication by innovation here. Any extensions inevitable lead to a lot of people starting to use those additional APIs, others trying to fix old code to use new APIs, and a few will start religiously fighting for the use of the new 'right' API. Software that handles time would be more diverse. And you will never live to see that old API gone anyway, because we don't know where it hides, so in the future, we would have to deal with bugs in the (mis)usage of more APIs. > In my opinion it's somewhat silly to have the timekeeping standard of the world changed because someone at Bell labs made an unfortunate design decision regarding Unix time, rather than changing Unix time itself. Well, maybe it is silly. But that's really irrelevant. Don't be stubborn to insist on API changes/extensions, because you think a past decision was wrong. You can't change reality anyway: the vastly bigger problem is changing Unix time, because code is written and is running in critical systems. And because leap seconds were inserted manually anyway, we already have a solution that works: don't insert any more leap second manually. Also note that the danger was to do something new: remove a second. That is a very dangerous experiment. E.g., the German DCF77 cannot even announce that (I think the UK and US time signals can, IIRC -- I am not sure about Japan and the other time signals). Of course you could start to postulate we need a new time signal format in Germany (and effectively half of Europe) to fix that, but that is just a much more dangerous approach.
tannernelson 4y ago
TIL that we can't know for sure the timestamp of now + some time span greater than 6 months.
- nly 4y ago
  You cannot know the delta, but the interpretation of a timestamp set 6 months in the future can be unambiguous E.g. "27th March 2023 19:00 Europe/London" is not ambiguous
zokier 4y ago
I wonder how bad of a timeformat would be just a bitfield struct with all of the parts(year/month/day/hour/minute/second/usec) broken out to separate fields. It should fit quite well to 64bits, so you could pass it along as an opaque 64bit integer value.

133 points by zowie_vd 4y ago

Is it not irrelevant for most use cases? Leap seconds or whatever. Think of the time of this post. It is stored but how important is the accuracy?

fhd2 4y ago
Timestamps are used for a lot of scheduling tasks that need to be reasonably precise. So an event happening one second earlier than expected, or even twice, could be a problem I suppose. Not exactly impossible to deal with, but I guess lots of people screw it up? Haven't really worked on systems where this _truly_ matters.
- rapsey 4y ago
  Scheduling tasks on a system is done using UTC/local time which the system keeps accurate. Unix time is not used for that.
  - butlerm 4y ago
    I am not aware of any operating system kernel that schedules tasks or timer expiration using a strict definition of UTC. Unix time is the most common, and local monotonic (non-decreasing) clocks are becoming common for that as well. Of course there are application level schedulers that use wall clock time in UTC format, and if you set the clock back your task may run twice, which is a more serious real world problem than running twice around a leap second. That is why no one who is not attached to civil time should use UTC or Unix time for short duration timer expiration in particular.
- spookie 4y ago
  Synchronization between computers is hard. https://www.gnu.org/software/coreutils/manual/coreutils.html#Date-input-formats https://www.gnu.org/software/coreutils/manual/coreutils.html...
- sohkamyung 4y ago
  Could it matter for high-frequency trading? Since these trades occur in milliseconds, I presume trading being off by a second would be considered very damaging.
  - gpderetta 4y ago
    It is not just very damaging, it also has legal implications. For example MIFID II regulations require at least 100us resolution for trade reporting. In practice PTP and GPS clocks are a requirement.
globular-toast 4y ago
That was my thought when considering the difficult and impossible tasks of converting to/from UTC-based timestamps. If we accept that UTC is only for human use then we should also accept that humans can't tell the time of day down to the second just based on their circadian rhythm and the Sun. Given a TAI-based timestamp a hundred years in the future we can be pretty certain in saying "yeah it will be around lunch time" (assuming there is no major changes in the solar system that would probably have such an impact on the lives of every human that nobody would care about this event anyway).
- account42 4y ago
  But if you follow that argument then you might as well drop leap secnds from UTC (which also drops them from Unix time). Which is what was done. So computers can have nice math and conversions to UTC are not needlessly complicated for little benefit to humans.
zowie_vd 4y ago
I agree that for many use cases the exact second is irrelevant, like submission times and things likes that. That being said, programmers do seem to expect, for example, that code like `(int)time(NULL) - previous_time` works and accurately gives the difference in time between now and whenever `previous_time` was given its value. This does work, but only 99.99% of the time. Then it breaks 0.01% of the time. For bug-free code it's important that the expectations of the programmer are in line with reality, and the problem with Unix time is that programmers' expectations of the mathematical properties of Unix time and the real mathematical properties of Unix time do not line up. This is what leads to the countless bugs involving leap seconds. The fact that accuracy is often not important is also crucial to my proposal of legacy Unix time, because legacy Unix time would eventually go out of sync with the rotation of the Earth just like TAI. In almost all cases this would practically be an aesthetic bug where dates in old user interfaces would be a few seconds off from the real date, and therefore not particularly harmful.
rgmerk 4y ago
For things like mission-critical logging, having time jump around, or even skew, is a real PITA.

Tenoke 4y ago

>The exact way Unix time handles leap seconds* is complex and an unnecessary detail for this article, but it will suffice to say that (i) whenever a positive leap second occurs, a timestamp is used twice, and (ii) if a negative leap second were to occur, a timestamp is skipped. Huh, I always assumed that Unix time just ignores them and ticks forward. I get why e.g. UTC needs them but why would it make sense for Unix time to account for them given how it's defined.

Cu3PO42 4y ago
If Unix time ignored them, the conversion between timestamps and UTC would no longer be possible with just arithmetic operations. You'd need a list of leap seconds and take them into account. Now one can argue if this is better or worse than the current solution, but it's not without its tradeoffs.
- jesprenj 4y ago
  Edit: My original comment is incorrect. Please read child comments. Original comment: Yes, to convert from UNIX time to UTC, you do need a list of leap seconds.
  - masklinn 4y ago
    No. You need a list of leap seconds to convert from Unix time to TAI.
  - valleyer 4y ago
    Others have already corrected you, but in case it makes it more concrete, here is the implementation of timegm() in musl (the libc used in Alpine Linux and some other Linux distributions). timegm() takes a time in UTC and converts it to Unix time. http://git.musl-libc.org/cgit/musl/tree/src/time/timegm.c http://git.musl-libc.org/cgit/musl/tree/src/time/timegm.c http://git.musl-libc.org/cgit/musl/tree/src/time/__tm_to_secs.c http://git.musl-libc.org/cgit/musl/tree/src/time/__tm_to_sec... You'll notice there is no handling or knowledge of leap seconds. (Leap years are handled, of course.)
- 4y ago

dunno7456 4y ago

Everything is bad and needs replacement :)

PeterisP 4y ago

Can someone comment on the practical usefulness of having UTC stay strictly synchronized with the rotation of the Earth? Intuitively it should be synchronized, but giving it more thought it is not entirely obvious why; e.g. my current local time zone is more than half an hour off of local solar time, and that isn't really a practical problem, so if UTC would be, say, five minutes off of actual solar time in Greenwich, what issues does it cause for people? Astronomy would be using specific time systems and/or adjustments anyway.

gerikson 4y ago
I see leap seconds as regulatory capture by astronomers :D Basically it's convenient for them to have civil time (UTC) never more than 0.5s from UT1.
- masklinn 4y ago
  Astronomers don’t care, they already need like 5 different clocks. Also the allowable variance between UTC and ut1 is 0.9.
  - PeterisP 4y ago
    Exactly - and if astronomers don't care, what other reasons are there to require accurate synchronization of UTC (and civil time) with earth's rotation?
    - lostmsu 4y ago
      Considering the rate of progress, timestamps might need to be replaced with reference-invariant coordinates.
  - moffkalast 4y ago
    Or one clock that can switch between 5 different times.
    - 4y ago

ZiiS 4y ago

Large cloud providers smear time. A day containing a leap second has the normal number of Unixtime "seconds" but each is 1/86400 too long. The inaccuracy this introduces is relevant to such a tiny tiny percentage of use cases; who already have to deal with the extreme complexities of time at this accuracy that everyone wins.

zowie_vd 4y ago
Large cloud providers are actually exactly the ones who argued that leap seconds should be abolished, so I presume there were enough use cases where this was an issue. I faintly remember one article about the announcement of the voting results where they talked about the issues that smearing could cause but I don't remember exactly what it was, or where I read it.
- sllabres 4y ago
  Voting running after midnight local time (where leap seconds get inserted) and dependent on sub-seconds, with only five leap seconds since 2000 introduced reads like a really interesting case.
Ferret7446 4y ago
I find it difficult to believe there are use cases affected by this; given that NTP normally has an error window of some milliseconds, those use cases would never have worked.
- goodpoint 4y ago
  Wrong. The smearing cumulates during a day...

newaccount74 4y ago

Clocks are out of sync all the time, so any code that relies on precise timestamps needs some way to account for out-of-sync clocks anyway. So if you store two timestamps, you can't rely that the actual time that has passed between storing the two timestamps is equal to their difference, since the clock might have been slow, fast, or might have been synchronised with another clock between the recording of the two timestamps. I think dealing with leap seconds should work in a similar way as working with other clock synchronisation issues, so I really don't see the point of adding new types of clocks in unix.

DemocracyFTW2 4y ago

I think the article should not have left out the observation that in case you want to get to local time from UTC, you can only do so with certainty for the past, the present and the near future, because there could always be a change in daylight savings time (DST) regulations for a given locale. In my eyes that makes UTC a tiny little less useful for some purposes. OTOH human schedules should not be affected. Even for a country like [Egypt] which apparently has no issues with changing DST regulations with a mere few days of notice, meeting at 1:00pm in Cairo on some given day in the future will always mean whatever the local time stipulates it to be, come the day. OTOH I believe that for computers, adopting something like TAI for all timestamps would probably a good thing. Re-adjusting display dates for the past where needed is much less complex wrt leap seconds than it is for time zones and DST. The conceptual simplicity of TAI and the invariants afforded by it come with the slight disadvantage that "the future can not be predicted", but we already cannot predict the future of countries changing time zones and DST dates, so we'd not be any worse off in that regard. [Egypt](https://en.wikipedia.org/wiki/Daylight_saving_time_in_Egypt https://en.wikipedia.org/wiki/Daylight_saving_time_in_Egypt)

jesprenj 4y ago

EDIT: This comment was corrected by numerous replies below and the original comment contained incorrect information. Thanks to all posters for the corrections. Original comment: > The simplified definition of Unix time is that Unix time counts the amount of seconds that have passed since the first of January 1970 (which is referred to as “the epoch”, similar to the first of January of 1 AD being the epoch of the Gregorian calendar). This is of course not the complete definition, since it does not take leap seconds into consideration. I disagree with the statement that UNIX time is not the amount of seconds since the Epoch. Leap seconds never happened, they were added to UTC and UTC only in order to correct it for sunrise and sunset. If we started an experiment on 23:00 and ended it on 01:00 the next day, but a leap second occured in the mean time, the experiment wasn't running for 7200 seconds, but for 7199 seconds. The leap second had nothing to do with time counting, a stopwatch would not add a second to the reading. If we started an experiment on Epoch, the UNIX time tells us the amount of seconds this experiment has been running. If we added 86400 leap seconds in between, the experiment would count real time, not our made up time. One day wouldn't be added to the count. As far as I understand it, that's what UNIX time is, and there's nothing wrong with that. It's a feature. Please add your opinion.

account42 4y ago
Your understanding of Unix time is wrong. In your experiment the difference between the end and start Unix timestamps would be 7200 even though thats off from the experiment duration. That is exactly the problem: Unix time gets adjusted for leap seconds to stay in sync with UTC instead of real time.
masklinn 4y ago
> As far as I understand it, that's what UNIX time is Then you don’t understand Unix time. In the first case it would return 7200, and in the second case it would count “made up time”. UNIX time is 86400*days since epoch + seconds since midnight. It does not measure “real time”, whatever that is. > Leap seconds never happened, they were added to UTC and UTC only in order to correct it for sunrise and sunset. You’re apparently confusing leap seconds and DST. Leap seconds are used to compensate for minute variations in the speed at which the earth rotates (irregularities and an overall slowdown). So they serve to resynchronize atomic time (TAI) and solar time (UT1). This resynchronized version (UT1 ~ 1s) is UTC.

globular-toast 4y ago

The author is right. I couldn't believe it when I first learnt the difference between UTC and TAI that Unix time based on UTC. It makes no sense. And now they are talking about "correcting" UTC to make Unix time work? Thus defeating the entire point of UTC? There's something about time that people seem to find very difficult to reason with. It reminds me of the arguments between whether we should use standard time all year or daylight savings time all year.

atemerev 4y ago

Sorry, no. UNIX time is easy, works fine for most of cases and can be extended easily to the rest of the cases, and a single well-defined compromise approach. Replacing it with three different time representations is dangerous overengineering. We should all re-read "the rise of worse is better" until enlightenment (https://dreamsongs.com/RiseOfWorseIsBetter.html https://dreamsongs.com/RiseOfWorseIsBetter.html).

samwillis 4y ago

This is a good article, it's worth reading in full. It touches on the different use cases for different types of time/date storage but doesn't make it explicit enough that the is no "one size fits all" solution. It's true that internally a TIA timestamp is cleaner for storing a moment in time, but that is only half the issue. I like to think of it as a difference between physical time and human time. For engineering, science, precise measurement, simulation and such using TAI is clearly the cleaner, most robust solution. But as soon as you have a humans "perception" of time as part of the system a timezone aware timestamp is required. But even then, the most common version of that ISO 8601 is flawed. ISO 8601 only has a fixed offset, it has no concept of timezones or geolocation. Borders between timezones move, DST windows change or disappear, it has no way to account for that. What we need is an extended ISO 8601 that includes a location rather than offset. Human time is tied to a geographic location in time. I think we need something like this: 2022-11-23T08:52:02!Europe/London using the names from the TZ database: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones https://en.wikipedia.org/wiki/List_of_tz_database_time_zones or even coordinates: 2022-11-23T08:52:02!51.509865/-0.118092 we probably even want planetary body: 1969-07-20T20:17:00!Moon/0.67416/23.47314 This is the only way to store "human time" in a future proof way. Physical moment in time = TAI timestamp Human perception of a moment in time = Geolocated timestamp

Someone 4y ago
I expect you’ll find that a single spot on earth can have multiple values of the current time, depending on who you ask. For example, India, Pakistan, and China are each are in their own time zone, and (https://en.wikipedia.org/wiki/Jammu_and_Kashmir_(union_territory) https://en.wikipedia.org/wiki/Jammu_and_Kashmir_(union_terri...) “Jammu and Kashmir is a region administered by India as a union territory and consists of the southern portion of the larger Kashmir region, which has been the subject of a dispute between India and Pakistan since 1947, and between India and China since 1962” ⇒ I expect there are events that are recorded to have happened at three different times, depending on who you ask. https://en.wikipedia.org/wiki/List_of_territorial_disputes https://en.wikipedia.org/wiki/List_of_territorial_disputes has many other territorial disputes, including ones between Canada and the USA (https://en.wikipedia.org/wiki/List_of_areas_disputed_by_Canada_and_the_United_States https://en.wikipedia.org/wiki/List_of_areas_disputed_by_Cana...), but most of them won’t be across time zones. Also, there are other calendars in use than the Gregorian one. Most confusingly, some groups still use the Julian Calender.

beardyw 4y ago

Please don't bite, but why not move the UTC merdian?

DemocracyFTW2 4y ago
why not move the UTC merdian? You certainly mean to say the UTC nerdian, right? It's defined to cut through the median of all discussions around UTC, TAI, Unix timestamps and leap seconds that occur on HN and I see no reason to move it to another forum.

mytailorisrich 4y ago

There is an obvious need for time synchronized with the rotation of the Earth so leap seconds, or an equivalent mechanism, is also needed. The question is where to convert between linear time, more suited for computer systems, and Earth-synchronised time. In terms of computer system design I'd say that the best is usually to convert as late as possible in a presentation layer and to use linear time internally everywhere.

deleted 4y ago

[deleted]

deleted 4y ago

[deleted]

tommiegannert 4y ago

As much as I appreciate the topic, I fear this will be one of those 14 to 15 standards transitions. The proposal is complex in that it proposes three time units. I see a lower-bound case for two: sun/Earth time and universal time. Farmers must be able to express "I will plant seeds at noon on August 29, 2329". They care about how the sun affects Earth. The worker trying to schedule a meeting is the same, because they'll want to guarantee the meeting happens during daylight. As such, I find the obsession over "always use Unix timestamps" a bit weird. Yeah, they're simple, but semantically wonky for some practical things. Everyone else, dealing with durations and time deltas unrelated to geological conditions want TAI. For the farmer, the problem is the Gregorian calendar is deeply flawed/inaccurate, not that UTC has/had a leap second. Sun and Earth rotations are not aligned, so we cannot expect these to line up. We need year, day of year and time of day. The year is just the "number of solar turns" integer. The first and last day of the year would have to be cut up in unsavoury ways to account for misalignment. The time of day only needs to line up with Earth's rotation, but this means the speed relative TAI will need to shift as Earth is affected/slowed by the universe. (Now, I would suggest making the clock zenith based on something around the poles or leaning of the Earth axis, to end the Greenwich bias: that's pretty damn arbitrary.)

denton-scratch 4y ago

I hope that, when they redefine UTC, they define a new name for it. UTC has been the same timescale since it started; the history of redefinitions of GMT is a horror-story. I guess what I'm really saying is that UTC-without-LS won't be UTC; rather, we'd declare that the new universal civil-time timescale is to be UTC-without-LS. You could carry on computing old-fashioned UTC after the transition; but one wouldn't generally use UTC to render civil-time for display.

sgtnoodle 4y ago
Leap seconds are determined manually, though, so it seems like there's no practical difference other than UTC's error from solar time going out of spec?
- denton-scratch 4y ago
  > other than UTC's error from solar time going out of spec? Well, yeah. Except that if it's not in-spec, it's not UTC; it's something else, and it shouldn't be called UTC. If you don't change the name, you would end up unable to compare two UTC times without also knowing which version of UTC the two times were given in.
  - sgtnoodle 4y ago
    I don't follow. UTC with no more leap second updates isn't a new version. It's the same timebase, just without any future updates to the leap second table. If the folk that maintain UTC decide to stop adding or subtracting leap seconds, it's not a fork in the timebase. That fork already happened with TAI. TAI was the same as universal time in 1958, and has progressed without any leap seconds. The current offset between TAI and UTC is 37 seconds. We could all agree to switch unix time to TAI, but there would be a 37 second discontinuity. I currently work on autonomous aircraft, and we use GPS time internally for avionics. The monotonicity is very nice for correctness when developing real time code. Outside of avionics, it's very confusing for people that tend to work in UTC. I've written table based timebase converters a few times now.
4y ago

shpx 4y ago

If we’re going to change the timestamp let’s do it correctly this time and start counting from 0001-01-01 (like “ticks” in C#) or 0000-01-01 because adding another number (1970) for other people to have to learn about and memorize is bad design and unnecessary overhead and one subgroup of one profession naming a whole time epoch was kind of pretentious. It would also have the added benefit of being hard to confuse a value in this timestamp with a Unix timestamp.

gpderetta 4y ago
> let’s do it correctly this time and start counting from 0001-01-01 [...] > one subgroup of one profession naming a whole time epoch was kind of pretentious. I think that the UNIX epoch is bless contentious that the other epoch, as is not tied to a religious event. OS wars not withstanding. Also the UNIX time was specifically designed for UNIX.
sllabres 4y ago
You know, that they used a different calendar back then, right?

hoseja 4y ago

Not to mention all these are measured inside the gravity well of Earth and Sun, introducing relativistic dilation (of, apparently, about 600 picoseconds per second [this value would have resulted in the first relativistic leap second in Unix Time this October]).

ilyt 4y ago

Anyone find it weird that they just abolished it without picking up replacement first ? Feels very "let have future us worry about that problem". It does feel very unscientific but ultimately the UTC won't get out of sync from rotation for more than a minute in my lifetime (unless desync rate increases I guess) so I don't really care

beeforpork 4y ago

These are 'interesting', but ultimately naive ideas, and the headline is presumptuous. Instead, the world needs agreement. And, wow!, an agreement among countries to simplify time keeping is an achievement! We do not need a more complex POSIX time spec, but we need to cope with billions of written lines of code from the past that use the existing API. We need that API to be as stable as possible to reduce complexity. We need to see that no changes are needed to existing code. We need to reduce complexity, e.g., by removing an irregular, unpredictable drift between TAI and UTC. We do not need to rewrite legacy code to use a shiny new API, but we need to make sure that legacy code will not break, because we do not know, unfortunately, were all that code hides.

zowie_vd 4y ago
Thank you for your opinions. The point of my proposal was to keep the API entirely backwards compatible, so rewrites would not be necessary. The important part is the "Legacy Unix Time" part. The other two definitions I provided simply serve that definition. I hope that you will agree that this is not that much more complicated. Notably I don't disagree with the idea of abolishing leap seconds, but I also don't think it's entirely agreeable to have the world's timekeeping standards changed solely because of engineering needs with regards to Unix time. In my opinion it's somewhat silly to have the timekeeping standard of the world changed because someone at Bell labs made an unfortunate design decision regarding Unix time, rather than changing Unix time itself.
- beeforpork 4y ago
  > I hope that you will agree that this is not that much more complicated. Sorry, but I do not agree. The current Unix time API is good enough. Especially with the leap seconds gone. We need no more complication by innovation here. Any extensions inevitable lead to a lot of people starting to use those additional APIs, others trying to fix old code to use new APIs, and a few will start religiously fighting for the use of the new 'right' API. Software that handles time would be more diverse. And you will never live to see that old API gone anyway, because we don't know where it hides, so in the future, we would have to deal with bugs in the (mis)usage of more APIs. > In my opinion it's somewhat silly to have the timekeeping standard of the world changed because someone at Bell labs made an unfortunate design decision regarding Unix time, rather than changing Unix time itself. Well, maybe it is silly. But that's really irrelevant. Don't be stubborn to insist on API changes/extensions, because you think a past decision was wrong. You can't change reality anyway: the vastly bigger problem is changing Unix time, because code is written and is running in critical systems. And because leap seconds were inserted manually anyway, we already have a solution that works: don't insert any more leap second manually. Also note that the danger was to do something new: remove a second. That is a very dangerous experiment. E.g., the German DCF77 cannot even announce that (I think the UK and US time signals can, IIRC -- I am not sure about Japan and the other time signals). Of course you could start to postulate we need a new time signal format in Germany (and effectively half of Europe) to fix that, but that is just a much more dangerous approach.

tannernelson 4y ago

TIL that we can't know for sure the timestamp of now + some time span greater than 6 months.

nly 4y ago
You cannot know the delta, but the interpretation of a timestamp set 6 months in the future can be unambiguous E.g. "27th March 2023 19:00 Europe/London" is not ambiguous

zokier 4y ago

I wonder how bad of a timeformat would be just a bitfield struct with all of the parts(year/month/day/hour/minute/second/usec) broken out to separate fields. It should fit quite well to 64bits, so you could pass it along as an opaque 64bit integer value.