5 ms·

The main issue I have with existing time libraries, in Rust or other ecosystems is poor support for leap seconds. This is mostly caused by using UNIX timestamps

The main issue I have with existing time libraries, in Rust or other ecosystems is poor support for leap seconds. This is mostly caused by using UNIX timestamps instead of TAI internally, and this lib is no different unfortunately. There seems to be some way to support it with tzif files, but it does not have first-class support. Here is the relevant Jiff issue with more details: https://github.com/BurntSushi/jiff/issues/7 https://github.com/BurntSushi/jiff/issues/7 UNIX timestamps don't use the SI second definition (instead it's 1/86000th of the current day, all UNIX seconds don't have the same duration) which breaks all correct duration computations. I understand that the tradeoff was to inherit compat with older time tracking methods and enable faster calendar formatting, I disagree that it was the right trade-off. UNIX timestamps mix representation concerns with data. In my opinion, leap seconds should be treated exactly like the 29th of February or time zones.

burntsushi 2y ago
Why do you want it? What's your use case? And why doesn't a specialized scientific library like `hifitime` work for your use case? Whenever people talk about leap seconds, it always seems to be in some abstract notion. But it's very rare to see folks connect them to real world use cases. I get the scientific use case, and I feel like that's well served by specialized libraries. Do we need anything else? I'm not sure that we do.
- demurgos 2y ago
  This is about the "pit of success", being correct and predictable by default. A difference of timestamps not returning the corresponding elapsed wall-time is _very_ surprising. I want to be able to compute durations between timestamps stored in the DB, received from API calls or retrieved from the system and get the right duration "out of the box". Computing these durations lets me apply business logic relying on it. A message can be editable for x amount of time, a token is valid for y amount of time, a sanction expires after z amount of time, etc. For example, I want to issue some token valid for 60s. What should I set the expiry time to? `now + 60s`. Except if `now` is 2016-12-31T23:59:30Z, then most libs will return a time 61s in the future. 1 second may not be a big error, but it's still an error and depending on context it may be relevant. This is a systematic error unrelated to time sync / precision concerns, so it's pretty frustrating to see it being so common. It seems though that we won't have any new leap seconds in the near future so eventually it will just become a curiosity from the past and we'll be stuck with a constant offset between UNIX and TAI. > I feel like that's well served by specialized libraries. Agreed that you need a specialized lib for this, but my point is that _you shouldn't have to_ and the current situation is a failure of software engineering. Computing `t2 - t1` in a model assuming global synchronized time should not be hard. I don't mean it as a personal critique, this is not an easy problem solve since UNIX timestamps are baked almost everywhere. It's just disappointing that we still have to deal with this.
  - iknowstuff 2y ago
    Eh, is this ideal with chasing when it’s all fucked anyway when we start shipping code to Mars? https://en.m.wikipedia.org/wiki/Barycentric_Coordinate_Time https://en.m.wikipedia.org/wiki/Barycentric_Coordinate_Time
  - burntsushi 2y ago
    What I'm not clear on though is what the failure mode is in your scenario. What happens when it's wrong? Does something bad happen? If something is one second longer or shorter than what it ought to be on very rare occasions, then what does wrong? I would, for example, imagine that the business use case of "editable for x amount of time" would be perfectly fine with that being plus-or-minus 1 second. It's not just about correctness, it's about the error and what it means. A few months ago, Jiff did have leap second support. It worked. I know how to do it, but none of the arguments in its favor seem to justify its complexity. Especially when specialized libraries exist for it. You can't look at this in a vacuum. By make a general purpose datetime library more complex, you risk the introduction of new and different types of errors by users of the library that could be much worse than the errors introduced by missing leap second support.
    - demurgos 2y ago
      > You can't look at this in a vacuum. By make a general purpose datetime library more complex, you risk the introduction of new and different types of errors by users of the library that could be much worse than the errors introduced by missing leap second support. Agreed, I can easily imagine that it could cause situation where some numeric value is not interpreted correctly and it causes a constant offset of 37 seconds. UNIX timestamps are entrenched, so deviating from it introduces misuse risks. Regarding my use-cases, I agree that these ones should still work fine. I could also come up with issues where a 1s error is more meaningful, but they would be artificial. The main problem I can see is using some absolute timestamp instead of a more precise timer in a higher frequency context. Overall, it's the general discussion about correctness VS "good enough". I consider that the extra complexity in a lib is warranted if it means less edge cases.
- LegionMammal978 2y ago
  It's not really in Rust (at least, not unless I choose to rewrite it in WASM), but there is one case where a lack of leap-second support has really been a thorn in my side. I've been working on and off on a JS web application that performs certain high-precision astronomical calculations, and leap seconds are a big challenge w.r.t. future correctness. Ephemerides don't change very much, but leap seconds can. I can embed a big list into my application, but then I have to remember to periodically update it until 2035, or maybe longer if they decide to keep leap seconds after all. So I can try to download a leap-seconds.list from somewhere, except that only a few sources set "Access-Control-Allow-Origin: *", and none promise to keep it set indefinitely. The annoying part is, applications on both Windows (since 10) and Unix-like systems generally have access to up-to-date leap-second information; at worst, browsers can be expected to be updated regularly. But since the JS standards devs want to stick strictly to POSIX time, they provide no means to obtain this information from the inside, through Date, Intl.Locale, or the proposed Temporal. This is all to say, I would really love to always have some way available to get "all leap-second info that the current system knows about, if it does have any such info". For its part, hifitime either downloads from a fixed URL (which doesn't have that Access-Control-Allow-Origin header IIRC), consults a fixed list, or gets the user to locate the info, which isn't nearly as general or foolproof as it could be.
  - burntsushi 2y ago
    Yeah the way Jiff's leap second support worked (before I ripped it out) is that it would just read from `/usr/share/zoneinfo/leap-seconds.list` (or `/usr/share/zoneinfo/leapseconds`, whichever was available) and use that with some rudimentary caching. That way, Jiff isn't responsible for keeping the list up-to-date. The sysadmin is. Just like what we do for tzdb.
    - LegionMammal978 2y ago
      Indeed. The problem is, in the absence of one good solution like that, there are no good solutions, as is the case with web JS. Currently (on the desktop or laptop), the best bet is hifitime's leap-seconds parser, but then the programmer is still responsible for coming up with a sane fallback path for all their target systems. (Which can be tricky for individuals to do, e.g., I have no way to tell whether the tz database is in an expected location on macOS or iOS.)

burntsushi 2y ago

Why do you want it? What's your use case? And why doesn't a specialized scientific library like `hifitime` work for your use case? Whenever people talk about leap seconds, it always seems to be in some abstract notion. But it's very rare to see folks connect them to real world use cases. I get the scientific use case, and I feel like that's well served by specialized libraries. Do we need anything else? I'm not sure that we do.

demurgos 2y ago
This is about the "pit of success", being correct and predictable by default. A difference of timestamps not returning the corresponding elapsed wall-time is _very_ surprising. I want to be able to compute durations between timestamps stored in the DB, received from API calls or retrieved from the system and get the right duration "out of the box". Computing these durations lets me apply business logic relying on it. A message can be editable for x amount of time, a token is valid for y amount of time, a sanction expires after z amount of time, etc. For example, I want to issue some token valid for 60s. What should I set the expiry time to? `now + 60s`. Except if `now` is 2016-12-31T23:59:30Z, then most libs will return a time 61s in the future. 1 second may not be a big error, but it's still an error and depending on context it may be relevant. This is a systematic error unrelated to time sync / precision concerns, so it's pretty frustrating to see it being so common. It seems though that we won't have any new leap seconds in the near future so eventually it will just become a curiosity from the past and we'll be stuck with a constant offset between UNIX and TAI. > I feel like that's well served by specialized libraries. Agreed that you need a specialized lib for this, but my point is that _you shouldn't have to_ and the current situation is a failure of software engineering. Computing `t2 - t1` in a model assuming global synchronized time should not be hard. I don't mean it as a personal critique, this is not an easy problem solve since UNIX timestamps are baked almost everywhere. It's just disappointing that we still have to deal with this.
- iknowstuff 2y ago
  Eh, is this ideal with chasing when it’s all fucked anyway when we start shipping code to Mars? https://en.m.wikipedia.org/wiki/Barycentric_Coordinate_Time https://en.m.wikipedia.org/wiki/Barycentric_Coordinate_Time
- burntsushi 2y ago
  What I'm not clear on though is what the failure mode is in your scenario. What happens when it's wrong? Does something bad happen? If something is one second longer or shorter than what it ought to be on very rare occasions, then what does wrong? I would, for example, imagine that the business use case of "editable for x amount of time" would be perfectly fine with that being plus-or-minus 1 second. It's not just about correctness, it's about the error and what it means. A few months ago, Jiff did have leap second support. It worked. I know how to do it, but none of the arguments in its favor seem to justify its complexity. Especially when specialized libraries exist for it. You can't look at this in a vacuum. By make a general purpose datetime library more complex, you risk the introduction of new and different types of errors by users of the library that could be much worse than the errors introduced by missing leap second support.
  - demurgos 2y ago
    > You can't look at this in a vacuum. By make a general purpose datetime library more complex, you risk the introduction of new and different types of errors by users of the library that could be much worse than the errors introduced by missing leap second support. Agreed, I can easily imagine that it could cause situation where some numeric value is not interpreted correctly and it causes a constant offset of 37 seconds. UNIX timestamps are entrenched, so deviating from it introduces misuse risks. Regarding my use-cases, I agree that these ones should still work fine. I could also come up with issues where a 1s error is more meaningful, but they would be artificial. The main problem I can see is using some absolute timestamp instead of a more precise timer in a higher frequency context. Overall, it's the general discussion about correctness VS "good enough". I consider that the extra complexity in a lib is warranted if it means less edge cases.
LegionMammal978 2y ago
It's not really in Rust (at least, not unless I choose to rewrite it in WASM), but there is one case where a lack of leap-second support has really been a thorn in my side. I've been working on and off on a JS web application that performs certain high-precision astronomical calculations, and leap seconds are a big challenge w.r.t. future correctness. Ephemerides don't change very much, but leap seconds can. I can embed a big list into my application, but then I have to remember to periodically update it until 2035, or maybe longer if they decide to keep leap seconds after all. So I can try to download a leap-seconds.list from somewhere, except that only a few sources set "Access-Control-Allow-Origin: *", and none promise to keep it set indefinitely. The annoying part is, applications on both Windows (since 10) and Unix-like systems generally have access to up-to-date leap-second information; at worst, browsers can be expected to be updated regularly. But since the JS standards devs want to stick strictly to POSIX time, they provide no means to obtain this information from the inside, through Date, Intl.Locale, or the proposed Temporal. This is all to say, I would really love to always have some way available to get "all leap-second info that the current system knows about, if it does have any such info". For its part, hifitime either downloads from a fixed URL (which doesn't have that Access-Control-Allow-Origin header IIRC), consults a fixed list, or gets the user to locate the info, which isn't nearly as general or foolproof as it could be.
- burntsushi 2y ago
  Yeah the way Jiff's leap second support worked (before I ripped it out) is that it would just read from `/usr/share/zoneinfo/leap-seconds.list` (or `/usr/share/zoneinfo/leapseconds`, whichever was available) and use that with some rudimentary caching. That way, Jiff isn't responsible for keeping the list up-to-date. The sysadmin is. Just like what we do for tzdb.
  - LegionMammal978 2y ago
    Indeed. The problem is, in the absence of one good solution like that, there are no good solutions, as is the case with web JS. Currently (on the desktop or laptop), the best bet is hifitime's leap-seconds parser, but then the programmer is still responsible for coming up with a sane fallback path for all their target systems. (Which can be tricky for individuals to do, e.g., I have no way to tell whether the tz database is in an expected location on macOS or iOS.)