8 ms·

How to save datetimes for future events

89 points by laut 12y ago

VBprogrammer 12y ago
If I understand the objection here the problem isn't that the computer has the time wrong, it's that our understanding of time is wrong. The computer still has the calendar entry at the right instant in time identified when the user input the entry.
- laut 12y ago
  The problem is that the meeting time is defined in terms of the "wall time" in Chile. So if you store the time in terms of UTC, then you are going to have problems when the relation between UTC and the Santiago/Chile timezone changes. The computer in the example has the time wrong because it stored the time in UTC.
  - igrekel 12y ago
    If it was done properly, the target date in the future should be based on the UTC offset it will have at that time in the future before being converted to UTC for storage. The error is to convert to UTC based on the current UTC offset instead of the "political" timezone that would take in to account proper daylight saving changes. There are problems when dealing with timezones and daylight saving changes but the example is just a case where the implementation was too naïve.
    - laut 12y ago
      Author here. Yes, the error was to save any other time than the local time where the meeting is supposed to take place. The point of the post was to demonstrate that when storing future events that is supposed to happen in a certain timezone, save the local time (along with timezone) instead of converting to UTC.
      - padobson 12y ago
        Why store the UTC offset too, then? Couldn't you get that with the timezone? Then you've got the wall time for the meeting saved, and if you need to convert it to UTC for some reason, you could do it with the most current rules for the timezone. If the offset becomes meaningless in the event of a rule change anyway, why save it when the most current rules will help you get the offset?
        laut 12y ago
        Good question! It is optional to store the UTC offset. The reason to store the UTC offset is to avoid ambiguity there might be when the timezone rules do not change. If the timezone rules don't change and you happen to be storing a datetime that is during changing from DST to non-DST, you specify which specific time you are talking about. In autumn, clocks could be set back from 3:00 to 2:00. 2:30 happens twice. When you schedule the time you can ask the user "2:30 summer time or 2:30 winter time"? So you only use the offset for anything in rare case there is an ambiguity (usually because of going off of DST), otherwise you just ignore it. In most cases even if the rules change like in the Chile example you would ignore it as well and use the offset provided by the timezone database.
    - mark-r 12y ago
      The problem is not naive software, the problem is that you can't predict the future. There will always be dates in the future that will be impacted by some rule change that hasn't been officially enacted yet.
  - sangnoir 12y ago
    Pegging against "wall time" in Chile causes it's own problems. What if the meeting is with a person in "Europe/Berlin" via Skype and the appointment is in a shared calendar. How would the new Berlin time be calculated for the unfortunate Berliner after the Chilean bureaucrats have had their fun?
    - pimlottc 12y ago
      The Berliners should have the meeting entered in their calendar as 10 AM Chilean time. At any given moment when the meeting time must be evaluated or displayed on their own calendars, it can be converted to European time using the current time zone conversion rules.
    - laut 12y ago
      In the example in the blog post the meeting takes place in Chile, so Berlin time is irrelevant. But let's say you are arranging a conference call between someone in Chile and someone in Berlin. When you arrange the meeting you have to define when the meeting takes place. So you have to choose a time zone along with a time for when the meeting starts. You could choose UTC, Berlin or Santiago. But you have to choose one.
  - VBprogrammer 12y ago
    Well, it has it wrong, depending on what you are using it for. If you are putting in a calendar entry for the next predicted Lunar eclipse storing them in anything but UTC is wrong.
    - laut 12y ago
      TAI would be better than UTC for the next Lunar eclipse in case you want to be exact to the second ;) But yes, if you schedule the meeting in UTC, use UTC. The point is not to convert from non-UTC to UTC.
      - arielby 12y ago
        UTC just provides a labeling for TAI/TT seconds, so a count-of-UTC-seconds timestamp is identical to a count-of-TAI-seconds timestamp (assuming the same epoch). Both are different from a count-of-non-leap-UTC-seconds aka POSIX time.
        taejo 12y ago
        Yes, but we don't know what that labeling will be in the future.
        laut 12y ago
        You cannot know for sure how many leap seconds there will be in for instance the next 5 years. So you cannot know for sure exactly how many seconds there will elapse from now until for instance July 1st 2020 midnight UTC.
bickfordb 12y ago
Don't most time zone databases store the date of rule changes? http://en.wikipedia.org/wiki/Tz_database#File_formats http://en.wikipedia.org/wiki/Tz_database#File_formats
- x1798DE 12y ago
  Yeah, this was always my understanding - time zones, DST participation and dates, etc all change more frequently than you'd expect, and time zone databases keep track of when things happen precisely for this reason. If that weren't happening, I think people would have found that many of their timestamps become obsolete / inaccurate relatively quickly. That said, it does tie the stability of your datetimestamp to the stability of the tz database. I think that's a pretty good bet, but it's nice to be aware of where your abstractions can leak.
- stygiansonic 12y ago
  That's what I thought too. Based on the discussion surrounding this example[0], I believe that's the case. Still, date/time is a tricky subject to get right. 0. http://stackoverflow.com/questions/6841333/why-is-subtracting-these-two-times-in-1927-giving-a-strange-result http://stackoverflow.com/questions/6841333/why-is-subtractin...
- mark-r 12y ago
  That doesn't help if you don't save the time zone rules that were in effect when you converted the original timestamp to UTC. The problem is that you've already converted to UTC using the wrong rules, so to make things right you need to unconvert then reconvert again. But there's no way to know which timestamps need correcting and which ones don't. The databases keep track of the zone changes so you can use the proper rules for conversions in the past. They don't protect you from changes in the future.
- laut 12y ago
  Yes. But in this case the timeline is something like this: 1) You arrange a meeting 2) Then, after you have arranged the meeting, politicians/bureaucrats decide to change the time zone rules 3) Your meeting takes place At 1) the rules are different from at 2). At 1) you cannot see into the future and anticipate how the rules will be at 2), because you don't have a time machine. So you have to be ready for it by implementing in a way that is resistant to timezone rule changes. For instance like the described solution.
yourapostasy 12y ago
Would it be possible to avoid any ambiguity at all by saving the timezone rules themselves as separate most-previous and current versions within the application context itself? Then when a new set of rules is detected (perhaps notification of updates at the OS level should be turned into a standardized publish-subscribe API to avoid too many applications polling for changes all the time, especially wasteful on battery-powered devices), move the current version to the previous version, scan for timezone rule differences, then scan your collection of records that use those timezones, parse them down to only those records affected by the new rules, and finally apply corrections to those records affected by the new rules. Extending further, if this approach works, perhaps an OS-level, perhaps git-backed storage of versions of timezone rules could avoid duplication of effort. Combined with a publish-subscribe service, it can auto-prune itself based upon the list of subscribed applications and when the oldest subscriber last hit the version store.
- laut 12y ago
  This post describes examples of ambiguity and non-existing times: http://www.creativedeletion.com/2015/01/28/falsehoods-programmers-date-time-zones.html http://www.creativedeletion.com/2015/01/28/falsehoods-progra... Note non-existing times mostly happen when you go from winter time to summer time. You set the clock ahead an hour at let's say 2:00 in the morning. So you go directly from 2:00 to 3:00. 2:30 does not exist. So let's say that you plan a meeting at 2:30 in the morning in a country that does not use DST. Then they decide to use DST and set the clocks forward one hour at 2:00. Your meeting was supposed to take place at 2:30 local time, but that time no longer exists! As I see it, the right thing to do is to alert the user of the problem.
- x1798DE 12y ago
  As pointed out in another post, the way timezone rule changes are stored actually does effectively what you are saying. For each time zone / locale, when there is a rule change, it is appended to the existing rules along with the date of the change. Example from the tzdata2015b file for the New_York time zone: # Zone NAME GMTOFF RULES FORMAT [UNTIL] Zone America/New_York -4:56:02 - LMT 1883 Nov 18 12:03:58 -5:00 US E%sT 1920 -5:00 NYC E%sT 1942 -5:00 US E%sT 1946 -5:00 NYC E%sT 1967 -5:00 US E%sT
abfan1127 12y ago
It seems that the example provided is a poor example. Software systems compute based on some rules. If you change the rules, you must change the software. Are there other more meaningful examples?
- laut 12y ago
  Author here. When the timezone rules change, the software updates its timezone database. This is how it should be. You don't have to change the software, the software should be able to handle the changes to the timezone database. The updates are available here: http://www.iana.org/time-zones http://www.iana.org/time-zones So far there has been two new releases of the database in 2015. Last year there were about 10 changes.
  - abfan1127 12y ago
    sorry, thats what I meant by the software (change its ability to understand the rules). IANA link is helpful!
- fryguy 12y ago
  The example is a good one, you are just misunderstanding what the problem is. Here is the synopsis: 1. User makes an appointment for 11am local time when he meets his friend in Chile. Using current timezone rules (GMT-3), the program saves this in the database as 1400 UTC. 2. Chile decides to change the rules for their time zone. Software gets updated with new table. Now, the appointment is at 10am local time in Chile, since they are GMT-4 now (1400 UTC = 10am GMT-4)
- mark-r 12y ago
  The example is perfect. It isn't enough to change the software, you must also change the existing data, because with the rule change it is now wrong. Or you can take the advice and store your data in a form that doesn't become obsolete when the rules change.
  - abfan1127 12y ago
    If you store it in UTC, it becomes obsolete when the rules change for that location. If you store it in the local time, it becomes obsolete for other timezones (think the timezone of the traveler before they leave). If you store both UTC and the local, then one (of the 2) becomes obsolete, but recoverable assuming you can scrub the data. However, you must know which is correct based on the rules change itself.
myfonj 12y ago
There is a great (humorous) Computerphile video about this topic: "The Problem with Time & Timezones" [0] [0] https://www.youtube.com/watch?v=-5wpm-gesOY https://www.youtube.com/watch?v=-5wpm-gesOY
fryguy 12y ago
Unfortunately, there's an ambiguity here. Suppose that you're going to watch the Super Bowl with your friends in Chile. So you set the appointment for 1830 local time in Chile. Then Chile changes their timezone, and your local time is wrong since the actual event is based on Eastern time, and not Chile time.
- mark-r 12y ago
  You simply need to observe the rule that event times should be specified in the time zone of the event itself. If the event is a meeting, then the meeting organizer will be in charge of setting the appropriate time zone.
  - icebraining 12y ago
    That doesn't sound simple for the user at all. If I'm meeting locally with friends to see the game/oscars/wtv, I don't know or care about the time in the original timezone. My local TV guide / newspaper shows it in local time, and that's what everyone in the meeting cares about, so why would I set it to a completely different TZ?
    - mark-r 12y ago
      Admittedly setting a time zone different than your local zone is an edge case. 99% of the time your local zone is the correct default. It would come up if you're scheduling a meeting for your trip to a different zone, or setting your TV to record the football game being broadcast from Chile.
- laut 12y ago
  That is outside of the scope of the article. In this case you have to ask yourself. Do you want to save an appointment for 18:30 in Chile? Or for the Superbowl? You have to choose one. If you really want to be sure tell your friends to use the timezone of the event. Maybe say "let's meet at 19:00 New York time" if the game starts at 20:00 in New York and you want to meet an hour before.
kbenson 12y ago
This comes from mismatched expectations of what users are storing. Often they think, or assume without thinking, that they are storing a localized time. For ease of implementation, or just because the developer doesn't know any better, we often store in absolute time (UTC) with an offset and call it done. The problem is, nobody really thinks about this stuff. It's easy to pick out an error case and say "The users expect local (relative) time, we need to deliver that." That's not always right. There are plenty of cases where the user really wants absolute time. Fro example, when scheduling an international meeting where the other side is dictating the time. When timing a broadcast from another country. Really, any time when you need to deal with events scheduled outside your local time, or have components (or people) that run on times that are not local to you. It gets trickier. What if you are tracking that international event, and that locality changes their time rules? For example, what if I'm in the US, and I wanted to schedule a time to watch a sporting event in Chile, and I scheduled it before the changed mentioned in the article? Whether I store it as a local time or an absolute time, it's wrong. Really, you need three pieces of information. The UTC time representation, the timezone offset it applies to, and whether it should be treated as an absolute or localized time. Edit: Corrected what type of stored offset needs to be kept
- mark-r 12y ago
  If you stored it as your local time, it would be wrong. If you stored it as Chile's local time, it would be correct. The trick is to simply record the appropriate time zone along with the timestamp.
  - kbenson 12y ago
    Absolutely correct. I was thinking that, even if we words said something else. :/ I've updated the last paragraph to reflect that.
  - kevin_thibedeau 12y ago
    There is still the problem of handling the effects of DST on repeating events. You have to know the base UTC offset and any additional seasonal offset for the local zone. Then it becomes an issue if you travel to a different locality that doesn't observe DST.
protomyth 12y ago
For future events, you expect to schedule a meeting / appointment at [Local Time] on [Date] at [Location]. If a time change happens between now and the future event (e.g. daylight savings time), it doesn't matter from the perspective of the meeting because you have a fixed time. So, I would for future events record [Local Time], [Date], and [Location]. Then make sure I have a [Location] -> [Time Zone] table. [Local Time] and [Date] would probably be combined into one field. The crux of the problem is shifting rules on time zone (and location is my time zone determinant), so recalculation for other time zones is going to have to be dynamic. Storing UTC or UTC offset doesn't help the problem and actually creates a false solution.
- logicuce 12y ago
  Spot on. You don't need the UTC at all to be stored. What matters is the local time at the venue. If its a virtual meeting (eg. over Skype) with participants from multiple time zones, it is safe to assume always one time zone will take precedence over others and time should be saved with respect to that. If its a natural event like a solar eclipse, it can be saved in UTC. There is always one time zone in which the event won't be adjusted. Just use that.
  - protomyth 12y ago
    "There is always one time zone in which the event won't be adjusted. Just use that." I agree with everything except I would modify that statement to be "There is always one location in the which the event won't be adjusted. Just use that." It seems like people are assuming time zone is fixed when it can change. The fixed point is the location which is used to determine the time zone. Natural events are trickier because they have different assumptions than human created events. UTC is probably best for those.
    - mark-r 12y ago
      You're correct, the relation between time zones and locations isn't fixed. Unfortunately the time zone databases don't have enough granularity, you're forced to choose the nearest location that shares your current time zone.
ska 12y ago
This doesn't really seem to solve the problem, just makes some of the error modes less surprising to the user. Surely the right approach to handling scheduling like this is (sticking with the calender example) a) initiating calender event is canonical (but may or may not be in the calenders local timezone) , but b) metadata about when the schedule was made and last updated is stored, and c) every future computation takes this metadata and looks up canonical (i.e. IANA) timezone/change information every time it touches the event, and d) a warning is given to any user, at any time their localization has a change in rules that might affect their apparent wall time, plus e) any changes in the localization of the initiating/owning calender trigger a warning for everyone. Am I missing something? [edit: besides the way to format this list...] (added - I should have been more clear that the server is only responsible for keeping track of the canonical event and updating clients if and when that changes. The clients will always be responsible for resolving locally apparent changes)
- kbenson 12y ago
  Yes, whether the location/context of the event is relative to you or some other location. I.e. Someone outside Chile creates an event to track the start of a football game in Chile before the change mentioned in the article. What you really want is an absolute time, and a localized timezone that applies to the event (which may not be your own).
  - ska 12y ago
    Ah, I wasn't clear enough ... I was thinking in that case the canonical calender event would be encoded in time in Chile, even if the local calender was in Germany or whatever. Looking back that could have been written much better so I've added a parenthetical.
burnt1ce 12y ago
I just did a project that implemented the same solution that author mentioned. For those who need a web fron-tend implementation, I would like to recommend Moment Timezone (http://momentjs.com/timezone/ http://momentjs.com/timezone/).
baddox 12y ago
> Instead of saving the time in UTC along with the time zone, developers can save what the user expects us to save: the wall time. Ie. what the clock on the wall will say. That's not at all foolproof, although it works for the case the author describes and probably works for most people most of the time. It wouldn't work if, for some reason, I'm entering a calendar event for something I know will happen exactly n hours from now.
- mark-r 12y ago
  In that case, go ahead and convert to UTC and save UTC as the relevant zone. In no case is timestamp+zone less flexible than UTC timestamp by itself.
- laut 12y ago
  The article is about saving datetimes. Usually calendars use datetime too. If you need to count down X hours you are not looking to save a datetime. Maybe use an egg timer or the countdown feature of a smartphone.
ldenneau 12y ago
This article is way overthinking it. It's better to simply store the UTC time of the event, and nothing else. Local time zone conversions can be done when displaying/editing. Presumably at the time you are reading your calendar, your local software knows the current rules to offset from GMT. We have Skype meetings all the time involving parties in different US timezones and/or different countries. There's no "place" or local time zone for the meeting. We schedule in UTC and our local software or calendar program knows what time zone we are in to display the correct local time.
- andrewfong 12y ago
  You missed the whole bit about the timezone rule changing in between when the event was last edited and when it was displayed.
- tghw 12y ago
  I think you're missing the point of the article. What he's getting at is that timezone rules change, so the function to go from local to UTC when you save the meeting is not the same as the function to go from UTC back to local at the time of the meeting.
  - ldenneau 12y ago
    I do get the point, but simply preserving the local time isn't the right answer. I'd argue that when you change your local timezone offset, then your future schedule is indeterminate and all events have to be reverified. What if the event is for an international flight -- is it still leaving or arriving at the same local time? How about that standing 9AM meeting -- is at 9 because of the corporate office in a different time zone calls in, so your local time breaks their calendar?
- mapgrep 12y ago
  "It's better to simply store the UTC time of the event, and nothing else." The whole point of the article is that /you cannot know with certainty the UTC time of a future event/ if you are starting with a wallclock time in a particular location. In this article, a 10:00 am meeting in Santiago has one UTC value prior to a new law, and a different UTC value after the new law. If you are saying everyone should "schedule in UTC" like you do, good luck imposing that on the world!
tantalor 12y ago
Another solution is "floating time" which disregards timezones entirely. This is an option in iCalendar datetime formats. In floating time, 10:00 in Chile is "10:00 in Chile". No timezone offset. https://www.ietf.org/rfc/rfc2445.txt https://www.ietf.org/rfc/rfc2445.txt https://scottworldblog.wordpress.com/2009/02/20/how-to-miss-your-flight-and-other-important-life-events-courtesy-of-ical/ https://scottworldblog.wordpress.com/2009/02/20/how-to-miss-...
gravypod 12y ago
Why not just use a unix timestamp and convert to the local time format of all users? Is that a poor solution for this? Most unix timestamp to Date converters already take DST into account. This also fixes the "MM/DD/YYYY", "DD/MM/YYYY", and "YYYY/MM/DD" fiasco as you can rely that the users computer is configured to the correct timezone/localization settings.
- andrewfong 12y ago
  Because the Unix timestamp is based on UTC and may not accurately capture the user's intent if the DST rules change in between when the timestamp is created and when the timestamp is read. That is, suppose I schedule an event on 2016 April 1, at 10am in San Francisco, California. Under current DST rules, this translates into 2016 April 1, 5pm UTC or a Unix timestamp of 1459530000. Now suppose DST is abolished in California between now and the event occurring. A Unix timestamp of 1459530000 would then correspond to 2016 April 1, at 9am in San Francisco, which is one hour too early. Another way to think about this: You have to apply the DST conversion rules applicable at the time the event was created, NOT the (potentially different) rules at the time the event occurs. But maintaining different sets of conversion rules based on when an event is scheduled is a lot more cumbersome than simply recording the local time and location.
jrochkind1 12y ago
Am I missing something, or does their solution not actually solve their specific example problem case at all? Saving local time plus offset is _exactly_ the same information as saving GMT, isn't it? You'd really need to save local datetime plus location... which they don't seem to actually be recommending? Of course, for actually triggering an alarm or whatever, you'd still need to plot this on the actual timeline somehow, and deal with periodically checking to see if it should be changed because of a DST or timezone rule change or whatever.
- laut 12y ago
  The important part is local time plus timezone. The offset is there only for the very rare case that you save a time that you know is ambiguous when you save it. And when DST rules don't change. Usually it happens when you change from DST to non-DST in the autumn and a certain hour happens twice. In the example, the offset will not be used. Only local time plus time zone. edit: I have updated the article to explain this part.
NamTaf 12y ago
What about saving the present time (the datetime when the information is recorded) and the time offset as calculated at that moment? In theory then it doesn't matter what happens, you're still able to adjust. The only ambiguity I can think of is if leap (seconds|hours|days) are introduced, but surely you just add them on to the delta too? Of course, this system fails when you suddenly travel anything approaching relativistic speeds, but I guess you could save your time delta as a velocity-time delta :) Edit: I'm not actually advocating this as a solution, I just found it an interesting thought experiment.
- mark-r 12y ago
  Time offset by itself isn't sufficient to know if the rules for calculating that time offset have changed. You need to know the rules themselves, and the easiest way to record that is with a time zone.
pinaceae 12y ago
this assumes both users use calendar apps that work the same way. so if google fixes gcal on mobile, but the other user puts it in (unfixed) outlook, they will still miss each other. date/time is hard.

89 points by laut 12y ago

If I understand the objection here the problem isn't that the computer has the time wrong, it's that our understanding of time is wrong. The computer still has the calendar entry at the right instant in time identified when the user input the entry.

laut 12y ago
The problem is that the meeting time is defined in terms of the "wall time" in Chile. So if you store the time in terms of UTC, then you are going to have problems when the relation between UTC and the Santiago/Chile timezone changes. The computer in the example has the time wrong because it stored the time in UTC.
- igrekel 12y ago
  If it was done properly, the target date in the future should be based on the UTC offset it will have at that time in the future before being converted to UTC for storage. The error is to convert to UTC based on the current UTC offset instead of the "political" timezone that would take in to account proper daylight saving changes. There are problems when dealing with timezones and daylight saving changes but the example is just a case where the implementation was too naïve.
  - laut 12y ago
    Author here. Yes, the error was to save any other time than the local time where the meeting is supposed to take place. The point of the post was to demonstrate that when storing future events that is supposed to happen in a certain timezone, save the local time (along with timezone) instead of converting to UTC.
    - padobson 12y ago
      Why store the UTC offset too, then? Couldn't you get that with the timezone? Then you've got the wall time for the meeting saved, and if you need to convert it to UTC for some reason, you could do it with the most current rules for the timezone. If the offset becomes meaningless in the event of a rule change anyway, why save it when the most current rules will help you get the offset?
      - laut 12y ago
        Good question! It is optional to store the UTC offset. The reason to store the UTC offset is to avoid ambiguity there might be when the timezone rules do not change. If the timezone rules don't change and you happen to be storing a datetime that is during changing from DST to non-DST, you specify which specific time you are talking about. In autumn, clocks could be set back from 3:00 to 2:00. 2:30 happens twice. When you schedule the time you can ask the user "2:30 summer time or 2:30 winter time"? So you only use the offset for anything in rare case there is an ambiguity (usually because of going off of DST), otherwise you just ignore it. In most cases even if the rules change like in the Chile example you would ignore it as well and use the offset provided by the timezone database.
  - mark-r 12y ago
    The problem is not naive software, the problem is that you can't predict the future. There will always be dates in the future that will be impacted by some rule change that hasn't been officially enacted yet.
- sangnoir 12y ago
  Pegging against "wall time" in Chile causes it's own problems. What if the meeting is with a person in "Europe/Berlin" via Skype and the appointment is in a shared calendar. How would the new Berlin time be calculated for the unfortunate Berliner after the Chilean bureaucrats have had their fun?
  - pimlottc 12y ago
    The Berliners should have the meeting entered in their calendar as 10 AM Chilean time. At any given moment when the meeting time must be evaluated or displayed on their own calendars, it can be converted to European time using the current time zone conversion rules.
  - laut 12y ago
    In the example in the blog post the meeting takes place in Chile, so Berlin time is irrelevant. But let's say you are arranging a conference call between someone in Chile and someone in Berlin. When you arrange the meeting you have to define when the meeting takes place. So you have to choose a time zone along with a time for when the meeting starts. You could choose UTC, Berlin or Santiago. But you have to choose one.
- VBprogrammer 12y ago
  Well, it has it wrong, depending on what you are using it for. If you are putting in a calendar entry for the next predicted Lunar eclipse storing them in anything but UTC is wrong.
  - laut 12y ago
    TAI would be better than UTC for the next Lunar eclipse in case you want to be exact to the second ;) But yes, if you schedule the meeting in UTC, use UTC. The point is not to convert from non-UTC to UTC.
    - arielby 12y ago
      UTC just provides a labeling for TAI/TT seconds, so a count-of-UTC-seconds timestamp is identical to a count-of-TAI-seconds timestamp (assuming the same epoch). Both are different from a count-of-non-leap-UTC-seconds aka POSIX time.
      - taejo 12y ago
        Yes, but we don't know what that labeling will be in the future.
      - laut 12y ago
        You cannot know for sure how many leap seconds there will be in for instance the next 5 years. So you cannot know for sure exactly how many seconds there will elapse from now until for instance July 1st 2020 midnight UTC.

bickfordb 12y ago

Don't most time zone databases store the date of rule changes? http://en.wikipedia.org/wiki/Tz_database#File_formats http://en.wikipedia.org/wiki/Tz_database#File_formats

x1798DE 12y ago
Yeah, this was always my understanding - time zones, DST participation and dates, etc all change more frequently than you'd expect, and time zone databases keep track of when things happen precisely for this reason. If that weren't happening, I think people would have found that many of their timestamps become obsolete / inaccurate relatively quickly. That said, it does tie the stability of your datetimestamp to the stability of the tz database. I think that's a pretty good bet, but it's nice to be aware of where your abstractions can leak.
stygiansonic 12y ago
That's what I thought too. Based on the discussion surrounding this example[0], I believe that's the case. Still, date/time is a tricky subject to get right. 0. http://stackoverflow.com/questions/6841333/why-is-subtracting-these-two-times-in-1927-giving-a-strange-result http://stackoverflow.com/questions/6841333/why-is-subtractin...
mark-r 12y ago
That doesn't help if you don't save the time zone rules that were in effect when you converted the original timestamp to UTC. The problem is that you've already converted to UTC using the wrong rules, so to make things right you need to unconvert then reconvert again. But there's no way to know which timestamps need correcting and which ones don't. The databases keep track of the zone changes so you can use the proper rules for conversions in the past. They don't protect you from changes in the future.
laut 12y ago
Yes. But in this case the timeline is something like this: 1) You arrange a meeting 2) Then, after you have arranged the meeting, politicians/bureaucrats decide to change the time zone rules 3) Your meeting takes place At 1) the rules are different from at 2). At 1) you cannot see into the future and anticipate how the rules will be at 2), because you don't have a time machine. So you have to be ready for it by implementing in a way that is resistant to timezone rule changes. For instance like the described solution.

yourapostasy 12y ago

Would it be possible to avoid any ambiguity at all by saving the timezone rules themselves as separate most-previous and current versions within the application context itself? Then when a new set of rules is detected (perhaps notification of updates at the OS level should be turned into a standardized publish-subscribe API to avoid too many applications polling for changes all the time, especially wasteful on battery-powered devices), move the current version to the previous version, scan for timezone rule differences, then scan your collection of records that use those timezones, parse them down to only those records affected by the new rules, and finally apply corrections to those records affected by the new rules. Extending further, if this approach works, perhaps an OS-level, perhaps git-backed storage of versions of timezone rules could avoid duplication of effort. Combined with a publish-subscribe service, it can auto-prune itself based upon the list of subscribed applications and when the oldest subscriber last hit the version store.

laut 12y ago
This post describes examples of ambiguity and non-existing times: http://www.creativedeletion.com/2015/01/28/falsehoods-programmers-date-time-zones.html http://www.creativedeletion.com/2015/01/28/falsehoods-progra... Note non-existing times mostly happen when you go from winter time to summer time. You set the clock ahead an hour at let's say 2:00 in the morning. So you go directly from 2:00 to 3:00. 2:30 does not exist. So let's say that you plan a meeting at 2:30 in the morning in a country that does not use DST. Then they decide to use DST and set the clocks forward one hour at 2:00. Your meeting was supposed to take place at 2:30 local time, but that time no longer exists! As I see it, the right thing to do is to alert the user of the problem.
x1798DE 12y ago
As pointed out in another post, the way timezone rule changes are stored actually does effectively what you are saying. For each time zone / locale, when there is a rule change, it is appended to the existing rules along with the date of the change. Example from the tzdata2015b file for the New_York time zone: # Zone NAME GMTOFF RULES FORMAT [UNTIL] Zone America/New_York -4:56:02 - LMT 1883 Nov 18 12:03:58 -5:00 US E%sT 1920 -5:00 NYC E%sT 1942 -5:00 US E%sT 1946 -5:00 NYC E%sT 1967 -5:00 US E%sT

abfan1127 12y ago

It seems that the example provided is a poor example. Software systems compute based on some rules. If you change the rules, you must change the software. Are there other more meaningful examples?

laut 12y ago
Author here. When the timezone rules change, the software updates its timezone database. This is how it should be. You don't have to change the software, the software should be able to handle the changes to the timezone database. The updates are available here: http://www.iana.org/time-zones http://www.iana.org/time-zones So far there has been two new releases of the database in 2015. Last year there were about 10 changes.
- abfan1127 12y ago
  sorry, thats what I meant by the software (change its ability to understand the rules). IANA link is helpful!
fryguy 12y ago
The example is a good one, you are just misunderstanding what the problem is. Here is the synopsis: 1. User makes an appointment for 11am local time when he meets his friend in Chile. Using current timezone rules (GMT-3), the program saves this in the database as 1400 UTC. 2. Chile decides to change the rules for their time zone. Software gets updated with new table. Now, the appointment is at 10am local time in Chile, since they are GMT-4 now (1400 UTC = 10am GMT-4)
mark-r 12y ago
The example is perfect. It isn't enough to change the software, you must also change the existing data, because with the rule change it is now wrong. Or you can take the advice and store your data in a form that doesn't become obsolete when the rules change.
- abfan1127 12y ago
  If you store it in UTC, it becomes obsolete when the rules change for that location. If you store it in the local time, it becomes obsolete for other timezones (think the timezone of the traveler before they leave). If you store both UTC and the local, then one (of the 2) becomes obsolete, but recoverable assuming you can scrub the data. However, you must know which is correct based on the rules change itself.

myfonj 12y ago

There is a great (humorous) Computerphile video about this topic: "The Problem with Time & Timezones" [0] [0] https://www.youtube.com/watch?v=-5wpm-gesOY https://www.youtube.com/watch?v=-5wpm-gesOY

fryguy 12y ago

Unfortunately, there's an ambiguity here. Suppose that you're going to watch the Super Bowl with your friends in Chile. So you set the appointment for 1830 local time in Chile. Then Chile changes their timezone, and your local time is wrong since the actual event is based on Eastern time, and not Chile time.

mark-r 12y ago
You simply need to observe the rule that event times should be specified in the time zone of the event itself. If the event is a meeting, then the meeting organizer will be in charge of setting the appropriate time zone.
- icebraining 12y ago
  That doesn't sound simple for the user at all. If I'm meeting locally with friends to see the game/oscars/wtv, I don't know or care about the time in the original timezone. My local TV guide / newspaper shows it in local time, and that's what everyone in the meeting cares about, so why would I set it to a completely different TZ?
  - mark-r 12y ago
    Admittedly setting a time zone different than your local zone is an edge case. 99% of the time your local zone is the correct default. It would come up if you're scheduling a meeting for your trip to a different zone, or setting your TV to record the football game being broadcast from Chile.
laut 12y ago
That is outside of the scope of the article. In this case you have to ask yourself. Do you want to save an appointment for 18:30 in Chile? Or for the Superbowl? You have to choose one. If you really want to be sure tell your friends to use the timezone of the event. Maybe say "let's meet at 19:00 New York time" if the game starts at 20:00 in New York and you want to meet an hour before.

kbenson 12y ago

This comes from mismatched expectations of what users are storing. Often they think, or assume without thinking, that they are storing a localized time. For ease of implementation, or just because the developer doesn't know any better, we often store in absolute time (UTC) with an offset and call it done. The problem is, nobody really thinks about this stuff. It's easy to pick out an error case and say "The users expect local (relative) time, we need to deliver that." That's not always right. There are plenty of cases where the user really wants absolute time. Fro example, when scheduling an international meeting where the other side is dictating the time. When timing a broadcast from another country. Really, any time when you need to deal with events scheduled outside your local time, or have components (or people) that run on times that are not local to you. It gets trickier. What if you are tracking that international event, and that locality changes their time rules? For example, what if I'm in the US, and I wanted to schedule a time to watch a sporting event in Chile, and I scheduled it before the changed mentioned in the article? Whether I store it as a local time or an absolute time, it's wrong. Really, you need three pieces of information. The UTC time representation, the timezone offset it applies to, and whether it should be treated as an absolute or localized time. Edit: Corrected what type of stored offset needs to be kept

mark-r 12y ago
If you stored it as your local time, it would be wrong. If you stored it as Chile's local time, it would be correct. The trick is to simply record the appropriate time zone along with the timestamp.
- kbenson 12y ago
  Absolutely correct. I was thinking that, even if we words said something else. :/ I've updated the last paragraph to reflect that.
- kevin_thibedeau 12y ago
  There is still the problem of handling the effects of DST on repeating events. You have to know the base UTC offset and any additional seasonal offset for the local zone. Then it becomes an issue if you travel to a different locality that doesn't observe DST.

protomyth 12y ago

For future events, you expect to schedule a meeting / appointment at [Local Time] on [Date] at [Location]. If a time change happens between now and the future event (e.g. daylight savings time), it doesn't matter from the perspective of the meeting because you have a fixed time. So, I would for future events record [Local Time], [Date], and [Location]. Then make sure I have a [Location] -> [Time Zone] table. [Local Time] and [Date] would probably be combined into one field. The crux of the problem is shifting rules on time zone (and location is my time zone determinant), so recalculation for other time zones is going to have to be dynamic. Storing UTC or UTC offset doesn't help the problem and actually creates a false solution.

logicuce 12y ago
Spot on. You don't need the UTC at all to be stored. What matters is the local time at the venue. If its a virtual meeting (eg. over Skype) with participants from multiple time zones, it is safe to assume always one time zone will take precedence over others and time should be saved with respect to that. If its a natural event like a solar eclipse, it can be saved in UTC. There is always one time zone in which the event won't be adjusted. Just use that.
- protomyth 12y ago
  "There is always one time zone in which the event won't be adjusted. Just use that." I agree with everything except I would modify that statement to be "There is always one location in the which the event won't be adjusted. Just use that." It seems like people are assuming time zone is fixed when it can change. The fixed point is the location which is used to determine the time zone. Natural events are trickier because they have different assumptions than human created events. UTC is probably best for those.
  - mark-r 12y ago
    You're correct, the relation between time zones and locations isn't fixed. Unfortunately the time zone databases don't have enough granularity, you're forced to choose the nearest location that shares your current time zone.

ska 12y ago

This doesn't really seem to solve the problem, just makes some of the error modes less surprising to the user. Surely the right approach to handling scheduling like this is (sticking with the calender example) a) initiating calender event is canonical (but may or may not be in the calenders local timezone) , but b) metadata about when the schedule was made and last updated is stored, and c) every future computation takes this metadata and looks up canonical (i.e. IANA) timezone/change information every time it touches the event, and d) a warning is given to any user, at any time their localization has a change in rules that might affect their apparent wall time, plus e) any changes in the localization of the initiating/owning calender trigger a warning for everyone. Am I missing something? [edit: besides the way to format this list...] (added - I should have been more clear that the server is only responsible for keeping track of the canonical event and updating clients if and when that changes. The clients will always be responsible for resolving locally apparent changes)

kbenson 12y ago
Yes, whether the location/context of the event is relative to you or some other location. I.e. Someone outside Chile creates an event to track the start of a football game in Chile before the change mentioned in the article. What you really want is an absolute time, and a localized timezone that applies to the event (which may not be your own).
- ska 12y ago
  Ah, I wasn't clear enough ... I was thinking in that case the canonical calender event would be encoded in time in Chile, even if the local calender was in Germany or whatever. Looking back that could have been written much better so I've added a parenthetical.

burnt1ce 12y ago

I just did a project that implemented the same solution that author mentioned. For those who need a web fron-tend implementation, I would like to recommend Moment Timezone (http://momentjs.com/timezone/ http://momentjs.com/timezone/).

baddox 12y ago

> Instead of saving the time in UTC along with the time zone, developers can save what the user expects us to save: the wall time. Ie. what the clock on the wall will say. That's not at all foolproof, although it works for the case the author describes and probably works for most people most of the time. It wouldn't work if, for some reason, I'm entering a calendar event for something I know will happen exactly n hours from now.

mark-r 12y ago
In that case, go ahead and convert to UTC and save UTC as the relevant zone. In no case is timestamp+zone less flexible than UTC timestamp by itself.
laut 12y ago
The article is about saving datetimes. Usually calendars use datetime too. If you need to count down X hours you are not looking to save a datetime. Maybe use an egg timer or the countdown feature of a smartphone.

ldenneau 12y ago

This article is way overthinking it. It's better to simply store the UTC time of the event, and nothing else. Local time zone conversions can be done when displaying/editing. Presumably at the time you are reading your calendar, your local software knows the current rules to offset from GMT. We have Skype meetings all the time involving parties in different US timezones and/or different countries. There's no "place" or local time zone for the meeting. We schedule in UTC and our local software or calendar program knows what time zone we are in to display the correct local time.

andrewfong 12y ago
You missed the whole bit about the timezone rule changing in between when the event was last edited and when it was displayed.
tghw 12y ago
I think you're missing the point of the article. What he's getting at is that timezone rules change, so the function to go from local to UTC when you save the meeting is not the same as the function to go from UTC back to local at the time of the meeting.
- ldenneau 12y ago
  I do get the point, but simply preserving the local time isn't the right answer. I'd argue that when you change your local timezone offset, then your future schedule is indeterminate and all events have to be reverified. What if the event is for an international flight -- is it still leaving or arriving at the same local time? How about that standing 9AM meeting -- is at 9 because of the corporate office in a different time zone calls in, so your local time breaks their calendar?
mapgrep 12y ago
"It's better to simply store the UTC time of the event, and nothing else." The whole point of the article is that /you cannot know with certainty the UTC time of a future event/ if you are starting with a wallclock time in a particular location. In this article, a 10:00 am meeting in Santiago has one UTC value prior to a new law, and a different UTC value after the new law. If you are saying everyone should "schedule in UTC" like you do, good luck imposing that on the world!

tantalor 12y ago

Another solution is "floating time" which disregards timezones entirely. This is an option in iCalendar datetime formats. In floating time, 10:00 in Chile is "10:00 in Chile". No timezone offset. https://www.ietf.org/rfc/rfc2445.txt https://www.ietf.org/rfc/rfc2445.txt https://scottworldblog.wordpress.com/2009/02/20/how-to-miss-your-flight-and-other-important-life-events-courtesy-of-ical/ https://scottworldblog.wordpress.com/2009/02/20/how-to-miss-...

gravypod 12y ago

Why not just use a unix timestamp and convert to the local time format of all users? Is that a poor solution for this? Most unix timestamp to Date converters already take DST into account. This also fixes the "MM/DD/YYYY", "DD/MM/YYYY", and "YYYY/MM/DD" fiasco as you can rely that the users computer is configured to the correct timezone/localization settings.

andrewfong 12y ago
Because the Unix timestamp is based on UTC and may not accurately capture the user's intent if the DST rules change in between when the timestamp is created and when the timestamp is read. That is, suppose I schedule an event on 2016 April 1, at 10am in San Francisco, California. Under current DST rules, this translates into 2016 April 1, 5pm UTC or a Unix timestamp of 1459530000. Now suppose DST is abolished in California between now and the event occurring. A Unix timestamp of 1459530000 would then correspond to 2016 April 1, at 9am in San Francisco, which is one hour too early. Another way to think about this: You have to apply the DST conversion rules applicable at the time the event was created, NOT the (potentially different) rules at the time the event occurs. But maintaining different sets of conversion rules based on when an event is scheduled is a lot more cumbersome than simply recording the local time and location.

jrochkind1 12y ago

Am I missing something, or does their solution not actually solve their specific example problem case at all? Saving local time plus offset is _exactly_ the same information as saving GMT, isn't it? You'd really need to save local datetime plus location... which they don't seem to actually be recommending? Of course, for actually triggering an alarm or whatever, you'd still need to plot this on the actual timeline somehow, and deal with periodically checking to see if it should be changed because of a DST or timezone rule change or whatever.

laut 12y ago
The important part is local time plus timezone. The offset is there only for the very rare case that you save a time that you know is ambiguous when you save it. And when DST rules don't change. Usually it happens when you change from DST to non-DST in the autumn and a certain hour happens twice. In the example, the offset will not be used. Only local time plus time zone. edit: I have updated the article to explain this part.

NamTaf 12y ago

What about saving the present time (the datetime when the information is recorded) and the time offset as calculated at that moment? In theory then it doesn't matter what happens, you're still able to adjust. The only ambiguity I can think of is if leap (seconds|hours|days) are introduced, but surely you just add them on to the delta too? Of course, this system fails when you suddenly travel anything approaching relativistic speeds, but I guess you could save your time delta as a velocity-time delta :) Edit: I'm not actually advocating this as a solution, I just found it an interesting thought experiment.

mark-r 12y ago
Time offset by itself isn't sufficient to know if the rules for calculating that time offset have changed. You need to know the rules themselves, and the easiest way to record that is with a time zone.

pinaceae 12y ago

this assumes both users use calendar apps that work the same way. so if google fixes gcal on mobile, but the other user puts it in (unfixed) outlook, they will still miss each other. date/time is hard.