6 ms·

Python Enhancement Proposal 495: Local Time Disambiguation

51 points by philipn 11y ago

IanCal 11y ago
When the clocks change, don't you shift timezone? There aren't duplicate times in BST, we just switch from GMT to BST and then back. > In these situations, the information displayed on a local clock (or stored in a Python datetime instance) is insufficient to identify a particular moment in time. Does the datetime instance not store the timezone?
- Tehnix 11y ago
  Isn't it about summer and winter time? I'm pretty sure you're still in the same timezone in those cases. That said, I don't see the reason for this PEP. You should only store a value in the DB that doesn't change like that, and then when displayed you can change it to fit the local time of the user.
  - Marazan 11y ago
    In Britian during the summer we use the BST timezone and during winter we use GMT. But once again this is one of those "assumptions about time that everyone gets wrong" just because we shift timezones in Britian doesn't necessarily mean that people in other countries do the same - they might just change what the timezone offset is and keep the timezone the same.
    - Schwolop 11y ago
      Absolutely agree with the "one of those assumptions" bit. See http://infiniteundo.com/post/25326999628/falsehoods-programmers-believe-about-time http://infiniteundo.com/post/25326999628/falsehoods-programm... for a great list. (And related, see https://www.mjt.me.uk/posts/falsehoods-programmers-believe-about-addresses/ https://www.mjt.me.uk/posts/falsehoods-programmers-believe-a... for more about addresses.)
      - A_Beer_Clinked 11y ago
        Another great resource is the tz database [0]. It includes some great vignettes as well as the time zone rules: From Larry M. Smith (2006-04-26) re Wisconsin: http://www.legis.state.wi.us/statutes/Stat0175.pdf http://www.legis.state.wi.us/statutes/Stat0175.pdf ... is currently enforced at the 01:00 time of change. Because the local "bar time" in the state corresponds to 02:00, a number of citations are issued for the "sale of class 'B' alcohol after prohibited hours" within the deviated hour of this change every year.... From Winston Churchill (1934-04-28): It is one of the paradoxes of history that we should owe the boon of summer time, which gives every year to the people of this country between 160 and 170 hours more daylight leisure, to a war which plunged Europe into darkness for four years, and shook the foundations of civilization throughout the world. [0] http://www.iana.org/time-zones http://www.iana.org/time-zones
    - IanCal 11y ago
      > But once again this is one of those "assumptions about time that everyone gets wrong" just because we shift timezones in Britian doesn't necessarily mean that people in other countries do the same - they might just change what the timezone offset is and keep the timezone the same. Well, for a proposed change I'd like to see examples of the actual problem caused. The only example I can see is EST and EDT, which seems to be the same as the UK doing GMT->BST. Same for Andorra. My question wasn't rhetorical, I was actually asking if there were places that don't do this.
    - kbenson 11y ago
      Isn't that just a matter of having your library differentiate the timezones slightly,even if the canonical name is identical for the two different timezones (no matter what the country calls them, they really are different)? Additionally, there are ways to reference the timezone in the local time that are guaranteed to be unique. For example, offset from UTC. 1:15 AM UTC-09 is not the same as 1:15 AM UTC-08. What localities do you know of that just change the hour and do not have a time zone assigned to deal with it? I'm fairly sure the point of time zones it to track all those changes, and a name would be assigned to deal with it.
    - deleted 11y ago
      [deleted]
    - mark-r 11y ago
      That's not the way it works in the US at all. We keep the same zone all year long but change the offset and change the name slightly. The zones are geographically based: Eastern, Central, Mountain, and Pacific. They are abbreviated differently depending on whether Daylight Saving is in effect or not, for example Central is either CST (Standard, -06:00) or CDT (Daylight, -05:00). The important part here is that the API to get a timezone usually doesn't include a Daylight Saving flag. For example pytz would be used as pytz.timezone('US/Central'). There needs to be a process that adjusts the tzinfo object to the proper offset; in pytz this is the localize method, but that's non-standard.
- msm23 11y ago
  Yes, you could get the information from the timezone, but how would one do that in code? The only time that one has the time fold is when you turn the clock backward (let's just call that shifting from daylight savings to standard timezone). And this would only affect code which used wall clock time (time as it's read, e.g. 1:30am PDT), and would also only affect code which wanted to run something only once at a time within that fold (e.g. 1:30am ... not on both 1:30am's). So, using the timezone method, just check to see if your current 1:30am is in your daylight savings timezone. Hurray! You're in the clear. Go ahead and do that thing you wanted to do only on the first 1:30am. But the next day you're going to run into a problem. The only 1:30am you're going to get is in the standard timezone. So now you have to check for this timezone change only on the day of the change, which is yet another piece of data you have to keep track of. On the day of the change, do this timezone comparison, and on every other day don't worry about it. When the clock hits your interesting time of 1:30am, just check to see if today is the day of the change, check what the current timezone is, check what the daylight savings time zone is, check to see if those those two values are the same, and now do your thing. Otherwise, just do your thing. All of the above also ignores that people change times at different times (11pm, 1am, 2am, 3am), some don't change a full hour, and some don't change at all. The proposal gets rid of all of that convoluted logic in everyone's programs, and instead it provides a single boolean value: is this the second time I've seen this time because of daylight savings shenanigans.
  - ars 11y ago
    You don't need to do any of that. Just get a unixtime, and move on. Why are you doing time math in local time? Simply do all the time math in universal time and be done with it.
  - IanCal 11y ago
    > The proposal gets rid of all of that convoluted logic in everyone's programs Does it? It doesn't cover the scheduling problem the other half of the year when the clocks move the other direction. > So now you have to check for this timezone change only on the day of the change, which is yet another piece of data you have to keep track of. If running a job twice is a problem, then why not check that the job has not already been run? > is this the second time I've seen this time Is this unambiguous? If it's 2015-10-25-01-30-00 GMT, have I seen that time before? In the UK, yes, in Mali no.
- danbruc 11y ago
  You are always in the same time zone but the zone time changes. BST and GMT are not time zones - time zones are (usually) named by geographic features like large cities and is Europe/London in your case - but zone times. Therefore when entering or leaving daylight saving time you remain in the same time zone but the zone time of your time zone changes.
  - wyldfire 11y ago
    It would be nice if those two things which have an important but very subtle distinction had names which were not just the reverse of each other. It seems like despite you explaining this, it would be very easy for one to mistake "zone time" for "time zone".
kbenson 11y ago
I'm not sure the rationale behind this. That is, the rationale section of the proposal does a poor just of explaining any case where this is actually a problem. In every case where I've seen this problem, it's a matter of people either not storing the timezone along with the local time, or not storing in UTC time. A local time with a timezone is a unique time, it does not occur twice. A UTC time additionally does not occur twice. Store a time zone along with the date and time or store in UTC and convert on use. Note: If there are instances where a second is repeated, it's rare special occurrence that developing a formalized interface for seems like overkill.
- lifeisstillgood 11y ago
  Agreed - and more succinctly put than I managed - clearly it's the wrong time for me :-)
- msm23 11y ago
  I believe the use case is for scheduling systems that need to do something at (say) 1:30am. On every day at 1:30am, please run the accounting job. In the US, once a year you either have to have some locking system set up to avoid the second run (leave a trace that the job has already been started for the day). I believe the person proposing this thinks that it would make this determination easier. The problem is that it only solves half the trouble caused by dual timezones. There's another day of the year when 2:30am (in the US) doesn't happen at all. If something is scheduled to occur once per day at 2:30am, then that day it is not going to happen. There are other workarounds available such as avoiding the magic hours around 2am (in the US). But it seems to be a common problem that everyone seems to keep re-solving. Storing the timezone with the time doesn't really solve the above issue.
  - kbenson 11y ago
    > I believe the use case is for scheduling systems that need to do something at (say) 1:30am. On every day at 1:30am, please run the accounting job. In the instance where someone wants something to happen at 1:30 AM, if they aren't specific in the specification, then they should expect that it may happen twice or not at all at at certain times of the year. This is an imprecise specification problem, not a problem in representing time in structures that can and do contain timezones. That is, it's a failure of cron, or the user specifying the time, take your pick. What it's not is a failure of dates, times and timezones, which specifically address this problem. Timezones or UTC (which is just timezone offset 0), are what we have to deal with this specific problem. For example, specifying the originating timezone would disambiguate the time, as would specifying it in UTC, (or automatically converting to and using the UTC equivalent on entry). > Storing the timezone with the time doesn't really solve the above issue. It doesn't address the "do this thing at this local time daily" problem when someone chooses a time that has special behavior, but it does address the "do this thing at this offset from UTC daily" which may be the best you can expect when specifying a time for a recurring action and not taking into account timezones. If you want to use local time, you have to deal with either the actually time the job runs possibly shifting slightly throughout the year in some locations, or possibly running twice or not at all. My real problem with the proposal is that adding a .first() method doesn't solve anything, and really just makes half the problem (it doesn't work if the time doesn't exist), and in a way that's already easily solved, since you can't get a valid result from .first without knowing the timezone already.
lifeisstillgood 11y ago
So : Time in UK is currently UTC +1 (BST) At 2am on 25 OCT we will return to GMT / UTC. It will therefore become 1am, and for the next hour all times will have happened before The idea is to put a bit flag that says "alreadyseenthistime" It seems to me this is a solution to the wrong problem. Store all strings as bytes, assuming UTF-8, store all times as longs assuming UTC If we convert all python datelines to non-naive (ie embedded with a TZ) then we are forced always to choose an encoding just like in strings. The right encoding is to always assume incoming dates are UTC, to throw error if they are non naive, and to assume that local clocks are set correctly (which we do anyway) I need to read it more carefully - but it seems the wrong solution
toyg 11y ago
Terrible PEP, I hope it gets rejected. One-off flags like this are hacks that shouldn't be in stdlib. It simply stinks, in an area (time handling) where the stdlib does not really smell like roses already. Dealing with time adjustments is the OS's job, not userland. If your job has to be scheduled exactly and cannot rely on the OS, and you refuse to deal with UTC, it's your own damn fault and you can always just use a long-running process with timers.
- bliti 11y ago
  I agree. It's library baggage.
Marazan 11y ago
In general any solution to datetimes that doesn't involve the time being in UTC is solving thew wrong problem.
- akvadrako 11y ago
  I would disagree. UTC is ambiguous and unpredictable. Computers should really be based on GPST (seconds since 1980-Jan-6 UTC).
  - ubernostrum 11y ago
    If you want to be pedantic about getting rid of unpredictability in your time measurement, use TAI.
  - heinrich5991 11y ago
    That's defined as TAI minus some seconds, just go for TAI. :)
mayoff 11y ago
The more I'm exposed to other date/time libraries, the more impressed I become with Apple's date/time library. https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/DatesAndTimes/DatesAndTimes.html#//apple_ref/doc/uid/10000039-SW1 https://developer.apple.com/library/mac/documentation/Cocoa/...

51 points by philipn 11y ago

When the clocks change, don't you shift timezone? There aren't duplicate times in BST, we just switch from GMT to BST and then back. > In these situations, the information displayed on a local clock (or stored in a Python datetime instance) is insufficient to identify a particular moment in time. Does the datetime instance not store the timezone?

Tehnix 11y ago
Isn't it about summer and winter time? I'm pretty sure you're still in the same timezone in those cases. That said, I don't see the reason for this PEP. You should only store a value in the DB that doesn't change like that, and then when displayed you can change it to fit the local time of the user.
- Marazan 11y ago
  In Britian during the summer we use the BST timezone and during winter we use GMT. But once again this is one of those "assumptions about time that everyone gets wrong" just because we shift timezones in Britian doesn't necessarily mean that people in other countries do the same - they might just change what the timezone offset is and keep the timezone the same.
  - Schwolop 11y ago
    Absolutely agree with the "one of those assumptions" bit. See http://infiniteundo.com/post/25326999628/falsehoods-programmers-believe-about-time http://infiniteundo.com/post/25326999628/falsehoods-programm... for a great list. (And related, see https://www.mjt.me.uk/posts/falsehoods-programmers-believe-about-addresses/ https://www.mjt.me.uk/posts/falsehoods-programmers-believe-a... for more about addresses.)
    - A_Beer_Clinked 11y ago
      Another great resource is the tz database [0]. It includes some great vignettes as well as the time zone rules: From Larry M. Smith (2006-04-26) re Wisconsin: http://www.legis.state.wi.us/statutes/Stat0175.pdf http://www.legis.state.wi.us/statutes/Stat0175.pdf ... is currently enforced at the 01:00 time of change. Because the local "bar time" in the state corresponds to 02:00, a number of citations are issued for the "sale of class 'B' alcohol after prohibited hours" within the deviated hour of this change every year.... From Winston Churchill (1934-04-28): It is one of the paradoxes of history that we should owe the boon of summer time, which gives every year to the people of this country between 160 and 170 hours more daylight leisure, to a war which plunged Europe into darkness for four years, and shook the foundations of civilization throughout the world. [0] http://www.iana.org/time-zones http://www.iana.org/time-zones
  - IanCal 11y ago
    > But once again this is one of those "assumptions about time that everyone gets wrong" just because we shift timezones in Britian doesn't necessarily mean that people in other countries do the same - they might just change what the timezone offset is and keep the timezone the same. Well, for a proposed change I'd like to see examples of the actual problem caused. The only example I can see is EST and EDT, which seems to be the same as the UK doing GMT->BST. Same for Andorra. My question wasn't rhetorical, I was actually asking if there were places that don't do this.
  - kbenson 11y ago
    Isn't that just a matter of having your library differentiate the timezones slightly,even if the canonical name is identical for the two different timezones (no matter what the country calls them, they really are different)? Additionally, there are ways to reference the timezone in the local time that are guaranteed to be unique. For example, offset from UTC. 1:15 AM UTC-09 is not the same as 1:15 AM UTC-08. What localities do you know of that just change the hour and do not have a time zone assigned to deal with it? I'm fairly sure the point of time zones it to track all those changes, and a name would be assigned to deal with it.
  - deleted 11y ago
    [deleted]
  - mark-r 11y ago
    That's not the way it works in the US at all. We keep the same zone all year long but change the offset and change the name slightly. The zones are geographically based: Eastern, Central, Mountain, and Pacific. They are abbreviated differently depending on whether Daylight Saving is in effect or not, for example Central is either CST (Standard, -06:00) or CDT (Daylight, -05:00). The important part here is that the API to get a timezone usually doesn't include a Daylight Saving flag. For example pytz would be used as pytz.timezone('US/Central'). There needs to be a process that adjusts the tzinfo object to the proper offset; in pytz this is the localize method, but that's non-standard.
msm23 11y ago
Yes, you could get the information from the timezone, but how would one do that in code? The only time that one has the time fold is when you turn the clock backward (let's just call that shifting from daylight savings to standard timezone). And this would only affect code which used wall clock time (time as it's read, e.g. 1:30am PDT), and would also only affect code which wanted to run something only once at a time within that fold (e.g. 1:30am ... not on both 1:30am's). So, using the timezone method, just check to see if your current 1:30am is in your daylight savings timezone. Hurray! You're in the clear. Go ahead and do that thing you wanted to do only on the first 1:30am. But the next day you're going to run into a problem. The only 1:30am you're going to get is in the standard timezone. So now you have to check for this timezone change only on the day of the change, which is yet another piece of data you have to keep track of. On the day of the change, do this timezone comparison, and on every other day don't worry about it. When the clock hits your interesting time of 1:30am, just check to see if today is the day of the change, check what the current timezone is, check what the daylight savings time zone is, check to see if those those two values are the same, and now do your thing. Otherwise, just do your thing. All of the above also ignores that people change times at different times (11pm, 1am, 2am, 3am), some don't change a full hour, and some don't change at all. The proposal gets rid of all of that convoluted logic in everyone's programs, and instead it provides a single boolean value: is this the second time I've seen this time because of daylight savings shenanigans.
- ars 11y ago
  You don't need to do any of that. Just get a unixtime, and move on. Why are you doing time math in local time? Simply do all the time math in universal time and be done with it.
- IanCal 11y ago
  > The proposal gets rid of all of that convoluted logic in everyone's programs Does it? It doesn't cover the scheduling problem the other half of the year when the clocks move the other direction. > So now you have to check for this timezone change only on the day of the change, which is yet another piece of data you have to keep track of. If running a job twice is a problem, then why not check that the job has not already been run? > is this the second time I've seen this time Is this unambiguous? If it's 2015-10-25-01-30-00 GMT, have I seen that time before? In the UK, yes, in Mali no.
danbruc 11y ago
You are always in the same time zone but the zone time changes. BST and GMT are not time zones - time zones are (usually) named by geographic features like large cities and is Europe/London in your case - but zone times. Therefore when entering or leaving daylight saving time you remain in the same time zone but the zone time of your time zone changes.
- wyldfire 11y ago
  It would be nice if those two things which have an important but very subtle distinction had names which were not just the reverse of each other. It seems like despite you explaining this, it would be very easy for one to mistake "zone time" for "time zone".

kbenson 11y ago

I'm not sure the rationale behind this. That is, the rationale section of the proposal does a poor just of explaining any case where this is actually a problem. In every case where I've seen this problem, it's a matter of people either not storing the timezone along with the local time, or not storing in UTC time. A local time with a timezone is a unique time, it does not occur twice. A UTC time additionally does not occur twice. Store a time zone along with the date and time or store in UTC and convert on use. Note: If there are instances where a second is repeated, it's rare special occurrence that developing a formalized interface for seems like overkill.

lifeisstillgood 11y ago
Agreed - and more succinctly put than I managed - clearly it's the wrong time for me :-)
msm23 11y ago
I believe the use case is for scheduling systems that need to do something at (say) 1:30am. On every day at 1:30am, please run the accounting job. In the US, once a year you either have to have some locking system set up to avoid the second run (leave a trace that the job has already been started for the day). I believe the person proposing this thinks that it would make this determination easier. The problem is that it only solves half the trouble caused by dual timezones. There's another day of the year when 2:30am (in the US) doesn't happen at all. If something is scheduled to occur once per day at 2:30am, then that day it is not going to happen. There are other workarounds available such as avoiding the magic hours around 2am (in the US). But it seems to be a common problem that everyone seems to keep re-solving. Storing the timezone with the time doesn't really solve the above issue.
- kbenson 11y ago
  > I believe the use case is for scheduling systems that need to do something at (say) 1:30am. On every day at 1:30am, please run the accounting job. In the instance where someone wants something to happen at 1:30 AM, if they aren't specific in the specification, then they should expect that it may happen twice or not at all at at certain times of the year. This is an imprecise specification problem, not a problem in representing time in structures that can and do contain timezones. That is, it's a failure of cron, or the user specifying the time, take your pick. What it's not is a failure of dates, times and timezones, which specifically address this problem. Timezones or UTC (which is just timezone offset 0), are what we have to deal with this specific problem. For example, specifying the originating timezone would disambiguate the time, as would specifying it in UTC, (or automatically converting to and using the UTC equivalent on entry). > Storing the timezone with the time doesn't really solve the above issue. It doesn't address the "do this thing at this local time daily" problem when someone chooses a time that has special behavior, but it does address the "do this thing at this offset from UTC daily" which may be the best you can expect when specifying a time for a recurring action and not taking into account timezones. If you want to use local time, you have to deal with either the actually time the job runs possibly shifting slightly throughout the year in some locations, or possibly running twice or not at all. My real problem with the proposal is that adding a .first() method doesn't solve anything, and really just makes half the problem (it doesn't work if the time doesn't exist), and in a way that's already easily solved, since you can't get a valid result from .first without knowing the timezone already.

lifeisstillgood 11y ago

So : Time in UK is currently UTC +1 (BST) At 2am on 25 OCT we will return to GMT / UTC. It will therefore become 1am, and for the next hour all times will have happened before The idea is to put a bit flag that says "alreadyseenthistime" It seems to me this is a solution to the wrong problem. Store all strings as bytes, assuming UTF-8, store all times as longs assuming UTC If we convert all python datelines to non-naive (ie embedded with a TZ) then we are forced always to choose an encoding just like in strings. The right encoding is to always assume incoming dates are UTC, to throw error if they are non naive, and to assume that local clocks are set correctly (which we do anyway) I need to read it more carefully - but it seems the wrong solution

toyg 11y ago

Terrible PEP, I hope it gets rejected. One-off flags like this are hacks that shouldn't be in stdlib. It simply stinks, in an area (time handling) where the stdlib does not really smell like roses already. Dealing with time adjustments is the OS's job, not userland. If your job has to be scheduled exactly and cannot rely on the OS, and you refuse to deal with UTC, it's your own damn fault and you can always just use a long-running process with timers.

bliti 11y ago
I agree. It's library baggage.

Marazan 11y ago

In general any solution to datetimes that doesn't involve the time being in UTC is solving thew wrong problem.

akvadrako 11y ago
I would disagree. UTC is ambiguous and unpredictable. Computers should really be based on GPST (seconds since 1980-Jan-6 UTC).
- ubernostrum 11y ago
  If you want to be pedantic about getting rid of unpredictability in your time measurement, use TAI.
- heinrich5991 11y ago
  That's defined as TAI minus some seconds, just go for TAI. :)

mayoff 11y ago

The more I'm exposed to other date/time libraries, the more impressed I become with Apple's date/time library. https://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/DatesAndTimes/DatesAndTimes.html#//apple_ref/doc/uid/10000039-SW1 https://developer.apple.com/library/mac/documentation/Cocoa/...