5 ms·

Dealing with Timezones in Python

158 points by samrat 15y ago

espeed 15y ago
Another good time read is "Time, What Every Programmer Should Know" (http://unix4lyfe.org/time/ http://unix4lyfe.org/time/) -- in short, it's usually best to store time/dates as Unix time (it's UTC and a single number).
- wulczer 15y ago
  Storing time as Unix timestamps is often not possible. Often you need to be able to answer questions like "did this moment fall on a weekday?", "what day was it 3 hours after that moment" and so on. My rule is to either work with integer Unix timestamps or timezone-aware datetime objects. A naive datetime object is, well, naive. If it's the number of seconds since the Epoch, represent it as a number, not an object.
  - cbs 15y ago
    >Storing time as Unix timestamps is often not possible. If only there were mechanisms that could use a stored set of predefined rules enabling those other values to be derived when given a unix timestamp.
  - the_mitsuhiko 15y ago
    > My rule is to either work with integer Unix timestamps or timezone-aware datetime objects. A naive datetime object is, well, naive. If it's the number of seconds since the Epoch, represent it as a number, not an object. The object is basically just a wrapper around that number anyways. From the programmer's perspective there is not really a difference.
  - espeed 15y ago
    You can convert unix time to whatever format you want.
    - wulczer 15y ago
      But you frequently need to store the original timezone, to be able to do date calculations after storing it. For instance, a user added an appointment and her timezone is America/Los_Angeles. Then she moved to China and changed her timezone in your app to Asia/Shanghai. Then you run analytics to see how many users had had appointments on weekdays. When storing date information in persistent stores (like databases), it's always advisable to store the time and the timezone.
      - the_mitsuhiko 15y ago
        Even in that situation it's a terrible idea to store local time alone. In the calendar situation (which is the exception not the rule!) I personally would store the time in UTC, in local time and the name of the country separated. I would not store the local time alone because what do you show in the calendar if you are in a different country? If the assumption is that the problem is a theoretical one because nobody has appointments in the middle of the night when DST changes come, I present you with sysadmins, rescue workers and other people that do have these things scheduled at odd times. I know that around DST changes many people explicitly not work over night to avoid these issues. Same with year changes and other situations where computing and time does not work properly. For instance overnight trains in Europe just wait for an hour and do nothing.
        wulczer 15y ago
        You store the local time and timezone. You show the value in whatever timezone the user configured the application. If you store only Unix time, you don't have enough information. If you store both the Unix timestamp, the local time and the timezone, you have too much information. You can derive Unix time from local time and the timezone. Ergo: store time with time zone.
        DannoHung 15y ago
        Why not store the Unix timestamp and the timezone that was set when created? Then you can just as easily determine what the original local time was and also convert to whatever the current local time is. And you save yourself the misery of endless conversion when trying to compare anything internally.
- adobriyan 15y ago
  And yet another good read is Erik Naggum's "The Long, Painful History of Time" http://naggum.no/lugm-time.html http://naggum.no/lugm-time.html which describes better way for storing date(time)?s.
- david_a_r_kemp 15y ago
  It seems such a shame that every new language has to learn this - I read a similar post relating to .Net a few years ago
grovulent 15y ago
So I've been coding for about a year and a half now and had to learn all this the hard way. Which isn't a bad thing at all. It's little stuff like this that really makes it true that you should try to build an app or two like a blog-n-whatnot so that one's ignorance of these issues can be exposed.
- brendoncrawford 15y ago
  To be fair, even a lot of experienced programmers are tripped up by these issues.
nikcub 15y ago
a few more gotcha's: * when storing a UTC timestamp and you need to get a datetime object back in order to add dates or do some calcualtions, make sure you use datetime.datetime.utcfromtimestamp(), eg. >>> datetime.datetime.utcfromtimestamp(1310727758.472) datetime.datetime(2011, 7, 15, 11, 2, 38, 472000) * python timestamps are longs and represent time in seconds. in Javascript it is the number of milliseconds since epoch, so you need to divide/multiple by 1000 depending on which way you are going. for eg. > new Date().valueOf() 1310728084389 >>> datetime.datetime.utcfromtimestamp(1310728084389) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: year is out of range >>> datetime.datetime.utcfromtimestamp(1310728084389 / 1000) datetime.datetime(2011, 7, 15, 11, 8, 4) * always set your server to UTC, always use the utc* functions (eg. utctimetuple, utcfromtimestamp, utctime etc.), store as a long, and when passing back to the client send the current UTC timestamp as well rather than relying on the client system clock, which is frequently wrong. You can also use: http://json-time.appspot.com/time.json?tz=UTC http://json-time.appspot.com/time.json?tz=UTC or implement something similar. I resorted to always outputting the current UTC time into a javascript variable and using that to calibrate the local computer time (again, never trusting client time) * never use time.time() to get a timestamp back because it will munge the value and re-add tz info * in Javascript, to go from your server-provided UTC timestamp to a local date object that you can then work with, use: > new Date(1310728084 * 1000).toUTCString() "Fri, 15 Jul 2011 11:08:04 GMT" * If you are providing an API or a feed, please include a UTC timestamp in the output - parsing all the different date formats and timestamp info and attempting to work out when something happen (such as when a post was published) is a real pain in the ass. store the timezone in a sep field called 'utc_offset' or similar
- xiaoqmaku 15y ago
  welcometo: http://www.fullmalls.com http://www.fullmalls.com The website wholesale for many kinds of fashion shoes, like the nike,jordan,prada,, also including the jeans,shirts,bags,hat and the decorations. All the products are free shipping, and the the price is competitive, and also can accept the paypal payment.,after the payment, can ship within short time. free shippingcompetitive priceany size availableaccept the paypal ===== http://www.fullmalls.com http://www.fullmalls.com ===== jordan shoes $32nike shox $32Christan Audigier bikini $23 Ed Hardy Bikini $23Smful short_t-shirt_woman $15ed hardy short_tank_woman $16Sandal $32christian loubo utin $80 Sunglass $15 COACH_Necklace $27handbag $33AF tank woman $17puma slipper woman $30 ===== http://www.fullmalls.com http://www.fullmalls.com ===== ===== http://www.fullmalls.com http://www.fullmalls.com ===== ===== http://www.fullmalls.com http://www.fullmalls.com ===== ===== http://www.fullmalls.com http://www.fullmalls.com ===== ===== http://www.fullmalls.com http://www.fullmalls.com =====
briansfrank 15y ago
In Fantom we model DateTimes with both an absolute time in nanosecond ticks and a relative timezone: http://fantom.org/doc/docLang/DateTime.html#dateTime http://fantom.org/doc/docLang/DateTime.html#dateTime Works great for dealing with timestamps in absolute terms, but also when you need to deal with local time.
riobard 15y ago
I'm just bitten by this today: I was trying to get the time zone offset in Python to print the local time for an app. Doing date +%z in console gives me correct offset +0800 of China (we do not have DST anymore) while doing import time time.strftime('%z', time.localtime()) gives me +0706 WTF??!!! I can understand Python might not be able to get the correct timezone info from the system, but giving this kind of obviously incorrect offset is just dumb. Still searching for a solution other than manually configure Python code. If anyone knows why, I would be really glad for your help. Thanks in advance!
- wulczer 15y ago
  Use pytz. import pytz from datetime import datetime tz = pytz.timezone('Asia/Shanghai') tz.normalize(pytz.utc.localize(datetime.utcnow()).astimezone(tz))
  - riobard 15y ago
    Thanks for the response! I'm looking for a solution such that Python will go and load system's time zone and use that---it's a nightmare to deploy an app to multiple severs located in different timezone and I have to change the timezone in Python manually.
jriddycuz 15y ago
I'd just like to point out in reference to his Galileo quote that there would still be a need for time zones even if the sun revolved around the earth.
mvzink 15y ago
I have never heard any convincing argument (other than "we're already doing it") to use timezones or DSTs in the modern world. Can someone offer some articles or arguments?

158 points by samrat 15y ago

Another good time read is "Time, What Every Programmer Should Know" (http://unix4lyfe.org/time/ http://unix4lyfe.org/time/) -- in short, it's usually best to store time/dates as Unix time (it's UTC and a single number).

wulczer 15y ago
Storing time as Unix timestamps is often not possible. Often you need to be able to answer questions like "did this moment fall on a weekday?", "what day was it 3 hours after that moment" and so on. My rule is to either work with integer Unix timestamps or timezone-aware datetime objects. A naive datetime object is, well, naive. If it's the number of seconds since the Epoch, represent it as a number, not an object.
- cbs 15y ago
  >Storing time as Unix timestamps is often not possible. If only there were mechanisms that could use a stored set of predefined rules enabling those other values to be derived when given a unix timestamp.
- the_mitsuhiko 15y ago
  > My rule is to either work with integer Unix timestamps or timezone-aware datetime objects. A naive datetime object is, well, naive. If it's the number of seconds since the Epoch, represent it as a number, not an object. The object is basically just a wrapper around that number anyways. From the programmer's perspective there is not really a difference.
- espeed 15y ago
  You can convert unix time to whatever format you want.
  - wulczer 15y ago
    But you frequently need to store the original timezone, to be able to do date calculations after storing it. For instance, a user added an appointment and her timezone is America/Los_Angeles. Then she moved to China and changed her timezone in your app to Asia/Shanghai. Then you run analytics to see how many users had had appointments on weekdays. When storing date information in persistent stores (like databases), it's always advisable to store the time and the timezone.
    - the_mitsuhiko 15y ago
      Even in that situation it's a terrible idea to store local time alone. In the calendar situation (which is the exception not the rule!) I personally would store the time in UTC, in local time and the name of the country separated. I would not store the local time alone because what do you show in the calendar if you are in a different country? If the assumption is that the problem is a theoretical one because nobody has appointments in the middle of the night when DST changes come, I present you with sysadmins, rescue workers and other people that do have these things scheduled at odd times. I know that around DST changes many people explicitly not work over night to avoid these issues. Same with year changes and other situations where computing and time does not work properly. For instance overnight trains in Europe just wait for an hour and do nothing.
      - wulczer 15y ago
        You store the local time and timezone. You show the value in whatever timezone the user configured the application. If you store only Unix time, you don't have enough information. If you store both the Unix timestamp, the local time and the timezone, you have too much information. You can derive Unix time from local time and the timezone. Ergo: store time with time zone.
        DannoHung 15y ago
        Why not store the Unix timestamp and the timezone that was set when created? Then you can just as easily determine what the original local time was and also convert to whatever the current local time is. And you save yourself the misery of endless conversion when trying to compare anything internally.
adobriyan 15y ago
And yet another good read is Erik Naggum's "The Long, Painful History of Time" http://naggum.no/lugm-time.html http://naggum.no/lugm-time.html which describes better way for storing date(time)?s.
david_a_r_kemp 15y ago
It seems such a shame that every new language has to learn this - I read a similar post relating to .Net a few years ago

grovulent 15y ago

So I've been coding for about a year and a half now and had to learn all this the hard way. Which isn't a bad thing at all. It's little stuff like this that really makes it true that you should try to build an app or two like a blog-n-whatnot so that one's ignorance of these issues can be exposed.

brendoncrawford 15y ago
To be fair, even a lot of experienced programmers are tripped up by these issues.

nikcub 15y ago

a few more gotcha's: * when storing a UTC timestamp and you need to get a datetime object back in order to add dates or do some calcualtions, make sure you use datetime.datetime.utcfromtimestamp(), eg. >>> datetime.datetime.utcfromtimestamp(1310727758.472) datetime.datetime(2011, 7, 15, 11, 2, 38, 472000) * python timestamps are longs and represent time in seconds. in Javascript it is the number of milliseconds since epoch, so you need to divide/multiple by 1000 depending on which way you are going. for eg. > new Date().valueOf() 1310728084389 >>> datetime.datetime.utcfromtimestamp(1310728084389) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: year is out of range >>> datetime.datetime.utcfromtimestamp(1310728084389 / 1000) datetime.datetime(2011, 7, 15, 11, 8, 4) * always set your server to UTC, always use the utc* functions (eg. utctimetuple, utcfromtimestamp, utctime etc.), store as a long, and when passing back to the client send the current UTC timestamp as well rather than relying on the client system clock, which is frequently wrong. You can also use: http://json-time.appspot.com/time.json?tz=UTC http://json-time.appspot.com/time.json?tz=UTC or implement something similar. I resorted to always outputting the current UTC time into a javascript variable and using that to calibrate the local computer time (again, never trusting client time) * never use time.time() to get a timestamp back because it will munge the value and re-add tz info * in Javascript, to go from your server-provided UTC timestamp to a local date object that you can then work with, use: > new Date(1310728084 * 1000).toUTCString() "Fri, 15 Jul 2011 11:08:04 GMT" * If you are providing an API or a feed, please include a UTC timestamp in the output - parsing all the different date formats and timestamp info and attempting to work out when something happen (such as when a post was published) is a real pain in the ass. store the timezone in a sep field called 'utc_offset' or similar

xiaoqmaku 15y ago
welcometo: http://www.fullmalls.com http://www.fullmalls.com The website wholesale for many kinds of fashion shoes, like the nike,jordan,prada,, also including the jeans,shirts,bags,hat and the decorations. All the products are free shipping, and the the price is competitive, and also can accept the paypal payment.,after the payment, can ship within short time. free shippingcompetitive priceany size availableaccept the paypal ===== http://www.fullmalls.com http://www.fullmalls.com ===== jordan shoes $32nike shox $32Christan Audigier bikini $23 Ed Hardy Bikini $23Smful short_t-shirt_woman $15ed hardy short_tank_woman $16Sandal $32christian loubo utin $80 Sunglass $15 COACH_Necklace $27handbag $33AF tank woman $17puma slipper woman $30 ===== http://www.fullmalls.com http://www.fullmalls.com ===== ===== http://www.fullmalls.com http://www.fullmalls.com ===== ===== http://www.fullmalls.com http://www.fullmalls.com ===== ===== http://www.fullmalls.com http://www.fullmalls.com ===== ===== http://www.fullmalls.com http://www.fullmalls.com =====

briansfrank 15y ago

In Fantom we model DateTimes with both an absolute time in nanosecond ticks and a relative timezone: http://fantom.org/doc/docLang/DateTime.html#dateTime http://fantom.org/doc/docLang/DateTime.html#dateTime Works great for dealing with timestamps in absolute terms, but also when you need to deal with local time.

riobard 15y ago

I'm just bitten by this today: I was trying to get the time zone offset in Python to print the local time for an app. Doing date +%z in console gives me correct offset +0800 of China (we do not have DST anymore) while doing import time time.strftime('%z', time.localtime()) gives me +0706 WTF??!!! I can understand Python might not be able to get the correct timezone info from the system, but giving this kind of obviously incorrect offset is just dumb. Still searching for a solution other than manually configure Python code. If anyone knows why, I would be really glad for your help. Thanks in advance!

wulczer 15y ago
Use pytz. import pytz from datetime import datetime tz = pytz.timezone('Asia/Shanghai') tz.normalize(pytz.utc.localize(datetime.utcnow()).astimezone(tz))
- riobard 15y ago
  Thanks for the response! I'm looking for a solution such that Python will go and load system's time zone and use that---it's a nightmare to deploy an app to multiple severs located in different timezone and I have to change the timezone in Python manually.

jriddycuz 15y ago

I'd just like to point out in reference to his Galileo quote that there would still be a need for time zones even if the sun revolved around the earth.

mvzink 15y ago

I have never heard any convincing argument (other than "we're already doing it") to use timezones or DSTs in the modern world. Can someone offer some articles or arguments?