19 ms·

What every programmer should know about time

444 points by enneff 15y ago

rubashov 15y ago
Fails to mention the distinction between Zulu and Solar time. All the times mentioned are based on cesium atomic clocks. Satellite and military applications are often instead based on when the Sun is exactly opposite the Greenwich Meridian, re-synced daily.
geuis 15y ago
"Timezones are a presentation-layer problem! Most of your code shouldn't be dealing with timezones or local time, it should be passing Unix time around." I can attest to this. At a previous job our entire API used UTC. It was clean and worked at every layer of the app, from django to our client-side javascript. When we needed to display a human readable version, we did the translation at render time. All interactions with time as data was done with the UTC timestamp and saved much headache. A couple months before I left, one of the engineers proposed switching everything over to a textual representation according to ISO_8601. I forget the nature of the argument, but it was inane(to me). This actually led an extensive back/forth email exchange between various members of engineering, me as one of the frontend engineers, and even the engineering manager who seemed to favor the idea. I argued, "why change the entire stack which works just fine, etc etc". Fortunately, in this instance a heavy workload and group apathy about taking on unnecessary additional work allowed this entire concept to wither and disappear after a couple days.
- tseabrooks 15y ago
  Yup. In our architecture (TVs and other devices) time is kept as UTC through the entire system - We actually have two APIs. One that is UTC time that is used in all middleware.. and a "GetCurrentTime" API that is used only in the presentation layer to know how time should be displayed on screen. GetTime returns UTC time getCurrentTime returns a time struct of hours / minutes / seconds... This works well and it allows developers to easily identify API misuse during code reviews.
- cdcarter 15y ago
  Though, in calendar apps, I find this behavior annoying. My phone automatically adjusts my calendar when I change timezones, but I put all events into my calendar as local time for wherever I will be when that event is going to occur. I don't want to have to think about how to enter things into my calendar when I'm planning a trip back to where I grew up...
  - makmanalp 15y ago
    Well, this is also a presentation layer problem. You should be able to tell it to use your home zone instead of the current local zone, for example.
    - cdcarter 15y ago
      I don't want to tell it that though. I just want to enter things at a time, and have them stay at that time.
      - baddox 15y ago
        Any calendar app either has to base its time zone on location or ask you to manually choose a zone. How would you propose changing this?
        andrewcooke 15y ago
        i think what is being suggested is something like storing it as a string and raising the alarm when the string matches the current local time. so it's "time in whatever time zone i am local to when it matches the time". (which has problems with uniqueness, but does seem like an intuitive "dwim" high level interface).
        invalidOrTaken 15y ago
        In the realm of unrealistic-now-but-maybe-cool-later, if it could access your flight reservations, it could infer where you'll be at the time and base the time zone on that.
- nolite 15y ago
  What about when you're storing things at a date/hour/or even minute granularity.. not seconds.. is this still as important?
  - geuis 15y ago
    We solved this by storing metadata around the timestamps. For us, it was an ID that referenced a global table of polities that use daylight savings and we could derive the current GMT offset from that. We could theoretically update the polities table over time as these changed, though while I was there we never did. As for granularity, you can derive day/minute/hour etc based on the timestamp. For us, we were able to do those calculations in the application layer. For other types of projects, you can store that data in the db if you need to do more efficient queries for example.
- wvenable 15y ago
  Storing all your time as UTC can create problems depending on what you're doing with the time. If your application is a calendaring application and people can book things well into the future, you can have problems with daylight saving time and timezones this way. For my most recent app, timestamps are UTC and everything else is stored local time.
  - barrkel 15y ago
    You'll need to store the time zone along with the local time, otherwise you won't be able to handle the situation where you have multiple users in different time zones. About the only reason this is preferable to storing UTC along with time zone, is because there are sometimes political decisions made to change daylight savings time (i.e. the mere fact of DST/TZs etc. isn't enough).
    - wvenable 15y ago
      Yes, I store the timezone (associated with the user) along with the local time. All date operations in the application are done related to their timezone. Storing it this way is preferable because it makes working with times across DST boundaries a non-existent problem. DST is a pain in the ass and you want to use whatever language / database facilities to exist to make this as smooth as possible -- storing local time (and setting the timezone appropriately) makes it not your problem. Also users frequently set their own timezone incorrectly, and when they fix their timezone, they don't want all their appointments to be at the wrong time.
  - yaix 15y ago
    A calendar application is a bit different, since the whole subject of the application is time, time zones, etc. Still I'd store all in UTC along with the user's time zone (you'd have to store the time zone anyway with localized time too).
    - wvenable 15y ago
      Storing local time is preferable because it makes working with times across DST boundaries a non-existent problem and because users frequently set their own timezone incorrectly. I've done previous projects where I've stored UTC dates or unix timestamps for this situation and it was a hassle to deal with. You really need to consider the nature of the data -- for straight forward timestamps (like the date of a post) UTC makes the most sense. For user-entered dates, I think local time is much more appropriate and a lot simpler.
sgerrand 15y ago
As per gakman's comment in the Google+ crosspost[1], be wary of the Unix millenium bug[2] if you use integers for timestamp storage. [1] https://plus.google.com/106356964679457436995/posts/Hzq2P7V6KUw https://plus.google.com/106356964679457436995/posts/Hzq2P7V6... [2] http://en.wikipedia.org/wiki/Year_2038_problem http://en.wikipedia.org/wiki/Year_2038_problem
- sneak 15y ago
  "But there's no way my code will be running unchanged in 2038!"
- yoyar 15y ago
  I wonder about storing the timestamp in a varchar field of sufficient size to avoid all of these headaches? Although I guess 64 bit will suffice for a very long time.
  - caf 15y ago
    As a point of reference, the universe is estimated at less than 2^59 seconds old.
- andrewcooke 15y ago
  man 3p time suggests using time_t to "ease the eventual fix".
tseabrooks 15y ago
As someone working working on time sensitive code on embedded systems (DVRs that get UTC from the broadcast), I can certainly agree with the issues laid out in the post. As an example: We have some certifications our product must pass and the certification body plays a 4 minute looping broadcast stream with the test condition in it. It turns out I handled the time jump hat occurred when the stream would loop around poorly and this caused about 1 week worth of headaches and delays in getting our certification. None of my code expects time to be ever increasing now.
- deleted 15y ago
  [deleted]
- andrewcooke 15y ago
  just to add to the examples here: last week i was writing code to handle calibration data from seismic detectors. these are connected to GPS receivers and so have pretty accurate times. yet when i triggered a calibration it would start 1 minute in the past. somehow the receiver was moving back in time before starting the calibration.... ...or ntpd on the test computer had failed and the machine was a minute fast :o) so the code to search for data now looks backwards in time as well as forwards. incidentally, does anyone know of a really good API for time (including calendars etc)? python's (which is largely a thin layer over C) is a horrible mess, for example.
  - roel_v 15y ago
    boost::date_time. I've cursed at it in the beginning, but it does things the Right Way, and I've been saved a number of times by its fighting back against shortcuts (like using Unix time - the issues identified in the article are real and a good summary, but the solution only works for the case where you need to work within a limited time span, from 1970 until a few decades into this century.)
    - exDM69 15y ago
      C++0x standard library is pretty nice too. std::chrono and std::thread work nicely together. Right now I'm working with C, pthreads and struct timespec's and it makes me wish for a good time handling library.
maurycy 15y ago
For a much more detailed perspective about time, and still rather accessible, check Poul-Henning Kamp's work: http://phk.freebsd.dk/pubs/timecounter.pdf http://phk.freebsd.dk/pubs/timecounter.pdf http://people.freebsd.org/~phk/ http://people.freebsd.org/~phk/
dkarl 15y ago
Backwards jumps in time burned me once. The user was running my software on machines that had a bug specific to certain Opteron multiprocessor chipsets where a process migrating from one processor to another would sometimes see a backwards jump in time, even when the system's time was marching forward predictably on each processor. It just goes to show that you're always doing distributed computing, even if you don't know it.
melvinmt 15y ago
One notable addition: relative time is the same for everyone. What I mean by this is that instead of messing with timezones (by trying to guess the user's timezone or even worse, asking for it) in most cases it is sufficient to tell the user something has happened x hours ago, or y days ago. If you're programming in PHP, I can recommend this book: http://www.amazon.com/architects-Guide-Date-Time-Programming/dp/0981034500 http://www.amazon.com/architects-Guide-Date-Time-Programming...
- derefr 15y ago
  I'm really quite fond of Reddit's convention of specifying that the site will go down for maintenance "when this post is X hours old" made as a self-post (where all posts have a relative timestamp like you're talking about as standard metadata.) It's the only time I don't have to do any mental calculations to figure out when the thing's going to happen.
- codyrobbins 15y ago
  Ugh, I feel that this is one of the single poorest time-related pratices—from a UX perspective—short of not displaying the time at all. Please don’t follow this advice. It makes it really easy for the developer because there’s no need to deal with time zones and things like DST. However, showing a relative time in many situations is completely opaque. If something happened ‘14 hours ago’, did it happen at lunch time or 3 PM? A lot of times people want to know the time of day something happened, not just how long ago it occurred. Here you can do some arithmetic in your head to figure it out, but that’s a major annoyance. And once the relative time flips over to ‘X days ago’ the time information is completely lost. Similarly if it says ‘About 3 months ago’ it’s impossible to know if that means March 15 or April 1 or April 15 or anywhere in between, never mind the time of day. At the very least the full date and time should be displayed in a tooltip so that it’s available if needed. Ideally relative times shouldn’t be used alone except in situations where the relative time is absolutely, unarguably the only information that could ever need to be known.
  - wvenable 15y ago
    I agree! And one of the more useful things you can do with a web app is render time using JavaScript (with an appropriate fallback) and then it will show the timestamp in the user's timezone automatically.
projectileboy 15y ago
I'd like to add one to the list - store your Unix time as a 64-bit value, to save your client/employer some headaches in 2032. I doubt I'm the only HN user who was spent a lot of time in '98 and '99 fixing Y2K problems.
- caf 15y ago
  Personally I'm looking forward to cashing in on some lucrative contracts as a graybeard C programmer in 20 years time, the same way all those COBOL survivors were able to in the late 90s ;)
- mambodog 15y ago
  *2038 http://en.wikipedia.org/wiki/Year_2038_problem http://en.wikipedia.org/wiki/Year_2038_problem
  - projectileboy 15y ago
    D'oh! I misfired typing. Thanks for the correction.
- masklinn 15y ago
  > I'd like to add one to the list - store your Unix time as a 64-bit value How about actually doing the right thing and using time_t?
  - nuxi 15y ago
    I'm not sure that's the right thing, due to how time_t is (not) defined. First, it can be an integer or a floating number (although the latter is unlikely). Second, the size of time_t is not defined, so it could be 32-bit or 64-bit or something else. And then there may also be endianness issues when storing time from multiple different systems. So I'd say store it either as a 64-bit value (ignoring the possible floats, converting 32- to 64-bit and handling endianness), or use a textual representation of time_t.
kamagmar 15y ago
> UTC (which is an arbitrary human invention) Hmm, i wouldn't call it totally arbitrary. UTC = TAI + LS, such that |UTC - UT1| < 1 second, where: * LS are leap seconds, * TAI is "physicist" time, based on the ticking of atomic clocks at mean sea level. The length of a second is constant. * UT1 is "astronomer" time, the rotation angle of the Earth with respect to the quasar reference frame. The length of a second is not constant. The Earth's rotation is slowing down, so UT1 is gradually drifting away from TAI. UTC is a pretty natural scheme to reconcile these two systems.
psykotic 15y ago
Erik Naggum's paper The Long, Painful History of Time is a must-read: http://naggum.no/lugm-time.html http://naggum.no/lugm-time.html
jen_h 15y ago
"The system clock can, and will, jump backwards and forwards in time due to things outside of your control. Your program should be designed to survive this." This is one of my favorite go-to test cases. I've found some really fantastically interesting, catastrophic network halting badness with this really simple test.
- jodrellblank 15y ago
  Ooh, such as?
  - jen_h 15y ago
    The most fun one I ever found was a situation where a daemon analyzing & forwarding traffic through a bridged interface would lock up & stop passing traffic when you popped backwards and forwards through time on the box.
- wladimir 15y ago
  This literally just happened to a VPS of mine. Time was jumping forwards and back, every second the time could jump an hour ahead and back. Everything screwed up, from log rotation to sessions. My first thought was that this was some kind of prank :-) but seems it was a hardware issue on the parent machine combined with ntpd trying to compensate.
  - jen_h 15y ago
    This one was a blast back in the day: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006113 http://kb.vmware.com/selfservice/microsites/search.do?langua... No need to run the test case, you'd run into it soon enough on a 2.6 kernel on VMWare. ;)
  - justincormack 15y ago
    You should read this article (also applies to other VMs) http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachin... There are a lot of issues that can happen because of this...
lysol 15y ago
I like to rip on MySQL as much as the next guy, but the article is incorrect about MySQL DATETIMEs: DATETIME: Eight bytes: A four-byte integer packed as YYYY×10000 + MM×100 + DD A four-byte integer packed as HH×10000 + MM×100 + SS Storing UNIX time as an integer would be silly, considering: TIMESTAMP: A four-byte integer representing seconds UTC since the epoch ('1970-01-01 00:00:00' UTC)
- troels 15y ago
  MySql's datetime fields are not timezone aware. If one client has set one timezone and inserts a value in a datetime field and another client has a different timezone, the value will not be converted.
  - deleted 15y ago
    [deleted]
  - lysol 15y ago
    Not sure that's contradictory to my point about the backend storage. The overall points in the article are correct but its note about how MySQL stores its DATETIMEs is strange, and I'd hate to make people think MySQL is a bad database because of incorrect information, instead of things like the lack of transactional DDL.
lacerus 15y ago
How can I make Ruby on Rails use unix timestamps instead of MySQL DATETIMEs?
- Confusion 15y ago
  Use the TIMESTAMP datatype for those columns. Also see the comment by lysol (currently two above you) that indicates that the article is wrong about MySQL.
deleted 15y ago
[deleted]
oasisbob 15y ago
Unix time: Measured as the number of seconds since epoch (the beginning of 1970 in UTC). Unix time is not affected by time zones or daylight saving. I don't think this is strictly correct. This implies that someone could start an atomic stopwatch at midnight on Jan 1, 1970, and it would match Unix time. It won't. Because Unix time is non-linear and will either slew or repeat seconds when UTC has leap seconds, the hypothetical stopwatch would be ahead of Unix time by 34 seconds. ... at least this is how I understand it. Every time I try and wrap by head around the differences between UTC/TAI/UT1, my head really starts to hurt.
- kijinbear 15y ago
  You're right. It should say: Unix time is the number of seconds since epoch, not counting leap seconds.
  - bluedanieru 15y ago
    Any idea why it was done this way? It seems like counting leap seconds belongs in the same layer as sorting out timezones, i.e. not here.
    - thristian 15y ago
      Because people working with timestamps like to write code that says things like: // schedule another run for tomorrow schedule_event(now() + 86400) ...which doesn't actually work when leap-seconds are involved (things start to drift by a second). If you specify that leap seconds get replayed, it works.
      - shaggyfrog 15y ago
        An article previously linked on Hacker News goes into leap seconds in much greater detail and is a good read: http://cacm.acm.org/magazines/2011/5/107699-the-one-second-war/fulltext http://cacm.acm.org/magazines/2011/5/107699-the-one-second-w...
      - bluedanieru 15y ago
petercooper 15y ago
UTC used to be called Greenwich Mean Time (GMT) Sort of. This could be misleading because GMT and UTC are still two different things with different definitions. Wikipedia is a good source of info on this, but for starters: UTC is closely related to Universal Time and Greenwich Mean Time (GMT) and within informal or casual contexts where sub-second precision is not required, it can be used interchangeably. So not strictly, but practically.. ;-)
sehugg 15y ago
Also beware storing milliseconds in 32-bit quantities (as if you'd ever! but it happens). GetTickCount is the poster child for this class of bugs: http://en.wikipedia.org/wiki/GetTickCount http://en.wikipedia.org/wiki/GetTickCount In fact some versions of Windows CE intentionally set this value to (0xffffffff - 10 minutes) before bootup so that bugs were more likely to come out in testing, rather than showing up 42 days after bootup. Also, don't store time intervals as floating point, especially if you're working on a missile system: http://apps.ycombinator.com/item?id=1667060 http://apps.ycombinator.com/item?id=1667060
- CamperBob 15y ago
  You can store milliseconds in 32-bit quantities all you want, if you remember the One True Axiom of Time: never compare tick counts, always subtract and compare to the difference you were looking for. If you do it that way, you can't screw it up, at least in C.
  - deleted 15y ago
    [deleted]
  - cbr 15y ago
    Really? What if it wraps around? ts1=MAX_INT-40, ts2=20, ts2-ts1 < 0. Or does that actually work out correctly with signed integers in C?
    - CamperBob 15y ago
      It works regardless of signed/unsigned type, because signed ints still wrap at UINT_MAX (and not at INT_MAX).
    - wnoise 15y ago
      Technically, overflow of signed integers invokes undefined behavior (or possibly implementation defined. In my copy of a draft standard overflow is given as an example of undefined behavior, but 6.3.1.3 say implementation defined). In practice, it's the same bit values as unsigned integers, which does the right thing in this case.
altrego99 15y ago
MySQL (at least 4.x and 5.x) stores DATETIME columns as a "YYYY-MM-DD HH:MM:SS" string Wow that's terrible!
- ReturnTheGift 15y ago
  And false. See lysol's reply or the 5.0 reference manual's section on storage requirements: http://dev.mysql.com/doc/refman/5.0/en/storage-requirements.html http://dev.mysql.com/doc/refman/5.0/en/storage-requirements.... "YYYY-MM-DD HH:MM:SS" is the format of the DATETIME datatype. That's all.
dirtyaura 15y ago
> Timezones are a presentation-layer problem! I want to correct this common misconception that UTC is enough. Calendar time with all human traditions involved is more complex than a simple timestamp. The advice above is incorrect for calendar and scheduling apps, or anything that has a concept of a repeating event. An example: we have a weekly meeting occurring 9AM Monday in San Francisco. You are in London and want to attent the meeting over Skype. When is it in London time? It depends. On 7 Mar 2011 it's at 5pm On 14 Mar 2011 it's at 6pm On 29 Mar 2011 it's at 5pm To make these calculations, you need to know timezone & daylight saving time (DST) rules of both your current location and the home location of the event. A "DST zone" of a home location of a repeating event has to be saved together with a time and thus it's not just presentation-layer issue.
- raleec 15y ago
  And not just for repeating events. I worked on an app that dealt with scheduling travel, and you needed to record the zone of both locales so that if a trip crossed time zones the trip length(which was derived) would remain accurate. You also had to take into account what the DST would be for both locales at the scheduled time. Hint: Use Oracle, SQLServer doesn't have a way to stay updated with the constantly changing DST offsets worldwide..
- roel_v 15y ago
  Yes, very correct! I hope the 'lets just use Unix timestamp because it's easy' mentality will go away soon!
  - peteretep 15y ago
    What would you replace it with for storing a single date?
    - roel_v 15y ago
      Depends on the context it's used in. Usually YYYY-MM-DDThh:mm:ss of ISO 8601, explicitly extend it with the UTC timezone (with the + notation), sometimes that plus the local timezone in a separate column/field, sometimes a serialized version of a library representation of a date.
nivertech 15y ago
The most important thing a programmer need to know about time: "Don't use t+1 or t+N in your timeseries backtesting code ;)"
roel_v 15y ago
Don't blindly follow the advice at the end of this article! The issues he identifies are real, but the 'solution' only work in a small subset of use cases. When one needs a longer time span than [1970-2038], Unix timestamp is horrible - how are you going to represent a date of birth in it for people born before 1970 (yes they do still exist!)? There is no guarantee that negative timestamps will work! Also it doesn't take different calendars into account, still doesn't work with leap seconds, doesn't deal well with time spans (t1 - t2 specified in seconds can be a lot things in reality), ... Use a proper date time library to deal with dates and store them in your database in a string format, including the time zone. It depends on your application which time zone (UTC or local), but in general UTC is best, and the local time zone could be a second column if you need the info (or it could be a property of the user, but e.g. many calendaring application then screw it up in the UI layer...) I'd like to read a book on the UI issues associated with dates and times, anyone know of something like that?
rvkennedy 15y ago
Julian Date FTW! Seriously, JD is much smaller than Unix time, so it's less wasteful of numeric range for most practical purposes. And its use would eliminate a barrier between the computing world and that of science. http://en.wikipedia.org/wiki/Julian_day http://en.wikipedia.org/wiki/Julian_day
rmc 15y ago
Another thing to add: When asking people to put in the timezones, don't ask them for a UTC/GMT offset, and the dates that DST starts/ends. Instead ask them for the tzdata format (e.g. "Europe/London"). Then you can localize that wherever you want.
tintin 15y ago
Does NIST (and other sync tools) keep in mind the round-trip of a request (RTD / RTT)?
- derobert 15y ago
  NTPd does. However, in order the eliminate it, it assumes the RTT is symmetric, that is half is on the journey to the remote, half on the journey back. Asymmetric links do lead to systemic error in NTP. (It assumes symmetry because there isn't any way to measure the asymmetry without first having time sync, and obvious catch-22. If you need better than 10ms accuracy, you'll need a GPS, etc.)
jorangreef 15y ago
In the future, the world will use UTC and sunrise and sunset will happen at different times, relative to where you are.
- adestefan 15y ago
  No they won't. The idea that 12am is night and 12pm is day is pretty well entrenched in society.
  - georgefox 15y ago
    While I understand your point, I find that a surprising number of people have issues with 12am vs. 12pm, often confusing the two. These same people tend to think midnight occurs at the end of the day.
    - tomjen3 15y ago
      Everybody who isn't American or British has issues with am/pm. It is so counter intuitive it is not even funny.
  - demallien 15y ago
    jorangreef is probably right, at least assuming that we get to colonise other worlds. I mean, what does it mean to talk about 'day' when you're on the ISS? How are Martians going to deal with the fact that they have a Sidereal Rotation Period of 24h37mins? Or the Moon, with it's roughly 700hr SRP.
    - jorangreef 15y ago
      Right on. But maybe a better time would be TAI - UTC without the leap adjustments. Edit: We basically just need a logical clock though we may face problems reconciling this logical clock in a distributed space/time system.
  - dchest 15y ago
    Did you mean 0:00 is night and 12:00 is day?
JonnieCache 15y ago
For anyone else who has just learned about UT1 time and wants to know what it's current value is, here's the link: http://tf.nist.gov/pubs/bulletin/leapsecond.htm http://tf.nist.gov/pubs/bulletin/leapsecond.htm Today, UT1 - UTC = 82ms.
wooptoo 15y ago
> When storing time, store Unix time. It's a single number. This is BS. ISO-8601 (MySQL time) is way better and is not prone to the 2038 bug. Unix time has 'scalability' issues.
MarkMc 15y ago
The problem with time-based bugs is that they are often subtle. I've had time-based bugs that only appear between 11pm and midnight; or only during daylight savings time; or only where the client is in a different timezone to the server. Also, it is very common for business applications to deal with 'dates in the calendar', for example: a) John's birthday is 26 August 1966 b) The loan was borrowed on 16 January 2006 and repaid on 9 September 2009. I suspect most programmers will disagree with me, but in my experience it is NOT good practice to use a timestamp class to represent such things. It's better to use a class specifically designed to represent a date in the (Gregorian) calendar. In fact, I created an open-source Java class for this purpose: http://calendardate.sourceforge.net/ http://calendardate.sourceforge.net/
laut 15y ago
"Other timezones can be written as an offset from UTC. Australian Eastern Standard Time is UTC+1000. e.g. 10:00 UTC is 20:00 EST on the same day." Every programmer should know about DST. Offsets are not always enough. When it's winter north of the equator, some countries are on summer time (DST) south of the equator.
JohnLBevan 15y ago
It's about time someone posted about this. Great summary - when I first started working on an international system I was hunting for this kind of article. Now if someone could only do one on languages, regional settings, character sets and encodings. . .
closedbracket 15y ago
I thought this was some productivity spiel.
aaavw 15y ago
Other little know facts about time: The first day of the week can be sunday or monday depending on where you are. The way to count the weeks isn't the same everywhere. Some advice: Default to using ISO 8601 whenever possible http://en.wikipedia.org/wiki/ISO_8601 http://en.wikipedia.org/wiki/ISO_8601 Don't ask for more precision then you actually need (Don't ask date & time when you only need year & week) Don't store dates and times with more precision then was actually entered (e.g. Don't ask for year & week, and then store the (calculated) first day of the week. On my first large project I made the mistake of asking for the year & week while storing the calculated first day of the week, however using the wrong first day of the week and the wrong numbering of weeks…

444 points by enneff 15y ago

Fails to mention the distinction between Zulu and Solar time. All the times mentioned are based on cesium atomic clocks. Satellite and military applications are often instead based on when the Sun is exactly opposite the Greenwich Meridian, re-synced daily.

geuis 15y ago

"Timezones are a presentation-layer problem! Most of your code shouldn't be dealing with timezones or local time, it should be passing Unix time around." I can attest to this. At a previous job our entire API used UTC. It was clean and worked at every layer of the app, from django to our client-side javascript. When we needed to display a human readable version, we did the translation at render time. All interactions with time as data was done with the UTC timestamp and saved much headache. A couple months before I left, one of the engineers proposed switching everything over to a textual representation according to ISO_8601. I forget the nature of the argument, but it was inane(to me). This actually led an extensive back/forth email exchange between various members of engineering, me as one of the frontend engineers, and even the engineering manager who seemed to favor the idea. I argued, "why change the entire stack which works just fine, etc etc". Fortunately, in this instance a heavy workload and group apathy about taking on unnecessary additional work allowed this entire concept to wither and disappear after a couple days.

tseabrooks 15y ago
Yup. In our architecture (TVs and other devices) time is kept as UTC through the entire system - We actually have two APIs. One that is UTC time that is used in all middleware.. and a "GetCurrentTime" API that is used only in the presentation layer to know how time should be displayed on screen. GetTime returns UTC time getCurrentTime returns a time struct of hours / minutes / seconds... This works well and it allows developers to easily identify API misuse during code reviews.
cdcarter 15y ago
Though, in calendar apps, I find this behavior annoying. My phone automatically adjusts my calendar when I change timezones, but I put all events into my calendar as local time for wherever I will be when that event is going to occur. I don't want to have to think about how to enter things into my calendar when I'm planning a trip back to where I grew up...
- makmanalp 15y ago
  Well, this is also a presentation layer problem. You should be able to tell it to use your home zone instead of the current local zone, for example.
  - cdcarter 15y ago
    I don't want to tell it that though. I just want to enter things at a time, and have them stay at that time.
    - baddox 15y ago
      Any calendar app either has to base its time zone on location or ask you to manually choose a zone. How would you propose changing this?
      - andrewcooke 15y ago
        i think what is being suggested is something like storing it as a string and raising the alarm when the string matches the current local time. so it's "time in whatever time zone i am local to when it matches the time". (which has problems with uniqueness, but does seem like an intuitive "dwim" high level interface).
      - invalidOrTaken 15y ago
        In the realm of unrealistic-now-but-maybe-cool-later, if it could access your flight reservations, it could infer where you'll be at the time and base the time zone on that.
nolite 15y ago
What about when you're storing things at a date/hour/or even minute granularity.. not seconds.. is this still as important?
- geuis 15y ago
  We solved this by storing metadata around the timestamps. For us, it was an ID that referenced a global table of polities that use daylight savings and we could derive the current GMT offset from that. We could theoretically update the polities table over time as these changed, though while I was there we never did. As for granularity, you can derive day/minute/hour etc based on the timestamp. For us, we were able to do those calculations in the application layer. For other types of projects, you can store that data in the db if you need to do more efficient queries for example.
wvenable 15y ago
Storing all your time as UTC can create problems depending on what you're doing with the time. If your application is a calendaring application and people can book things well into the future, you can have problems with daylight saving time and timezones this way. For my most recent app, timestamps are UTC and everything else is stored local time.
- barrkel 15y ago
  You'll need to store the time zone along with the local time, otherwise you won't be able to handle the situation where you have multiple users in different time zones. About the only reason this is preferable to storing UTC along with time zone, is because there are sometimes political decisions made to change daylight savings time (i.e. the mere fact of DST/TZs etc. isn't enough).
  - wvenable 15y ago
    Yes, I store the timezone (associated with the user) along with the local time. All date operations in the application are done related to their timezone. Storing it this way is preferable because it makes working with times across DST boundaries a non-existent problem. DST is a pain in the ass and you want to use whatever language / database facilities to exist to make this as smooth as possible -- storing local time (and setting the timezone appropriately) makes it not your problem. Also users frequently set their own timezone incorrectly, and when they fix their timezone, they don't want all their appointments to be at the wrong time.
- yaix 15y ago
  A calendar application is a bit different, since the whole subject of the application is time, time zones, etc. Still I'd store all in UTC along with the user's time zone (you'd have to store the time zone anyway with localized time too).
  - wvenable 15y ago
    Storing local time is preferable because it makes working with times across DST boundaries a non-existent problem and because users frequently set their own timezone incorrectly. I've done previous projects where I've stored UTC dates or unix timestamps for this situation and it was a hassle to deal with. You really need to consider the nature of the data -- for straight forward timestamps (like the date of a post) UTC makes the most sense. For user-entered dates, I think local time is much more appropriate and a lot simpler.

sgerrand 15y ago

As per gakman's comment in the Google+ crosspost[1], be wary of the Unix millenium bug[2] if you use integers for timestamp storage. [1] https://plus.google.com/106356964679457436995/posts/Hzq2P7V6KUw https://plus.google.com/106356964679457436995/posts/Hzq2P7V6... [2] http://en.wikipedia.org/wiki/Year_2038_problem http://en.wikipedia.org/wiki/Year_2038_problem

sneak 15y ago
"But there's no way my code will be running unchanged in 2038!"
yoyar 15y ago
I wonder about storing the timestamp in a varchar field of sufficient size to avoid all of these headaches? Although I guess 64 bit will suffice for a very long time.
- caf 15y ago
  As a point of reference, the universe is estimated at less than 2^59 seconds old.
andrewcooke 15y ago
man 3p time suggests using time_t to "ease the eventual fix".

tseabrooks 15y ago

As someone working working on time sensitive code on embedded systems (DVRs that get UTC from the broadcast), I can certainly agree with the issues laid out in the post. As an example: We have some certifications our product must pass and the certification body plays a 4 minute looping broadcast stream with the test condition in it. It turns out I handled the time jump hat occurred when the stream would loop around poorly and this caused about 1 week worth of headaches and delays in getting our certification. None of my code expects time to be ever increasing now.

deleted 15y ago
[deleted]
andrewcooke 15y ago
just to add to the examples here: last week i was writing code to handle calibration data from seismic detectors. these are connected to GPS receivers and so have pretty accurate times. yet when i triggered a calibration it would start 1 minute in the past. somehow the receiver was moving back in time before starting the calibration.... ...or ntpd on the test computer had failed and the machine was a minute fast :o) so the code to search for data now looks backwards in time as well as forwards. incidentally, does anyone know of a really good API for time (including calendars etc)? python's (which is largely a thin layer over C) is a horrible mess, for example.
- roel_v 15y ago
  boost::date_time. I've cursed at it in the beginning, but it does things the Right Way, and I've been saved a number of times by its fighting back against shortcuts (like using Unix time - the issues identified in the article are real and a good summary, but the solution only works for the case where you need to work within a limited time span, from 1970 until a few decades into this century.)
  - exDM69 15y ago
    C++0x standard library is pretty nice too. std::chrono and std::thread work nicely together. Right now I'm working with C, pthreads and struct timespec's and it makes me wish for a good time handling library.

maurycy 15y ago

For a much more detailed perspective about time, and still rather accessible, check Poul-Henning Kamp's work: http://phk.freebsd.dk/pubs/timecounter.pdf http://phk.freebsd.dk/pubs/timecounter.pdf http://people.freebsd.org/~phk/ http://people.freebsd.org/~phk/

dkarl 15y ago

Backwards jumps in time burned me once. The user was running my software on machines that had a bug specific to certain Opteron multiprocessor chipsets where a process migrating from one processor to another would sometimes see a backwards jump in time, even when the system's time was marching forward predictably on each processor. It just goes to show that you're always doing distributed computing, even if you don't know it.

melvinmt 15y ago

One notable addition: relative time is the same for everyone. What I mean by this is that instead of messing with timezones (by trying to guess the user's timezone or even worse, asking for it) in most cases it is sufficient to tell the user something has happened x hours ago, or y days ago. If you're programming in PHP, I can recommend this book: http://www.amazon.com/architects-Guide-Date-Time-Programming/dp/0981034500 http://www.amazon.com/architects-Guide-Date-Time-Programming...

derefr 15y ago
I'm really quite fond of Reddit's convention of specifying that the site will go down for maintenance "when this post is X hours old" made as a self-post (where all posts have a relative timestamp like you're talking about as standard metadata.) It's the only time I don't have to do any mental calculations to figure out when the thing's going to happen.
codyrobbins 15y ago
Ugh, I feel that this is one of the single poorest time-related pratices—from a UX perspective—short of not displaying the time at all. Please don’t follow this advice. It makes it really easy for the developer because there’s no need to deal with time zones and things like DST. However, showing a relative time in many situations is completely opaque. If something happened ‘14 hours ago’, did it happen at lunch time or 3 PM? A lot of times people want to know the time of day something happened, not just how long ago it occurred. Here you can do some arithmetic in your head to figure it out, but that’s a major annoyance. And once the relative time flips over to ‘X days ago’ the time information is completely lost. Similarly if it says ‘About 3 months ago’ it’s impossible to know if that means March 15 or April 1 or April 15 or anywhere in between, never mind the time of day. At the very least the full date and time should be displayed in a tooltip so that it’s available if needed. Ideally relative times shouldn’t be used alone except in situations where the relative time is absolutely, unarguably the only information that could ever need to be known.
- wvenable 15y ago
  I agree! And one of the more useful things you can do with a web app is render time using JavaScript (with an appropriate fallback) and then it will show the timestamp in the user's timezone automatically.

projectileboy 15y ago

I'd like to add one to the list - store your Unix time as a 64-bit value, to save your client/employer some headaches in 2032. I doubt I'm the only HN user who was spent a lot of time in '98 and '99 fixing Y2K problems.

caf 15y ago
Personally I'm looking forward to cashing in on some lucrative contracts as a graybeard C programmer in 20 years time, the same way all those COBOL survivors were able to in the late 90s ;)
mambodog 15y ago
*2038 http://en.wikipedia.org/wiki/Year_2038_problem http://en.wikipedia.org/wiki/Year_2038_problem
- projectileboy 15y ago
  D'oh! I misfired typing. Thanks for the correction.
masklinn 15y ago
> I'd like to add one to the list - store your Unix time as a 64-bit value How about actually doing the right thing and using time_t?
- nuxi 15y ago
  I'm not sure that's the right thing, due to how time_t is (not) defined. First, it can be an integer or a floating number (although the latter is unlikely). Second, the size of time_t is not defined, so it could be 32-bit or 64-bit or something else. And then there may also be endianness issues when storing time from multiple different systems. So I'd say store it either as a 64-bit value (ignoring the possible floats, converting 32- to 64-bit and handling endianness), or use a textual representation of time_t.

kamagmar 15y ago

> UTC (which is an arbitrary human invention) Hmm, i wouldn't call it totally arbitrary. UTC = TAI + LS, such that |UTC - UT1| < 1 second, where: * LS are leap seconds, * TAI is "physicist" time, based on the ticking of atomic clocks at mean sea level. The length of a second is constant. * UT1 is "astronomer" time, the rotation angle of the Earth with respect to the quasar reference frame. The length of a second is not constant. The Earth's rotation is slowing down, so UT1 is gradually drifting away from TAI. UTC is a pretty natural scheme to reconcile these two systems.

psykotic 15y ago

Erik Naggum's paper The Long, Painful History of Time is a must-read: http://naggum.no/lugm-time.html http://naggum.no/lugm-time.html

jen_h 15y ago

"The system clock can, and will, jump backwards and forwards in time due to things outside of your control. Your program should be designed to survive this." This is one of my favorite go-to test cases. I've found some really fantastically interesting, catastrophic network halting badness with this really simple test.

jodrellblank 15y ago
Ooh, such as?
- jen_h 15y ago
  The most fun one I ever found was a situation where a daemon analyzing & forwarding traffic through a bridged interface would lock up & stop passing traffic when you popped backwards and forwards through time on the box.
wladimir 15y ago
This literally just happened to a VPS of mine. Time was jumping forwards and back, every second the time could jump an hour ahead and back. Everything screwed up, from log rotation to sessions. My first thought was that this was some kind of prank :-) but seems it was a hardware issue on the parent machine combined with ntpd trying to compensate.
- jen_h 15y ago
  This one was a blast back in the day: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006113 http://kb.vmware.com/selfservice/microsites/search.do?langua... No need to run the test case, you'd run into it soon enough on a 2.6 kernel on VMWare. ;)
- justincormack 15y ago
  You should read this article (also applies to other VMs) http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachin... There are a lot of issues that can happen because of this...

lysol 15y ago

I like to rip on MySQL as much as the next guy, but the article is incorrect about MySQL DATETIMEs: DATETIME: Eight bytes: A four-byte integer packed as YYYY×10000 + MM×100 + DD A four-byte integer packed as HH×10000 + MM×100 + SS Storing UNIX time as an integer would be silly, considering: TIMESTAMP: A four-byte integer representing seconds UTC since the epoch ('1970-01-01 00:00:00' UTC)

troels 15y ago
MySql's datetime fields are not timezone aware. If one client has set one timezone and inserts a value in a datetime field and another client has a different timezone, the value will not be converted.
- deleted 15y ago
  [deleted]
- lysol 15y ago
  Not sure that's contradictory to my point about the backend storage. The overall points in the article are correct but its note about how MySQL stores its DATETIMEs is strange, and I'd hate to make people think MySQL is a bad database because of incorrect information, instead of things like the lack of transactional DDL.

lacerus 15y ago

How can I make Ruby on Rails use unix timestamps instead of MySQL DATETIMEs?

Confusion 15y ago
Use the TIMESTAMP datatype for those columns. Also see the comment by lysol (currently two above you) that indicates that the article is wrong about MySQL.

deleted 15y ago

[deleted]

oasisbob 15y ago

Unix time: Measured as the number of seconds since epoch (the beginning of 1970 in UTC). Unix time is not affected by time zones or daylight saving. I don't think this is strictly correct. This implies that someone could start an atomic stopwatch at midnight on Jan 1, 1970, and it would match Unix time. It won't. Because Unix time is non-linear and will either slew or repeat seconds when UTC has leap seconds, the hypothetical stopwatch would be ahead of Unix time by 34 seconds. ... at least this is how I understand it. Every time I try and wrap by head around the differences between UTC/TAI/UT1, my head really starts to hurt.

kijinbear 15y ago
You're right. It should say: Unix time is the number of seconds since epoch, not counting leap seconds.
- bluedanieru 15y ago
  Any idea why it was done this way? It seems like counting leap seconds belongs in the same layer as sorting out timezones, i.e. not here.
  - thristian 15y ago
    Because people working with timestamps like to write code that says things like: // schedule another run for tomorrow schedule_event(now() + 86400) ...which doesn't actually work when leap-seconds are involved (things start to drift by a second). If you specify that leap seconds get replayed, it works.
    - shaggyfrog 15y ago
      An article previously linked on Hacker News goes into leap seconds in much greater detail and is a good read: http://cacm.acm.org/magazines/2011/5/107699-the-one-second-war/fulltext http://cacm.acm.org/magazines/2011/5/107699-the-one-second-w...
    - bluedanieru 15y ago

petercooper 15y ago

UTC used to be called Greenwich Mean Time (GMT) Sort of. This could be misleading because GMT and UTC are still two different things with different definitions. Wikipedia is a good source of info on this, but for starters: UTC is closely related to Universal Time and Greenwich Mean Time (GMT) and within informal or casual contexts where sub-second precision is not required, it can be used interchangeably. So not strictly, but practically.. ;-)

sehugg 15y ago

Also beware storing milliseconds in 32-bit quantities (as if you'd ever! but it happens). GetTickCount is the poster child for this class of bugs: http://en.wikipedia.org/wiki/GetTickCount http://en.wikipedia.org/wiki/GetTickCount In fact some versions of Windows CE intentionally set this value to (0xffffffff - 10 minutes) before bootup so that bugs were more likely to come out in testing, rather than showing up 42 days after bootup. Also, don't store time intervals as floating point, especially if you're working on a missile system: http://apps.ycombinator.com/item?id=1667060 http://apps.ycombinator.com/item?id=1667060

CamperBob 15y ago
You can store milliseconds in 32-bit quantities all you want, if you remember the One True Axiom of Time: never compare tick counts, always subtract and compare to the difference you were looking for. If you do it that way, you can't screw it up, at least in C.
- deleted 15y ago
  [deleted]
- cbr 15y ago
  Really? What if it wraps around? ts1=MAX_INT-40, ts2=20, ts2-ts1 < 0. Or does that actually work out correctly with signed integers in C?
  - CamperBob 15y ago
    It works regardless of signed/unsigned type, because signed ints still wrap at UINT_MAX (and not at INT_MAX).
  - wnoise 15y ago
    Technically, overflow of signed integers invokes undefined behavior (or possibly implementation defined. In my copy of a draft standard overflow is given as an example of undefined behavior, but 6.3.1.3 say implementation defined). In practice, it's the same bit values as unsigned integers, which does the right thing in this case.

altrego99 15y ago

MySQL (at least 4.x and 5.x) stores DATETIME columns as a "YYYY-MM-DD HH:MM:SS" string Wow that's terrible!

ReturnTheGift 15y ago
And false. See lysol's reply or the 5.0 reference manual's section on storage requirements: http://dev.mysql.com/doc/refman/5.0/en/storage-requirements.html http://dev.mysql.com/doc/refman/5.0/en/storage-requirements.... "YYYY-MM-DD HH:MM:SS" is the format of the DATETIME datatype. That's all.

dirtyaura 15y ago

> Timezones are a presentation-layer problem! I want to correct this common misconception that UTC is enough. Calendar time with all human traditions involved is more complex than a simple timestamp. The advice above is incorrect for calendar and scheduling apps, or anything that has a concept of a repeating event. An example: we have a weekly meeting occurring 9AM Monday in San Francisco. You are in London and want to attent the meeting over Skype. When is it in London time? It depends. On 7 Mar 2011 it's at 5pm On 14 Mar 2011 it's at 6pm On 29 Mar 2011 it's at 5pm To make these calculations, you need to know timezone & daylight saving time (DST) rules of both your current location and the home location of the event. A "DST zone" of a home location of a repeating event has to be saved together with a time and thus it's not just presentation-layer issue.

raleec 15y ago
And not just for repeating events. I worked on an app that dealt with scheduling travel, and you needed to record the zone of both locales so that if a trip crossed time zones the trip length(which was derived) would remain accurate. You also had to take into account what the DST would be for both locales at the scheduled time. Hint: Use Oracle, SQLServer doesn't have a way to stay updated with the constantly changing DST offsets worldwide..
roel_v 15y ago
Yes, very correct! I hope the 'lets just use Unix timestamp because it's easy' mentality will go away soon!
- peteretep 15y ago
  What would you replace it with for storing a single date?
  - roel_v 15y ago
    Depends on the context it's used in. Usually YYYY-MM-DDThh:mm:ss of ISO 8601, explicitly extend it with the UTC timezone (with the + notation), sometimes that plus the local timezone in a separate column/field, sometimes a serialized version of a library representation of a date.

nivertech 15y ago

The most important thing a programmer need to know about time: "Don't use t+1 or t+N in your timeseries backtesting code ;)"

roel_v 15y ago

Don't blindly follow the advice at the end of this article! The issues he identifies are real, but the 'solution' only work in a small subset of use cases. When one needs a longer time span than [1970-2038], Unix timestamp is horrible - how are you going to represent a date of birth in it for people born before 1970 (yes they do still exist!)? There is no guarantee that negative timestamps will work! Also it doesn't take different calendars into account, still doesn't work with leap seconds, doesn't deal well with time spans (t1 - t2 specified in seconds can be a lot things in reality), ... Use a proper date time library to deal with dates and store them in your database in a string format, including the time zone. It depends on your application which time zone (UTC or local), but in general UTC is best, and the local time zone could be a second column if you need the info (or it could be a property of the user, but e.g. many calendaring application then screw it up in the UI layer...) I'd like to read a book on the UI issues associated with dates and times, anyone know of something like that?

rvkennedy 15y ago

Julian Date FTW! Seriously, JD is much smaller than Unix time, so it's less wasteful of numeric range for most practical purposes. And its use would eliminate a barrier between the computing world and that of science. http://en.wikipedia.org/wiki/Julian_day http://en.wikipedia.org/wiki/Julian_day

rmc 15y ago

Another thing to add: When asking people to put in the timezones, don't ask them for a UTC/GMT offset, and the dates that DST starts/ends. Instead ask them for the tzdata format (e.g. "Europe/London"). Then you can localize that wherever you want.

tintin 15y ago

Does NIST (and other sync tools) keep in mind the round-trip of a request (RTD / RTT)?

derobert 15y ago
NTPd does. However, in order the eliminate it, it assumes the RTT is symmetric, that is half is on the journey to the remote, half on the journey back. Asymmetric links do lead to systemic error in NTP. (It assumes symmetry because there isn't any way to measure the asymmetry without first having time sync, and obvious catch-22. If you need better than 10ms accuracy, you'll need a GPS, etc.)

jorangreef 15y ago

In the future, the world will use UTC and sunrise and sunset will happen at different times, relative to where you are.

adestefan 15y ago
No they won't. The idea that 12am is night and 12pm is day is pretty well entrenched in society.
- georgefox 15y ago
  While I understand your point, I find that a surprising number of people have issues with 12am vs. 12pm, often confusing the two. These same people tend to think midnight occurs at the end of the day.
  - tomjen3 15y ago
    Everybody who isn't American or British has issues with am/pm. It is so counter intuitive it is not even funny.
- demallien 15y ago
  jorangreef is probably right, at least assuming that we get to colonise other worlds. I mean, what does it mean to talk about 'day' when you're on the ISS? How are Martians going to deal with the fact that they have a Sidereal Rotation Period of 24h37mins? Or the Moon, with it's roughly 700hr SRP.
  - jorangreef 15y ago
    Right on. But maybe a better time would be TAI - UTC without the leap adjustments. Edit: We basically just need a logical clock though we may face problems reconciling this logical clock in a distributed space/time system.
- dchest 15y ago
  Did you mean 0:00 is night and 12:00 is day?

JonnieCache 15y ago

For anyone else who has just learned about UT1 time and wants to know what it's current value is, here's the link: http://tf.nist.gov/pubs/bulletin/leapsecond.htm http://tf.nist.gov/pubs/bulletin/leapsecond.htm Today, UT1 - UTC = 82ms.

wooptoo 15y ago

> When storing time, store Unix time. It's a single number. This is BS. ISO-8601 (MySQL time) is way better and is not prone to the 2038 bug. Unix time has 'scalability' issues.

MarkMc 15y ago

The problem with time-based bugs is that they are often subtle. I've had time-based bugs that only appear between 11pm and midnight; or only during daylight savings time; or only where the client is in a different timezone to the server. Also, it is very common for business applications to deal with 'dates in the calendar', for example: a) John's birthday is 26 August 1966 b) The loan was borrowed on 16 January 2006 and repaid on 9 September 2009. I suspect most programmers will disagree with me, but in my experience it is NOT good practice to use a timestamp class to represent such things. It's better to use a class specifically designed to represent a date in the (Gregorian) calendar. In fact, I created an open-source Java class for this purpose: http://calendardate.sourceforge.net/ http://calendardate.sourceforge.net/

laut 15y ago

"Other timezones can be written as an offset from UTC. Australian Eastern Standard Time is UTC+1000. e.g. 10:00 UTC is 20:00 EST on the same day." Every programmer should know about DST. Offsets are not always enough. When it's winter north of the equator, some countries are on summer time (DST) south of the equator.

JohnLBevan 15y ago

It's about time someone posted about this. Great summary - when I first started working on an international system I was hunting for this kind of article. Now if someone could only do one on languages, regional settings, character sets and encodings. . .

closedbracket 15y ago

I thought this was some productivity spiel.

aaavw 15y ago

Other little know facts about time: The first day of the week can be sunday or monday depending on where you are. The way to count the weeks isn't the same everywhere. Some advice: Default to using ISO 8601 whenever possible http://en.wikipedia.org/wiki/ISO_8601 http://en.wikipedia.org/wiki/ISO_8601 Don't ask for more precision then you actually need (Don't ask date & time when you only need year & week) Don't store dates and times with more precision then was actually entered (e.g. Don't ask for year & week, and then store the (calculated) first day of the week. On my first large project I made the mistake of asking for the year & week while storing the calculated first day of the week, however using the wrong first day of the week and the wrong numbering of weeks…