10 ms·

Some Notes About Time

210 points by arb99 14y ago

defrost 14y ago
Sigh. 50 year old numerical / geophysical / real time data acquisition/processing/interpretation programmer here. Unix Time isn't much chop for "real time" continuous data from the "real world" - it's those pesky leap seconds. If you bother to read the first paragraph of the wikipedia article on Unix Time you'll see : > Unix time, or POSIX time, is a system for describing instances in time, defined as the number of seconds that have elapsed since midnight Coordinated Universal Time (UTC), 1 January 1970,[note 1] not counting leap seconds.[note 2] It is used widely in Unix-like and many other operating systems and file formats. It is neither a linear representation of time nor a true representation of UTC. It follows on with a definition of Unix Time and points out various examples when it is ambiguous. These are real issues and can occur when missiles fly, when planes navigate, and when stocks are traded. Time is tricky.
- martinced 14y ago
  Time is tricky but there is a simple fact: it exists such a thing as the number of actual seconds elapsed since 'time x' (typically the Unix epoch but anything else will do too). No leap seconds issues. No 25 hours-day issues. No 23 hours-day issues. No 59 seconds minutes issues. No 61 seconds minutes issues. And all it takes to store a time like that is a 64-bit integer and it is very convenient. And a lot of software do precisely that. Most timestamps are just that: they don't care about "real world" details like leap-seconds, {23,24,25}hours per day, etc. Because, in many case, you really don't give a flying shit about the "real-world" time. What is your point? That a server that needs to open trade precisely at 9am has to take leap seconds into account? Sure. The article ain't disputing that. But a lot of application basically only need timestamps and everything becomes much simpler when one starts to reason in milliseconds since the epoch instead of "real world" time. Btw... Many of your "real issues" are due to programmers not realizing that they could have realized the exact same system but way more resilient had they knew to use timestamps instead of "real world" time. There was an amazing paper by Google on the subject by the way, where they explained how they were reconciliating their hundreds of thousands of servers' time by allowing it to be "not correct" over x minutes / hours to dodge many issues. And they clearly did emphasize that: a) these problems shouldn't have occured in the first place and were due to poorly designed programs (poorly designed in that they relied on real-world time instead of simply internally using milliseconds since the epoch) and b) most programs on earth do not give a flying ^^^^ if a time displayed to the user is one third of a second off...
  - masklinn 14y ago
    > Time is tricky but there is a simple fact: it exists such a thing as the number of actual seconds elapsed since 'time x' Yes and no: it might just exist theoretically, but we have no way to get at it. The closest we have is TAI, which is only an approximation of the time elapsed at mean sea level on the earthly geoid, because clocks fall victim to gravitational time dilatation and compression. To accurately measure time you'd need a clock sitting perfectly still in space, and all other clocks in the universe would slowly drift behind it.
    - TeMPOraL 14y ago
      Not even that; relativity doesn't allow a single special frame of reference. There's no such thing as universal "time elapsed since x". Putting the clock in space would be as arbitrary choice as putting it on my roof (albeit definitely more practical).
      - ccozan 14y ago
        I think the one who solves this problem ( at least for humanity ) would get a Nobel. After the humans will wander thru space ( at some time - very probable), there will no time reference, but only time intervals ( like, day on a spaceship has 24h , etc ). So in this case, you would measure 86400 seconds and call it a new day. No more leap seconds, etc. Now the UNIX makes sense: count seconds since a certain event in time and meajure from there on, internally. Want to display it? Then use special computation to render it in the format ( read timezone, add relativistic skew, etc ).
    - Dylan16807 14y ago
      You're forgetting to factor in accuracy/resolution. TAI is completely perfect down to the picosecond level and further. That means you do have the number of actual seconds, milliseconds, microseconds, nanoseconds... You only encounter issues with TAI once you get down to femtosecond or smaller levels.
      - masklinn 14y ago
- skrebbel 14y ago
  Bummer. I hope that the author reads your comment, because it's otherwise a very useful page. I'd love to have a reference like the OP's about time and everything a programmer needs to know about it without problems like the one you're subscribing. This one's almost there. Thanks for clarifying!
- lnanek2 14y ago
  Yeah, even writing ditzy consumer apps on Android we're usually stuck with the number of milliseconds the system has been up if we want a reliable, step by step, count that doesn't jump around. Use something else and your animation delays will be screwed up occasionally, etc.. Of course then that number is useless outside that system and outside that boot of the system. Would have been much handier to have something better.
wereHamster 14y ago
He missed to explain TAI and its relation with the other times.
- wereHamster 14y ago
  Also, there have been many definitions of UTC, GMT etc over time. So to be truly correct you'd always have to mention which definition of UTC you are referring to when you use the term. And the ITU-R is debating to redefine UTC once again and remove leap seconds (http://www.ucolick.org/~sla/leapsecs/ http://www.ucolick.org/~sla/leapsecs/). If you are really interesting in the time stuff, here is a great resource: http://www.ucolick.org/~sla/ http://www.ucolick.org/~sla/.
  - Dylan16807 14y ago
    >mention which definition of UTC you are referring to Doesn't the time value tell you? Or have there been temporally overlapping definitions of UTC that disagreed?
    - sla29970 14y ago
      There have been many defintions of UTC http://www.ucolick.org/~sla/leapsecs/timescales.html#UTC http://www.ucolick.org/~sla/leapsecs/timescales.html#UTC which do not agree. When an API does not match what the providers supply, which definition qualifies as "official"?
timthorn 14y ago
GMT is not an old name for UTC. From http://www.bis.gov.uk/files/file32707.pdf http://www.bis.gov.uk/files/file32707.pdf : Coordinated Universal Time (UTC) is based on International Atomic Time (TAI), but offset by an integer number of seconds so that it remains in approximate agreement with more traditional time scales based on the Earth’s rotation (i.e. Universal Time or, equivalently, Greenwich Mean Time). There is no natural gearing between TAI and GMT and so additional ‘leap seconds’ are applied when required to keep UTC in agreement with GMT to the nearest 0.9 seconds (see notes on time scales in Annex A for background information). UTC is the global standard for civil-time keeping today. It provides the most stable time base available because it is based on TAI, but also acts as a good approximation to its antecedent, GMT, for everyday purposes.
kator 14y ago
Funny I read the title and thought: TVM but then realized it was the age old quest for time keeping and dreaming of ways to track it with reasonable sanity.. :-) Bonus points for anyone who knows what I mean by TVM and can describe why every programmer should know about it..
- chris_wot 14y ago
  I can only think of Time Value of Money...
  - nonamegiven 14y ago
    Unfortunately Google can only think of that too.
    - chris_wot 14y ago
      I was attempting to generate discussion. Your snark is not welcome.
      - nonamegiven 14y ago
        No snark intendend whatsoever. I was trying to figure out what the guy was talking about, and when I searched TBV the results were overwhelmed with Time Value of Money. I was observing that if the guy meant anything other than that, it would be all but impossible for someone to figure out if someone was not familiar with ... whatever he's talking about.
        chris_wot 14y ago
        In that case, I owe you apology :( Love to know what this TVM is.
- blowski 14y ago
  You explaining it would be more useful than the bonus points.
InclinedPlane 14y ago
Who was the genius who decided that unix time should handle leap seconds? It seems like such an obviously bad idea. The biggest advantage of unix time has always been that it is monotonically increasing and that it is precisely defined as seconds since the epoch, period. And then all of that is now broken because of this decision. Also, does the problem that the leap second solves actually warrant all of the problems that it causes?
- ArchD 14y ago
  Since Unix time does NOT count the leap seconds, it allows the computation of the human-readable time from the Unix time without knowing when the leap seconds occurred. If knowledge of when the leap seconds occurred is required, then those events would need to be available somehow and used, introducing i) more expensive computation ii) difficulty in future-proofing an app since a reliable and updated data source supplying the times of leap seconds is required.
  - ProblemFactory 14y ago
    The obvious solution would be to get rid of leap seconds entirely, and let UTC and solar time (when sun is highest at Greenwich) drift apart. Sun rising 30 seconds later every century is not really a problem in my opinion.
- deleted 14y ago
  [deleted]
- ProblemFactory 14y ago
  > Also, does the problem that the leap second solves actually warrant all of the problems that it causes? Definitely not. The only benefit of leap seconds is keeping UTC in sync with solar time: the sun is highest in the sky at 12:00 noon at Greenwich. But in the modern world, synchronizing solar time and clocks to <1 minute is of no value to anyone but nostalgic astronomers.
  - sla29970 14y ago
patrickmay 14y ago
Erik Naggum's "Long Painful History of Time" (http://www.scribd.com/doc/93991574/Erik-Naggum—ALongPainfulHistoryofTime http://www.scribd.com/doc/93991574/Erik-Naggum—ALongPainfulH...) is well worth reading, too.
- nonamegiven 14y ago
  Working through it, thanks. The link in the paper's headers is still live: http://naggum.no/lugm-time.html http://naggum.no/lugm-time.html The home page refers to his ill health: http://naggum.no/ http://naggum.no/ And he in fact died in 2009 at 44: https://en.wikipedia.org/wiki/Erik_Naggum https://en.wikipedia.org/wiki/Erik_Naggum Interesting guy, involved in specifying the Internet from link level to mid-email level, worked on Emacs, and among the first Usenet users to be known for flaming. I vaguely remember the name in that context. On balance, a contributor.
stanonyime 14y ago
The relativistic warping of space-time forbids standardization of time measurements
- nealabq 14y ago
  Unless you also standardize the representation of inertial reference frames.
jimktrains2 14y ago
http://infiniteundo.com/post/25326999628/falsehoods-programmers-believe-about-time http://infiniteundo.com/post/25326999628/falsehoods-programm... http://infiniteundo.com/post/25509354022/more-falsehoods-programmers-believe-about-time-wisdom http://infiniteundo.com/post/25509354022/more-falsehoods-pro... Also: http://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names http://www.kalzumeus.com/2010/06/17/falsehoods-programmers-b...
speeder 14y ago
He did not mention the issue that some processors might have different time counting for each core, sometimes resulting in very silly situations if you are reading time in milliseconds or below scale. For example: I had a game that physics behaved "woobly" in multi core systems. The issue is that as the OS juggle your process around, you get the delta time since last physics step inaccurate, sometimes even with backwards jump in time ( and according backward movement in the game ). Later I saw someone showing the effects of this on the file system, with a multithreaded file copy resulting in very strange timestamps. For my game, the solution was force time-dependant threads to request affinity with one core.
dmbaggett 14y ago
Also worth noting: there are many time zone variations beyond DST, and some are defined in terms of partial hours (30 or 15 minute variations). True story: when I was at ITA Software, Orbitz ran hundreds of instances of QPX, our low fare search software, on their own servers. We had an ops fire drill one weekend because customers were complaining that the site was showing incorrect prices. The root cause? A single machine in their server farm had the time wrong, so advance purchase was computed incorrectly for any query sent to that particular machine. That was fun to debug. Even the meaning of, say, a minium stay requirement is hard to precisely define. If you fly across the international date line and back on a Friday night, did you have a Saturday stay or not? There are so many flights going so many places, questions like this actually come up in practice.
- rolux 14y ago
  For other - often difficult, and sometimes surprising - problems with time, see Peter-Paul Koch's excellent essay "Making <time> safe for historians": http://www.quirksmode.org/blog/archives/2009/04/making_time_saf.html http://www.quirksmode.org/blog/archives/2009/04/making_time_...
lysol 14y ago
> When storing time, store Unix time. It's a single number. Don't do this in your database. Use your datetime types. Please. You might save some work with your timezone but you're not going to be able to use intervals etc in a smart way. This would be even dumber with PostgreSQL, which has much more robust date/time functionality (being able to use the - operator with datetimes and intervals, for example) and completely sane timezone support.
- RyanZAG 14y ago
  Why can't you use intervals in a smart way? Single number is the easiest and fastest method to use for intervals. The only true problem with storing unix time is leap seconds and relativistic effects, which can both be safely ignored for most time keeping usages.
  - ianburrell 14y ago
    PostgreSQL intervals are smarter than just number of seconds between two datetimes. They can represent "1 month" or "1 year" which can't be represented as a number of seconds. Also, intervals can be used between dates or times, not just datetimes.
    - jtheory 14y ago
      Exactly right; date arithmetic is horrible if you only have seconds to work with. Even just saying "24 hours from now" -- as your user would see it -- you'll sometimes get the wrong answer if you just add on 24 hours worth of seconds... suppose they have a DST shift tonight? What if you need to know how many days remain from today to March 1st of this year? ...well, are we in a leap year? Postgres datetime doesn't solve all of the problems that crop up, but it's certainly better than using unix time.
      - RyanZAG 14y ago
        24 hours from now is no problem if you're using unix time. Just add 24 hours worth of seconds -- when you convert back to the user's time, your conversion logic will take care of timezones. This is also incredibly important if serving an app over the internet as your user may have just flown from USA to Japan. Storing unix time solves this in the easiest way. Same for working out how many days remain - you convert March 1st to a unix date, get an interval in seconds, and then convert that into a human readable format on display. This isn't some revelation - unix time has been used successfully for decades now. EDIT: Also, Postgresql uses a number very similar to unix time in the actual storage of the date - it just handles the pretty display for you. So you're arguing for using the same thing whether you say use a postgre datetime or a unix date number.
NateEag 14y ago
As the article hints, even though timezones are usually presentation-only, there are some cases where business logic really does have to deal with timezones. If your software has business logic that cares about, say, what day a specific person perceived an event as happening, it needs to think about the timezone they have set. So, in such cases, you'd better keep a history of all the timezones they've ever chosen, and the time (UTC) they changed them. (Hopefully this lets someone else avoid my past mistakes.)
- rmc 14y ago
  Agreed. Often you need to care about timezones. Someone wants an alarm to go off at 8am every day. Well you need to start thinking about timezones, since that "8am" needs to be local time.
- icambron 14y ago
  I would go so far as to say that time zones are rarely presentation-level issues. Want to have a meeting every Tuesday at 2:00 New York time? Better make sure you know the New York part or daylight savings will break it. Want to wake up every day at 9:00? If you don't track the zone that preference is set in, you won't get that right when you travel. Time zone isn't a presentation-only thing because people's behavior depends on their time zone.
khitchdee 14y ago
I guess a related question is how do you synchronize a users perception of time with a program's perception of time -- a program that the user is interacting with. To effect this synchronization, you somehow need to know of the latencies involved in the user initiating actions that the program can sense and when those actions register with the program and in (latencies involved with) the program initiating actions that the user can sense and when the user senses those actions.
albertzeyer 14y ago
This topic comes up again and again. But I mostly read only about the problems, not about the solutions. This post advises to use Unix timestamps. That's already what I'm doing mostly everywhere. And it seems to me like most people do and like this is somewhat accepted as the best available option - but still not perfect. Then I am wondering again: Isn't there really a perfect solution? Esp., the problems with Unix timestamps are the non-monotony and that e.g. two equal timestamps can actually represent two different times (seconds). This could be bad in cases where seconds matter (e.g. some log-files which need to be very precise about time). Maybe TAI is and we all should store TAI timestamps instead of Unix timestamps? But is there an easy way to get TAI timestamps? Also, I haven't really seen other projects doing this - why? Maybe it is just too little gain over Unix timestamps and too less tools available to work with them... Btw., I just checked, there is http://cr.yp.to/libtai.html http://cr.yp.to/libtai.html and http://pypi.python.org/pypi/tai64n http://pypi.python.org/pypi/tai64n, maybe I should just start using that. This of course still doesn't solve anything about synchronization or inaccurate system clocks but it would be better than Unix times. Edit: Getting the TAI timestamp is probably not easy or maybe even not possible... I just saw this: https://github.com/stoni/libtai/blob/master/tai_now.c https://github.com/stoni/libtai/blob/master/tai_now.c ...
- lmm 14y ago
  On servers you control, do what Google does: skew time on leap-second days, making each second slightly longer, so that the day is still the normal number of seconds. Filter inputs coming from outside so that your internal systems never see a 60th second in a minute.
- sopooneo 14y ago
  I'm a noob. How can two equal timestamps represent different times? Or do you just mean as presented in different timezones?
  - albertzeyer 14y ago
    See: http://en.wikipedia.org/wiki/Unix_time#Non-synchronous_Network_Time_Protocol-based_variant http://en.wikipedia.org/wiki/Unix_time#Non-synchronous_Netwo... The Unix timestamp 915148800 (from the table) represents two times (separated by one second from each other).
mmahemoff 14y ago
Formatting timestamps is a difficult decision for REST APIs to make. I polled my networks a while back and got pretty much divided responses [1]. In the end, I opted for milliseconds over ISO 8601 and I've been happy with that choice for ease of processing and debugging, especially as I'm now using the same values in the URL for caching purposes. 1. Google Plus conversation remains https://plus.google.com/106413090159067280619/posts/Wtkhk9jUVHp https://plus.google.com/106413090159067280619/posts/Wtkhk9jU... Good luck finding it on Twitter
- btipling 14y ago
  Your API should return Unix time. The end.
  - Dylan16807 14y ago
    Because when someone wants millisecond resolution the best answer is an integer number of seconds, right? mmahemoff, are you using actual milliseconds, which means you have to keep track of leap seconds?
    - mmahemoff 14y ago
      In my case, that level of precision isn't the consideration. I'm just expressing Rails' created_at and updated_at fields as integers instead of ISOs.
chmike 14y ago
I disagree that Timezone is just a presentation issue. When information is to be transmitted between timezones, you want to be able to determine the locale time of the sender or the information relays. This is why you'll find a time zone information in dates and time stamps in mail headers. This is also why ISO 8601 bothered to standardize specification of the time zone.
- deleted 14y ago
  [deleted]
tantalor 14y ago
> A time format without an offset is useless. Not necessarily. The time might be "floating time", i.e., depends on some local context. For example, the show starts at 10 pm 19 Jan 2012. You don't need the time zone because people going to the show know what 10 pm means locally.
- TeMPOraL 14y ago
  But this will still be useless anywhere else than "locally". The timezone just became implicit and lost.
  - taeric 14y ago
    Of course, if you add a timezone, you are likely in trouble when daylight savings time comes/goes. That is, shows don't change times just because you effectively changed timezones.
    - rmc 14y ago
      Depends on how you're storing the timezone. If you store it as "UTC+2 hrs", then yes you'll have a problem. If you store it as the tzdata "Europe/Sofia", then it'll take care of all the timezones.
      - taeric 14y ago
        I, of course, can not disagree. Just wanted to add to the voices that point out much in dealing with time in computers depends on your goal.
- DougWebb 14y ago
  Until your business grows and you're storing dates for shows in multiple timezones, or if you need to book shows and store dates in the future when DST may or may not be in the same state as when you store the date.
egonk 14y ago
> When storing time, store Unix time. It's a single number. This won't work for any system that allows users to schedule events in the future. Let's say that you agree for a meeting in Moscow 1 dec 2013 3pm. The actual (astronomical) time of meeting might change significantly if they change daylight saving rules. So you need to store something like "2013-12-01 15:00 Moscow", there's no way around it.
lmm 14y ago
Third point is worded really unclearly. GMT still = UTC (modulo seconds point that has been made elsewhere). BST = GMT + 1; British time is either GMT or BST depending on the time of year. Advice to display an offset when displaying a time is wrong or at least incomplete; you should display a symbolic timezone (i.e. "London" or "Eastern US" etc.) as that's what's going to be meaningful to someone reading it.
steve8918 14y ago
A very good open source package for handling date-time is ICU (http://site.icu-project.org/ http://site.icu-project.org/) which handles a lot of globalization/internationalization issues.
tommorris 14y ago
Time is even more complicated than that because of vagueness, either intentional or unintentional. Humans represent time vaguely rather than precisely. Computers tend not to. A while back, the British Library catalogue was put online as linked data. The library cataloguers had added details of author birth and death years but not always birth dates. But the data model normalised "1941" to "1941-01-01". (They fixed it after I moaned on Twitter.) You should be able to represent in your data model, date-time library etc. vague dates and relative dates. When I say someone was born in 1941, I don't mean they are born at 1941-01-01T00:00:00Z. When I say that something happened on Thursday, I don't mean it happened at 2013-01-17T17:04:00Z. It may have happened at any time on Thursday. If you store everything as Unix epoch seconds, good luck representing vagueness. ISO 8601 gets around the problem rather neatly by being big-endian and allowing omission at any point. Wanna say 1941? "1941". Want to say January 1941? "1941-01". Want to say January 14, 1941? "1941-01-14". Want to get very detailed? You can add points of a second: "1941-01-14T12:14:03.0482Z". You can do timezones. Eventually, we'll have an even better way of representing date-times that are more like bitmasks. We can then represent in a sane way the concept of opening hours. If you say the shop is open from 9am to 4:30pm Monday to Saturday, how do you do that? If I am digitising an old text of unknown origin, and on a page it says "January 12", and on the page after that it says "January 15", can I represent those with a reference to an implicit but unknown year? I don't know the year, but I do know that the January 12 instance comes before the January 15 instance on the next page. And at some future point, we might deduce from circumstantial evidence what year it was and the sequential ordering will still make sense. Relative dates would be useful here. ISO 8601 allows you to represent ordinal dates (the number of days since the beginning of the calendar year). Can your date-time representation deal with moveable feasts like Easter? Have a look at the WHAT WG Wiki page on Time to see more obscure use cases: http://wiki.whatwg.org/wiki/Time http://wiki.whatwg.org/wiki/Time Your date-time representation, storage format and calculation libraries are probably inadequate.
- mbq 14y ago
  The author only advices to use UNIX time to precisely measure time intervals and store precise timestamps; your points are of course valid, but they are not a reason to degrade precise data into human-readable pandemonium.
btipling 14y ago
I hate that Mixpanel and Google Analytics APIs return data in timezones, and the former doesn't tell you what the timezone is while the latter in v2 of their API can give you the wrong timezone.
piqufoh 14y ago
"the Prime Meridian was (arbitrarily) chosen to pass through the Royal Observatory in Greenwich" At the International Meridian Conference in 1884 41 delegates from 25 nations met in Washington, D.C., USA and selected the meridian passing through Greenwich as the official prime meridian due to its popularity - not an arbitrary decision ;) However, the French abstained from the vote and French maps continued to use the Paris meridian for several decades.
- Dylan16807 14y ago
  How is a popularity contest not a shining ideal of completely arbitrary decision-making?
  - rimantas 14y ago
    In this case popularity was the best criterion. Imaginary line is just that—imaginary line, no one is intrinsically better than other, so the one already popular was the best choice.
    - icegreentea 14y ago
      Well, the choice of GMT certainly made picking the international date line earlier. Certainly useful that 180 degrees around the world is mostly water.
rmc 14y ago
A good article, but the first point is (essetially) incorrect: UTC: The time at zero degrees longitude (the Prime Meridian) is called Universal Coordinated Time (UTC). If you stand at the Prime Meridian in London during the summer and ask someone what time it is, the correct answer will be one hour away from UTC.
nonamegiven 14y ago
At a more coarse-grained level, "Calendrical Calculations" by Dershowitz and Reingold is very interesting. It describes many different world and historical calendar systems, and includes lisp code to convert between them. Among many other problems, converting between calendar systems requires that you define when a day starts; there are many ways to define that among the systems. I think I have the latest, the third edition, https://en.wikipedia.org/wiki/Special:BookSources/9780521885409 https://en.wikipedia.org/wiki/Special:BookSources/9780521885...
- mgunes 14y ago
  For those less practically interested in the history of the computation of time, Arno Borst's "The Ordering Of Time: From The Ancient Computus To The Modern Computer" is recommended reading.
mongol 14y ago
Maybe the most insidious "bug" I ever had to solve: a performance test environment showing very strange behavior. Eventually I found that time jumped 30 seconds forward every minute, and soon after 30 seconds backward every minute. Had some kind of redundant time servers configuration, messing time up. Took longer to find out than seems reasonable, but it is the kind of thing you don't really expect.
greenyoda 14y ago
With all the discussion about how Unix represents time, it's surprising that the article made no mention of the Year 2038 Problem: https://en.wikipedia.org/wiki/Year_2038_problem https://en.wikipedia.org/wiki/Year_2038_problem
emikulic 14y ago
What's the deal with the "?v=1" ?

210 points by arb99 14y ago

Sigh. 50 year old numerical / geophysical / real time data acquisition/processing/interpretation programmer here. Unix Time isn't much chop for "real time" continuous data from the "real world" - it's those pesky leap seconds. If you bother to read the first paragraph of the wikipedia article on Unix Time you'll see : > Unix time, or POSIX time, is a system for describing instances in time, defined as the number of seconds that have elapsed since midnight Coordinated Universal Time (UTC), 1 January 1970,[note 1] not counting leap seconds.[note 2] It is used widely in Unix-like and many other operating systems and file formats. It is neither a linear representation of time nor a true representation of UTC. It follows on with a definition of Unix Time and points out various examples when it is ambiguous. These are real issues and can occur when missiles fly, when planes navigate, and when stocks are traded. Time is tricky.

martinced 14y ago
Time is tricky but there is a simple fact: it exists such a thing as the number of actual seconds elapsed since 'time x' (typically the Unix epoch but anything else will do too). No leap seconds issues. No 25 hours-day issues. No 23 hours-day issues. No 59 seconds minutes issues. No 61 seconds minutes issues. And all it takes to store a time like that is a 64-bit integer and it is very convenient. And a lot of software do precisely that. Most timestamps are just that: they don't care about "real world" details like leap-seconds, {23,24,25}hours per day, etc. Because, in many case, you really don't give a flying shit about the "real-world" time. What is your point? That a server that needs to open trade precisely at 9am has to take leap seconds into account? Sure. The article ain't disputing that. But a lot of application basically only need timestamps and everything becomes much simpler when one starts to reason in milliseconds since the epoch instead of "real world" time. Btw... Many of your "real issues" are due to programmers not realizing that they could have realized the exact same system but way more resilient had they knew to use timestamps instead of "real world" time. There was an amazing paper by Google on the subject by the way, where they explained how they were reconciliating their hundreds of thousands of servers' time by allowing it to be "not correct" over x minutes / hours to dodge many issues. And they clearly did emphasize that: a) these problems shouldn't have occured in the first place and were due to poorly designed programs (poorly designed in that they relied on real-world time instead of simply internally using milliseconds since the epoch) and b) most programs on earth do not give a flying ^^^^ if a time displayed to the user is one third of a second off...
- masklinn 14y ago
  > Time is tricky but there is a simple fact: it exists such a thing as the number of actual seconds elapsed since 'time x' Yes and no: it might just exist theoretically, but we have no way to get at it. The closest we have is TAI, which is only an approximation of the time elapsed at mean sea level on the earthly geoid, because clocks fall victim to gravitational time dilatation and compression. To accurately measure time you'd need a clock sitting perfectly still in space, and all other clocks in the universe would slowly drift behind it.
  - TeMPOraL 14y ago
    Not even that; relativity doesn't allow a single special frame of reference. There's no such thing as universal "time elapsed since x". Putting the clock in space would be as arbitrary choice as putting it on my roof (albeit definitely more practical).
    - ccozan 14y ago
      I think the one who solves this problem ( at least for humanity ) would get a Nobel. After the humans will wander thru space ( at some time - very probable), there will no time reference, but only time intervals ( like, day on a spaceship has 24h , etc ). So in this case, you would measure 86400 seconds and call it a new day. No more leap seconds, etc. Now the UNIX makes sense: count seconds since a certain event in time and meajure from there on, internally. Want to display it? Then use special computation to render it in the format ( read timezone, add relativistic skew, etc ).
  - Dylan16807 14y ago
    You're forgetting to factor in accuracy/resolution. TAI is completely perfect down to the picosecond level and further. That means you do have the number of actual seconds, milliseconds, microseconds, nanoseconds... You only encounter issues with TAI once you get down to femtosecond or smaller levels.
    - masklinn 14y ago
skrebbel 14y ago
Bummer. I hope that the author reads your comment, because it's otherwise a very useful page. I'd love to have a reference like the OP's about time and everything a programmer needs to know about it without problems like the one you're subscribing. This one's almost there. Thanks for clarifying!
lnanek2 14y ago
Yeah, even writing ditzy consumer apps on Android we're usually stuck with the number of milliseconds the system has been up if we want a reliable, step by step, count that doesn't jump around. Use something else and your animation delays will be screwed up occasionally, etc.. Of course then that number is useless outside that system and outside that boot of the system. Would have been much handier to have something better.

wereHamster 14y ago

He missed to explain TAI and its relation with the other times.

wereHamster 14y ago
Also, there have been many definitions of UTC, GMT etc over time. So to be truly correct you'd always have to mention which definition of UTC you are referring to when you use the term. And the ITU-R is debating to redefine UTC once again and remove leap seconds (http://www.ucolick.org/~sla/leapsecs/ http://www.ucolick.org/~sla/leapsecs/). If you are really interesting in the time stuff, here is a great resource: http://www.ucolick.org/~sla/ http://www.ucolick.org/~sla/.
- Dylan16807 14y ago
  >mention which definition of UTC you are referring to Doesn't the time value tell you? Or have there been temporally overlapping definitions of UTC that disagreed?
  - sla29970 14y ago
    There have been many defintions of UTC http://www.ucolick.org/~sla/leapsecs/timescales.html#UTC http://www.ucolick.org/~sla/leapsecs/timescales.html#UTC which do not agree. When an API does not match what the providers supply, which definition qualifies as "official"?

timthorn 14y ago

GMT is not an old name for UTC. From http://www.bis.gov.uk/files/file32707.pdf http://www.bis.gov.uk/files/file32707.pdf : Coordinated Universal Time (UTC) is based on International Atomic Time (TAI), but offset by an integer number of seconds so that it remains in approximate agreement with more traditional time scales based on the Earth’s rotation (i.e. Universal Time or, equivalently, Greenwich Mean Time). There is no natural gearing between TAI and GMT and so additional ‘leap seconds’ are applied when required to keep UTC in agreement with GMT to the nearest 0.9 seconds (see notes on time scales in Annex A for background information). UTC is the global standard for civil-time keeping today. It provides the most stable time base available because it is based on TAI, but also acts as a good approximation to its antecedent, GMT, for everyday purposes.

kator 14y ago

Funny I read the title and thought: TVM but then realized it was the age old quest for time keeping and dreaming of ways to track it with reasonable sanity.. :-) Bonus points for anyone who knows what I mean by TVM and can describe why every programmer should know about it..

chris_wot 14y ago
I can only think of Time Value of Money...
- nonamegiven 14y ago
  Unfortunately Google can only think of that too.
  - chris_wot 14y ago
    I was attempting to generate discussion. Your snark is not welcome.
    - nonamegiven 14y ago
      No snark intendend whatsoever. I was trying to figure out what the guy was talking about, and when I searched TBV the results were overwhelmed with Time Value of Money. I was observing that if the guy meant anything other than that, it would be all but impossible for someone to figure out if someone was not familiar with ... whatever he's talking about.
      - chris_wot 14y ago
        In that case, I owe you apology :( Love to know what this TVM is.
blowski 14y ago
You explaining it would be more useful than the bonus points.

InclinedPlane 14y ago

Who was the genius who decided that unix time should handle leap seconds? It seems like such an obviously bad idea. The biggest advantage of unix time has always been that it is monotonically increasing and that it is precisely defined as seconds since the epoch, period. And then all of that is now broken because of this decision. Also, does the problem that the leap second solves actually warrant all of the problems that it causes?

ArchD 14y ago
Since Unix time does NOT count the leap seconds, it allows the computation of the human-readable time from the Unix time without knowing when the leap seconds occurred. If knowledge of when the leap seconds occurred is required, then those events would need to be available somehow and used, introducing i) more expensive computation ii) difficulty in future-proofing an app since a reliable and updated data source supplying the times of leap seconds is required.
- ProblemFactory 14y ago
  The obvious solution would be to get rid of leap seconds entirely, and let UTC and solar time (when sun is highest at Greenwich) drift apart. Sun rising 30 seconds later every century is not really a problem in my opinion.
deleted 14y ago
[deleted]
ProblemFactory 14y ago
> Also, does the problem that the leap second solves actually warrant all of the problems that it causes? Definitely not. The only benefit of leap seconds is keeping UTC in sync with solar time: the sun is highest in the sky at 12:00 noon at Greenwich. But in the modern world, synchronizing solar time and clocks to <1 minute is of no value to anyone but nostalgic astronomers.
- sla29970 14y ago

patrickmay 14y ago

Erik Naggum's "Long Painful History of Time" (http://www.scribd.com/doc/93991574/Erik-Naggum—ALongPainfulHistoryofTime http://www.scribd.com/doc/93991574/Erik-Naggum—ALongPainfulH...) is well worth reading, too.

nonamegiven 14y ago
Working through it, thanks. The link in the paper's headers is still live: http://naggum.no/lugm-time.html http://naggum.no/lugm-time.html The home page refers to his ill health: http://naggum.no/ http://naggum.no/ And he in fact died in 2009 at 44: https://en.wikipedia.org/wiki/Erik_Naggum https://en.wikipedia.org/wiki/Erik_Naggum Interesting guy, involved in specifying the Internet from link level to mid-email level, worked on Emacs, and among the first Usenet users to be known for flaming. I vaguely remember the name in that context. On balance, a contributor.

stanonyime 14y ago

The relativistic warping of space-time forbids standardization of time measurements

nealabq 14y ago
Unless you also standardize the representation of inertial reference frames.

jimktrains2 14y ago

http://infiniteundo.com/post/25326999628/falsehoods-programmers-believe-about-time http://infiniteundo.com/post/25326999628/falsehoods-programm... http://infiniteundo.com/post/25509354022/more-falsehoods-programmers-believe-about-time-wisdom http://infiniteundo.com/post/25509354022/more-falsehoods-pro... Also: http://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names http://www.kalzumeus.com/2010/06/17/falsehoods-programmers-b...

speeder 14y ago

He did not mention the issue that some processors might have different time counting for each core, sometimes resulting in very silly situations if you are reading time in milliseconds or below scale. For example: I had a game that physics behaved "woobly" in multi core systems. The issue is that as the OS juggle your process around, you get the delta time since last physics step inaccurate, sometimes even with backwards jump in time ( and according backward movement in the game ). Later I saw someone showing the effects of this on the file system, with a multithreaded file copy resulting in very strange timestamps. For my game, the solution was force time-dependant threads to request affinity with one core.

dmbaggett 14y ago

Also worth noting: there are many time zone variations beyond DST, and some are defined in terms of partial hours (30 or 15 minute variations). True story: when I was at ITA Software, Orbitz ran hundreds of instances of QPX, our low fare search software, on their own servers. We had an ops fire drill one weekend because customers were complaining that the site was showing incorrect prices. The root cause? A single machine in their server farm had the time wrong, so advance purchase was computed incorrectly for any query sent to that particular machine. That was fun to debug. Even the meaning of, say, a minium stay requirement is hard to precisely define. If you fly across the international date line and back on a Friday night, did you have a Saturday stay or not? There are so many flights going so many places, questions like this actually come up in practice.

rolux 14y ago
For other - often difficult, and sometimes surprising - problems with time, see Peter-Paul Koch's excellent essay "Making <time> safe for historians": http://www.quirksmode.org/blog/archives/2009/04/making_time_saf.html http://www.quirksmode.org/blog/archives/2009/04/making_time_...

lysol 14y ago

> When storing time, store Unix time. It's a single number. Don't do this in your database. Use your datetime types. Please. You might save some work with your timezone but you're not going to be able to use intervals etc in a smart way. This would be even dumber with PostgreSQL, which has much more robust date/time functionality (being able to use the - operator with datetimes and intervals, for example) and completely sane timezone support.

RyanZAG 14y ago
Why can't you use intervals in a smart way? Single number is the easiest and fastest method to use for intervals. The only true problem with storing unix time is leap seconds and relativistic effects, which can both be safely ignored for most time keeping usages.
- ianburrell 14y ago
  PostgreSQL intervals are smarter than just number of seconds between two datetimes. They can represent "1 month" or "1 year" which can't be represented as a number of seconds. Also, intervals can be used between dates or times, not just datetimes.
  - jtheory 14y ago
    Exactly right; date arithmetic is horrible if you only have seconds to work with. Even just saying "24 hours from now" -- as your user would see it -- you'll sometimes get the wrong answer if you just add on 24 hours worth of seconds... suppose they have a DST shift tonight? What if you need to know how many days remain from today to March 1st of this year? ...well, are we in a leap year? Postgres datetime doesn't solve all of the problems that crop up, but it's certainly better than using unix time.
    - RyanZAG 14y ago
      24 hours from now is no problem if you're using unix time. Just add 24 hours worth of seconds -- when you convert back to the user's time, your conversion logic will take care of timezones. This is also incredibly important if serving an app over the internet as your user may have just flown from USA to Japan. Storing unix time solves this in the easiest way. Same for working out how many days remain - you convert March 1st to a unix date, get an interval in seconds, and then convert that into a human readable format on display. This isn't some revelation - unix time has been used successfully for decades now. EDIT: Also, Postgresql uses a number very similar to unix time in the actual storage of the date - it just handles the pretty display for you. So you're arguing for using the same thing whether you say use a postgre datetime or a unix date number.

NateEag 14y ago

As the article hints, even though timezones are usually presentation-only, there are some cases where business logic really does have to deal with timezones. If your software has business logic that cares about, say, what day a specific person perceived an event as happening, it needs to think about the timezone they have set. So, in such cases, you'd better keep a history of all the timezones they've ever chosen, and the time (UTC) they changed them. (Hopefully this lets someone else avoid my past mistakes.)

rmc 14y ago
Agreed. Often you need to care about timezones. Someone wants an alarm to go off at 8am every day. Well you need to start thinking about timezones, since that "8am" needs to be local time.
icambron 14y ago
I would go so far as to say that time zones are rarely presentation-level issues. Want to have a meeting every Tuesday at 2:00 New York time? Better make sure you know the New York part or daylight savings will break it. Want to wake up every day at 9:00? If you don't track the zone that preference is set in, you won't get that right when you travel. Time zone isn't a presentation-only thing because people's behavior depends on their time zone.

khitchdee 14y ago

I guess a related question is how do you synchronize a users perception of time with a program's perception of time -- a program that the user is interacting with. To effect this synchronization, you somehow need to know of the latencies involved in the user initiating actions that the program can sense and when those actions register with the program and in (latencies involved with) the program initiating actions that the user can sense and when the user senses those actions.

albertzeyer 14y ago

This topic comes up again and again. But I mostly read only about the problems, not about the solutions. This post advises to use Unix timestamps. That's already what I'm doing mostly everywhere. And it seems to me like most people do and like this is somewhat accepted as the best available option - but still not perfect. Then I am wondering again: Isn't there really a perfect solution? Esp., the problems with Unix timestamps are the non-monotony and that e.g. two equal timestamps can actually represent two different times (seconds). This could be bad in cases where seconds matter (e.g. some log-files which need to be very precise about time). Maybe TAI is and we all should store TAI timestamps instead of Unix timestamps? But is there an easy way to get TAI timestamps? Also, I haven't really seen other projects doing this - why? Maybe it is just too little gain over Unix timestamps and too less tools available to work with them... Btw., I just checked, there is http://cr.yp.to/libtai.html http://cr.yp.to/libtai.html and http://pypi.python.org/pypi/tai64n http://pypi.python.org/pypi/tai64n, maybe I should just start using that. This of course still doesn't solve anything about synchronization or inaccurate system clocks but it would be better than Unix times. Edit: Getting the TAI timestamp is probably not easy or maybe even not possible... I just saw this: https://github.com/stoni/libtai/blob/master/tai_now.c https://github.com/stoni/libtai/blob/master/tai_now.c ...

lmm 14y ago
On servers you control, do what Google does: skew time on leap-second days, making each second slightly longer, so that the day is still the normal number of seconds. Filter inputs coming from outside so that your internal systems never see a 60th second in a minute.
sopooneo 14y ago
I'm a noob. How can two equal timestamps represent different times? Or do you just mean as presented in different timezones?
- albertzeyer 14y ago
  See: http://en.wikipedia.org/wiki/Unix_time#Non-synchronous_Network_Time_Protocol-based_variant http://en.wikipedia.org/wiki/Unix_time#Non-synchronous_Netwo... The Unix timestamp 915148800 (from the table) represents two times (separated by one second from each other).

mmahemoff 14y ago

Formatting timestamps is a difficult decision for REST APIs to make. I polled my networks a while back and got pretty much divided responses [1]. In the end, I opted for milliseconds over ISO 8601 and I've been happy with that choice for ease of processing and debugging, especially as I'm now using the same values in the URL for caching purposes. 1. Google Plus conversation remains https://plus.google.com/106413090159067280619/posts/Wtkhk9jUVHp https://plus.google.com/106413090159067280619/posts/Wtkhk9jU... Good luck finding it on Twitter

btipling 14y ago
Your API should return Unix time. The end.
- Dylan16807 14y ago
  Because when someone wants millisecond resolution the best answer is an integer number of seconds, right? mmahemoff, are you using actual milliseconds, which means you have to keep track of leap seconds?
  - mmahemoff 14y ago
    In my case, that level of precision isn't the consideration. I'm just expressing Rails' created_at and updated_at fields as integers instead of ISOs.

chmike 14y ago

I disagree that Timezone is just a presentation issue. When information is to be transmitted between timezones, you want to be able to determine the locale time of the sender or the information relays. This is why you'll find a time zone information in dates and time stamps in mail headers. This is also why ISO 8601 bothered to standardize specification of the time zone.

deleted 14y ago
[deleted]

tantalor 14y ago

> A time format without an offset is useless. Not necessarily. The time might be "floating time", i.e., depends on some local context. For example, the show starts at 10 pm 19 Jan 2012. You don't need the time zone because people going to the show know what 10 pm means locally.

TeMPOraL 14y ago
But this will still be useless anywhere else than "locally". The timezone just became implicit and lost.
- taeric 14y ago
  Of course, if you add a timezone, you are likely in trouble when daylight savings time comes/goes. That is, shows don't change times just because you effectively changed timezones.
  - rmc 14y ago
    Depends on how you're storing the timezone. If you store it as "UTC+2 hrs", then yes you'll have a problem. If you store it as the tzdata "Europe/Sofia", then it'll take care of all the timezones.
    - taeric 14y ago
      I, of course, can not disagree. Just wanted to add to the voices that point out much in dealing with time in computers depends on your goal.
DougWebb 14y ago
Until your business grows and you're storing dates for shows in multiple timezones, or if you need to book shows and store dates in the future when DST may or may not be in the same state as when you store the date.

egonk 14y ago

> When storing time, store Unix time. It's a single number. This won't work for any system that allows users to schedule events in the future. Let's say that you agree for a meeting in Moscow 1 dec 2013 3pm. The actual (astronomical) time of meeting might change significantly if they change daylight saving rules. So you need to store something like "2013-12-01 15:00 Moscow", there's no way around it.

lmm 14y ago

Third point is worded really unclearly. GMT still = UTC (modulo seconds point that has been made elsewhere). BST = GMT + 1; British time is either GMT or BST depending on the time of year. Advice to display an offset when displaying a time is wrong or at least incomplete; you should display a symbolic timezone (i.e. "London" or "Eastern US" etc.) as that's what's going to be meaningful to someone reading it.

steve8918 14y ago

A very good open source package for handling date-time is ICU (http://site.icu-project.org/ http://site.icu-project.org/) which handles a lot of globalization/internationalization issues.

tommorris 14y ago

Time is even more complicated than that because of vagueness, either intentional or unintentional. Humans represent time vaguely rather than precisely. Computers tend not to. A while back, the British Library catalogue was put online as linked data. The library cataloguers had added details of author birth and death years but not always birth dates. But the data model normalised "1941" to "1941-01-01". (They fixed it after I moaned on Twitter.) You should be able to represent in your data model, date-time library etc. vague dates and relative dates. When I say someone was born in 1941, I don't mean they are born at 1941-01-01T00:00:00Z. When I say that something happened on Thursday, I don't mean it happened at 2013-01-17T17:04:00Z. It may have happened at any time on Thursday. If you store everything as Unix epoch seconds, good luck representing vagueness. ISO 8601 gets around the problem rather neatly by being big-endian and allowing omission at any point. Wanna say 1941? "1941". Want to say January 1941? "1941-01". Want to say January 14, 1941? "1941-01-14". Want to get very detailed? You can add points of a second: "1941-01-14T12:14:03.0482Z". You can do timezones. Eventually, we'll have an even better way of representing date-times that are more like bitmasks. We can then represent in a sane way the concept of opening hours. If you say the shop is open from 9am to 4:30pm Monday to Saturday, how do you do that? If I am digitising an old text of unknown origin, and on a page it says "January 12", and on the page after that it says "January 15", can I represent those with a reference to an implicit but unknown year? I don't know the year, but I do know that the January 12 instance comes before the January 15 instance on the next page. And at some future point, we might deduce from circumstantial evidence what year it was and the sequential ordering will still make sense. Relative dates would be useful here. ISO 8601 allows you to represent ordinal dates (the number of days since the beginning of the calendar year). Can your date-time representation deal with moveable feasts like Easter? Have a look at the WHAT WG Wiki page on Time to see more obscure use cases: http://wiki.whatwg.org/wiki/Time http://wiki.whatwg.org/wiki/Time Your date-time representation, storage format and calculation libraries are probably inadequate.

mbq 14y ago
The author only advices to use UNIX time to precisely measure time intervals and store precise timestamps; your points are of course valid, but they are not a reason to degrade precise data into human-readable pandemonium.

btipling 14y ago

I hate that Mixpanel and Google Analytics APIs return data in timezones, and the former doesn't tell you what the timezone is while the latter in v2 of their API can give you the wrong timezone.

piqufoh 14y ago

"the Prime Meridian was (arbitrarily) chosen to pass through the Royal Observatory in Greenwich" At the International Meridian Conference in 1884 41 delegates from 25 nations met in Washington, D.C., USA and selected the meridian passing through Greenwich as the official prime meridian due to its popularity - not an arbitrary decision ;) However, the French abstained from the vote and French maps continued to use the Paris meridian for several decades.

Dylan16807 14y ago
How is a popularity contest not a shining ideal of completely arbitrary decision-making?
- rimantas 14y ago
  In this case popularity was the best criterion. Imaginary line is just that—imaginary line, no one is intrinsically better than other, so the one already popular was the best choice.
  - icegreentea 14y ago
    Well, the choice of GMT certainly made picking the international date line earlier. Certainly useful that 180 degrees around the world is mostly water.

rmc 14y ago

A good article, but the first point is (essetially) incorrect: UTC: The time at zero degrees longitude (the Prime Meridian) is called Universal Coordinated Time (UTC). If you stand at the Prime Meridian in London during the summer and ask someone what time it is, the correct answer will be one hour away from UTC.

nonamegiven 14y ago

At a more coarse-grained level, "Calendrical Calculations" by Dershowitz and Reingold is very interesting. It describes many different world and historical calendar systems, and includes lisp code to convert between them. Among many other problems, converting between calendar systems requires that you define when a day starts; there are many ways to define that among the systems. I think I have the latest, the third edition, https://en.wikipedia.org/wiki/Special:BookSources/9780521885409 https://en.wikipedia.org/wiki/Special:BookSources/9780521885...

mgunes 14y ago
For those less practically interested in the history of the computation of time, Arno Borst's "The Ordering Of Time: From The Ancient Computus To The Modern Computer" is recommended reading.

mongol 14y ago

Maybe the most insidious "bug" I ever had to solve: a performance test environment showing very strange behavior. Eventually I found that time jumped 30 seconds forward every minute, and soon after 30 seconds backward every minute. Had some kind of redundant time servers configuration, messing time up. Took longer to find out than seems reasonable, but it is the kind of thing you don't really expect.

greenyoda 14y ago

With all the discussion about how Unix represents time, it's surprising that the article made no mention of the Year 2038 Problem: https://en.wikipedia.org/wiki/Year_2038_problem https://en.wikipedia.org/wiki/Year_2038_problem

emikulic 14y ago

What's the deal with the "?v=1" ?