13 ms·

Maya – Python Datetimes for Humans

350 points by kenneth_reitz 10y ago

Drdrdrq 10y ago
I have found that no matter what language/platform I use, the one thing that is always supported is UNIX timestamp. That makes date+time operations much easier: 1) Whenever dealing with users, use local tz. 2) Always save and manipulate in utc.
- masklinn 10y ago
  Except (2) doesn't necessarily work when events are local and you factor in timezone variations (both DST and actual TZ changes). There are classes of events where you're much better off with zoned local dates e.g. local meetings. Example: in early 2011, Samoa announced that on December 29th at midnight local they would switch their timezone offset from -11 to +13. Before that announcement (or at least before your timezone database has been updated), store as UTC a meeting in local Samoa time, and the user will miss their meeting by a day. Of course storing a meeting on December 30th local would also have been fraught as there is no December 30th 2011 in Pacific/Apia but that is a common occurrence historically due to the unsynchronised julian/gregorian switches e.g. none of the dates between February 16th and February 28th 1923 (included) exist in Greece, and the US doesn't have a 9/11 in 1752.
  - appleflaxen 10y ago
    I think the recommendation still holds. You want a meeting at 1100 local time. Meeting is stored for that time in UTC, on the correct date (before Samoa changed, we all understood what date you meant) Samoa changes the rules. The calendar doesn't change. This stuff is mind-bending, so I could be missing something, but a more detailed walk through your mental debugger might clarify.
    - achamayou 10y ago
      Classic counter example: storing business hours. If the store/exchange opens at 8:00AM every weekday in Somethingania/Foocity, you want to store 8:00 and the name/reference of the timezone. Because the store/exchange will still open at 8:00AM local time, even across DST, government-mandated timezone changes etc (largely, I'm sure there are exceptions but for stock exchanges there definitely aren't). School times, times for religious services, lots of times are relative to local time, which itself is subject to change against UTC (sometimes predictable, sometimes not, the government gets to update the timezone and the DST scheme arbitrarily). So only ever storing UTC and using local time purely for display isn't a sustainable one-size-fits-all option.
      - deleted 10y ago
        [deleted]
    - masklinn 10y ago
      > You want a meeting at 1100 local time. On a specific day. > The calendar doesn't change. Of course it does, you've stored a UTC datetime, the timezone database is updated, the UTC datetime now maps to the wrong time (and because Samoa changed its offset by 24h it actually maps to the wrong day entirely every single time). > This stuff is mind-bending, so I could be missing something, but a more detailed walk through your mental debugger might clarify. * while Pacific/Apia is UTC-11, you create a meeting for January 5th at 11 local * this is stored as 2012-01-05T22:00:00+0000 * tzinfo gets updated with Pacific/Apia at UTC+13 * your calendar now tells you your meeting is on January 6th at 11 local * you're a day late
      - mjevans 10y ago
        You're both correct. The scheduling library should be more upfront about /what/ is being agreed to and force the user to pick. P) The event is at a precise internationally recognized moment (better for co-ordination globally). R) The event is in local time (like a lunch date) and expected to remain colloquially fixed. In the case of the first you store a precise UTC timestamp and drop the timezone (it's UTC). In the case of the second you store /how/ to pick a UTC timestamp based on a time in a timezone.
    - tedunangst 10y ago
      Samoa did change the calendar. The local <-> utc mapping changed. In order to update the stored database, you need to convert all times to local with the old mapping, then convert to utc again with the new mapping.
  - vacri 10y ago
    Using one of the most extreme edge cases is not a good argument. How often is Samoa (or another small dateline-neighbour country) going to 'jump sides'?
    - masklinn 10y ago
      > Using one of the most extreme edge cases is not a good argument. 1. How is it not a good argument? It's an actual historical fact which is not yet 5 years old rather than some intellectual exercise, you can hardly go better than "this stuff happened not 5 years ago". 2. The only "extreme" part here is the magnitude of the drift, it's a clear example and demonstration of the issue. > How often is Samoa going to 'jump sides'? How is that relevant to the problem specifically and clearly existing? Does it matter if your calendar is 24h off or 6h off? Your system is FUBAR either way. > or another small dateline-neighbour country The dateline isn't even relevant, are you somehow trying to win a prize in missing the point?
      - vacri 10y ago
        > It's an actual historical fact It's actual historic fact that the gregorian calendar we use skipped 11 days, but I don't see your suggestion solving that. Again, it was "just magnitude". > Does it matter if your calendar is 24h off or 6h off? Your system is FUBAR either way. Your theoretical appointment-haver might not blink at an appointment at a normal time + 1 day in the future, but might blink when their appointments start in the early afternoon and continue until midnight. The fact is that if you skip days, then it doesn't matter if you use local or UTC time. > are you somehow trying to win a prize in missing the point? I'm not the cretin who thinks that timekeeping is a binary good/bad state. Some applications have troubles when they're milliseconds out. Others have quite a bit of lag time that they can handle. A daily to-do list just needs to appear on the right day, for example; DST means little to it.
    - toyg 10y ago
      Timezones change all the time. There are almost 200 countries in the world, some so large that even their sub-states can and do change timezone rules on their own (US, Russia, India, China...). This is why Linux distributions have a dedicated package listing timezone info that is updated almost weekly, and Windows has similar arrangements. Any solid datetime implementation better consider that time is not, in fact, immutable - even UTC can and will shift here and there.
- paulddraper 10y ago
  Sure, if you limit yourself to timestamps. But dates/times have much more notions than an instant. Like, that I have a recurring event every weekday at 9am local time. Or that the weekend is two days (hello, DST). And even if you only need timestamps, "Whenever dealing with users, use local tz" isn't exactly a magic wand either. Also, "unix timestamp" isn't exactly one data type. Sometimes you need greater precision than one second.
- PeterisP 10y ago
  "Always manipulate in UTC" fails for datetime distance computation. Two very common examples: For a recurring scheduled event, "Same time tomorrow" may require adding 24, 23 or 25 hours to the UTC time, depending on the TZ. The question "how many full days have elapsed between two timestamps?" may need to return 1 for 23.5 hour distance or 0 for 24.5 hour distance depending on the TZ. I'd be so happy if we all could agree that a day always has 24 hours, but that doesn't seem likely any time soon.
charlex815 10y ago
I suppose this beats breaking out time delta, but I think it'll be hard for me to see an actual use in my projects that I couldn't accomplish with maybe just a couple extra lines
- kenneth_reitz 10y ago
  you'd be surprised, the amount of headache-inducing code that timezones can cause, especially when dealing with servers with different locales than your development machine when doing timezone algebra. This API avoids that problem entirely.
  - SFJulie 10y ago
    It does not: timezone (TZ) definitions are politicial, inaccurate, and stupid and non versioned. The problem with TZ is TZ in its core definition. 1) they always change: if you have not updated your TZ since 3 months they are probably inaccurate 2) your TZ definition maybe accurate but they may not have been applied for real in the concerned zone 3) you can have different local time for the same longitude 4) you can have different days on the same longitude ... 5) TZ are not versioned, if the TZ changed between 2 records you may have made, you have inaccurate intervals stored. We do NOT have an API to take TZ change over time in consideration. 6) CEST/DST is breaking the axiom that time is a growing monotonic function References 8.5.3 TZ in postgresql man https://www.postgresql.org/docs/9.2/static/datatype-datetime.html https://www.postgresql.org/docs/9.2/static/datatype-datetime... Computerphile what's wrong with timezones https://www.youtube.com/watch?v=-5wpm-gesOY https://www.youtube.com/watch?v=-5wpm-gesOY
    - DasIch 10y ago
      The TZ database has versions and you could at least in theory expose that at a higher level in libraries.
      - nicpottier 10y ago
        Pretty sure pytz will hand you the appropriate timezone for a given date. There aren't version numbers after all but it can do the right thing based on when you say it is. There is some weird behaviour due to this when trying to get a timezone without an associated date as it doesn't default to now. Then again you probably have all sorts of DST bugs if you have places in your code that do that.
bndr 10y ago
How does it differ from Arrow which was made several years ago? [1] [1] http://crsmithdev.com/arrow/ http://crsmithdev.com/arrow/
- smegel 10y ago
  That would be nice to know. Arrow still seems active, and has been consistently so over the last 3 years. I have a bit of doubt about a time library that defaults to UTC but uses human phrases like "tomorrow". Who thinks about UTC tomorrow?
  - kenneth_reitz 10y ago
    that's why there's `maya.when('yesterday', timezone='US/Eastern')` :)
    - Drdrdrq 10y ago
      That's great, but why this then: >>> tomorrow = maya.when('tomorrow') <MayaDT epoch=1481919067.23> >>> tomorrow.slang_date() 'tomorrow' >>> tomorrow.slang_time() '23 hours from now' Huh? Given that we never specified timezone, I would expect 24h (unless DST change happens). This is actually my biggest gripe with date+time libs. Imho API should always be explicit in what its default TZ is. EDIT: still, appreciate what you are doing. Requests rule, and there are many other areas (including datetime) that need libs with better APIs. Thumbs up!
      - StavrosK 10y ago
        I'm pretty sure that it's 23 because some time passed between keystrokes, so the delta was 23:59:50, and he only keeps the most significant digit. I had to do this exact thing yesterday for timetaco.com.
        prashnts 10y ago
        Shouldn't it be immutable? I'd expect this to be a default behaviour.
SFJulie 10y ago
It is heartbreaking to see such a contributor like Kenneth Reitz in conferences. I really enjoy using Kenneth Reitz's creation : I would like to see him smile. He is probably in the top 10% of the developers, he does not profit of his talent as much as an idiot that just code shit. When I sample over the vast majority of free software coder I know (100s) over the non free one (~same) and I compare the wealth/recognition I can't help but notice that for some of the 1% top FOSS devs you do have a breakthrough, but then, blind obedience seems to give more results in terms of social and wealth status. Something is rotten in the state of coding. My advice: don't make free software enjoy life, and close your ears to the voice of the voice of the crowd open them to the one you care about. And if you really want to make FOSS code, do it like others do knitting, pastry, moonshining, brewery. Keep the fun in your life.
- kenneth_reitz 10y ago
  Do I look unhappy at conferences or something?
  - deleted 10y ago
    [deleted]
  - cthalupa 10y ago
    I mean, they can get pretty smelly.
  - SFJulie 10y ago
    At pycon montréal you were looking a tad tired. Maybe I mistook jetlag for something else?
    - orf 10y ago
      Something else being a void within his soul, representative of 'something rotten at the heart of programming'? That's quite a leap to make from seeing someone tired.
  - jMyles 10y ago
djoser 10y ago
I understand the link with the maya believes but for a "digital" feature, this sound a lot like an Autodesk product... That said, nice API!
krautsourced 10y ago
I wonder if the naming isn't a bit unfortunate, seeing that Maya is one of the major 3d packages out there and googling for Maya and Python will almost always lead there (also, not sure whether Autodesk might object...)
- duncanawoods 10y ago
  Yep. Picking cute short brand names for libraries as though they were startups clutters up the namespace of programmer's conversational language causing ambiguity and obfuscation. It would make life easier if we adopted an informal naming convention for small libraries e.g. <lang><brand><purpose>. Lang is useful when libraries get ported so you get e.g. PyMayaTime, GoMayaTime etc. Still sounds cute but you know what its for when you pick up someone else's code and its easy to google.
  - Senji 10y ago
    Picking names which collide with stuff seems to be the standard these days. Don't get me started on Google(Chrome, Go)
- agumonkey 10y ago
  Years ago Maya had Python, now Python has Maya. The circle is complete. ps: note that the Maya in Alias Wavefront (now Autodesk but nope) Maya is not that Maya, it's Sanskrit for illusion, which is what Maya was all about considering what it did to your dollars.
- gohrt 10y ago
  MayaTime would be a responsible name for this project. Brand Name + Hint about is functionality. Like "JodaTime" and "Golang"
- dagmx 10y ago
  Also unfortunate because the python api in Autodesk Maya is in the maya namespace. So this library would need to be installed as a separate name to be usable there.
RubyPinch 10y ago
<MayaDT epoch=1481850660.9> not the most human readable is it? I guess it would help with like, "X happened before Y" situations, but I don't think I'd trust my eyes for that! I think I would of preferred the "this happened in X month" case instead, I find it easier to trust my eyes for that, instead of trying to look for differing digits! - - - UTC default is a godsend though
- jvdh 10y ago
  Fortunately it is really easy to print it in other formats. Which is certainly not the case for the standard datetime library.
meltingwax 10y ago
This is designed for programmers of user facing applications. For science and engineering, it does not address the problems, eg lack of leap seconds.
- cgvgffyv 10y ago
  You need to wait for another rewrite with a cute name for these.
- masklinn 10y ago
  > For science and engineering, it does not address the problems, eg lack of leap seconds. What do you mean by "lack of leap seconds"? A non-unix-timestamp-based time library?
  - meltingwax 10y ago
    When the mapping between a time_t and a "human readable format" (hours, minutes, seconds, etc) takes into account extra leap seconds. You could use time_t underneath, if that's what you really wanted to do. As my co-worker once put it, "time is a four letter word." At the very least, there is a need for showing the occasional 61st second of a minute, and adding a time and timedelta where leap seconds is taken into account.
    - masklinn 10y ago
      > When the mapping between a time_t and a "human readable format" (hours, minutes, seconds, etc) takes into account extra leap seconds. I still don't understand what you mean. > You could use time_t underneath, if that's what you really wanted to do. Well you've got unix timestamp (UTC) which "skips" leap seconds but as a result is very easy to map to "human time", or you've got TAI[0] which includes leap seconds but doesn't allow dates in the future since you don't know where and when new leap seconds will be added long in advance, and now you need regular updates/permanent connectivity so you can remap TAI onto human time. And you need an NTP replacement, though I guess having a GPS unit in everything would do. [0] and GPS time which is just TAI + 19s
sametmax 10y ago
I wish Kenneth would have contributed to an existed project for once. Arrow and pendulum (my current favorite) have a very decent API. The later one is especially well tested for the numerous corner cases of date handling, which I doubt Kenneth got right on first try. For request, a full rewrite made sense because urllib sucked so much and we had no good alternatives. But for date time, alternative exists and they are good. Do not split the open source effort, join forces!
- KODeKarnage 10y ago
  Who wishes that Requests didnt exist? That guy!!! ^^^^^^^
  - ergo14 10y ago
    You comment makes no sense...
    - KODeKarnage 10y ago
      He wishes the guy was the sort of guy who contributed to other projects rather than doing his own thing. But if he WAS that sort of guy we wouldn't have Requests. I thought you guys were supposed to be smart.
      - stable-point 10y ago
        No, he doesn't: > For request, a full rewrite made sense because urllib sucked so much and we had no good alternatives.
        KODeKarnage 10y ago
        > I wish Kenneth would have contributed to an existed project for once. Yes, he does. He wishes Kenneth was not the same as the existing Kenneth. He emphasizes this with the "for once". Kenneth went his own way with Requests, and everyone is better off for it. You do not want a guy with that track record abandoning his work patterns. I say, let the man be. Whining about "fracturing" in open source ignores those important things which makes progress possible; conflict, competition, and triumph. I see the original comment as being little more than a "yeah, it's nice, but you know, we already formed a committee and working group to do this thing".
dom0 10y ago
> Datetimes are a headache to deal with in Python, especially when dealing with timezones, especially when dealing with different machines with different locales. Anything with date/time calculations is always a pain, probably doesn't really have much to do with the library/language itself, but that the abstraction level that's used (and typically used in other libraries) means that the complexities of calendar and time systems are sprinkled all over application code. I do have to notice here that always using UTC is not always the right thing to do. For example, evaluating rrules in UTC is rather error-prone (DST).
- mjevans 10y ago
  If you're mixing UTC and DST you are doing something wrong.
  - deathanatos 10y ago
    I think what dom0 means is a situation like the following. Say you want to schedule, on a calendar, an event that happens daily at 2PM in your local timezone, say America/New_York. If you record that even as happening at 2200 UTC each day, when DST rolls around, it'll shift to 1PM or 3PM in the local timezone. This is not what the user wants. Here, despite storing everything in UTC and attempting to apply DST as late as possible as if it were a "display issue" — normally the right thing — here results in the wrong outcome.
    - mjevans 10y ago
      In your example what I said is still true. The user didn't want a precise coordinated time (DST is not a factor for storage/recall); the user wanted a fuzzy reference to local time (DST is a factor for evaluation).
BoppreH 10y ago
Awesome! Timekeeping is hard and I'm glad we now have one more tool do deal with it. One thing that is bothering me is that when you ask for `maya.when('tomorrow')`, or give only a date, you get back a timestamp with millisecond precision, representing 00:00 of that day. I understand this simplifies the implementation, but shouldn't `tomorrow` be a range, from 00:00 to 23:59? Treating imprecise dates as ranges would allow for stuff like # Did this event happen "yesterday" according to the US/Eastern time zone? timestamp in maya.when('yesterday', to_timezone='US/Eastern') # Get me all events that happened last Friday (UTC) [event for event in events if event.time in maya.when('2016-12-16')] Maybe I'm being naive, and there's a reason why this won't work, but this seems the way most humans deal with time. PS: it failed to install on Windows, so I opened an issue at https://github.com/kennethreitz/maya/issues/10 https://github.com/kennethreitz/maya/issues/10
- tokenizerrr 10y ago
  How would you serialize that range to, for example, ISO8601?
  - cheapsteak 10y ago
    From wiki: ISO8601 has four ways to express a time interval: 1. Start and end, such as "2007-03-01T13:00:00Z/2008-05-11T15:30:00Z". 2. Start and duration, such as "2007-03-01T13:00:00Z/P1Y2M10DT2H30M" 3. Duration and end, such as "P1Y2M10DT2H30M/2008-05-11T15:30:00Z" 4. Duration only, such as "P1Y2M10DT2H30M", with additional context information http://en.wikipedia.org/wiki/ISO_8601#Time_intervals http://en.wikipedia.org/wiki/ISO_8601#Time_intervals
  - BoppreH 10y ago
    Excellent question. ISO 8601 has features to encode time intervals ( https://en.wikipedia.org/wiki/ISO_8601#Time_intervals https://en.wikipedia.org/wiki/ISO_8601#Time_intervals ). If that's too verbose, and we only need to represent integer intervals, one could store a truncated representation, where the omitted parts mean "any" (like * in cron). For example, `2016-11` could mean the entire month of November. But those are just ideas for the representation. The important feature is that not all objects are precise points in time, but may be a block/interval/range.
sebastibe 10y ago
Hopefully we will have a "API for Humans" for each of the standard library modules.
etanol 10y ago
Not to undermine its merit, but most of the dirty work and heavy lifting is done by its dependencies: https://github.com/kennethreitz/maya/blob/d57a78c6bc6b5295f7d7207be0ee7cf0d62f063f/setup.py#L14-L20 https://github.com/kennethreitz/maya/blob/d57a78c6bc6b5295f7... And i18n support in humanize is a bit lacking, as it only translates to French, Korean and Russian. Given that most of the translations needed to render human dates can be found in the CLDR database, maintaining their own looks like a bit of a wasted effort. Reference: http://cldr.unicode.org/ http://cldr.unicode.org/
- masklinn 10y ago
  > Given that most of the translations needed to render human dates can be found in the CLDR database, maintaining their own looks like a bit of a wasted effort. You still need to parse the CLDR (the formats are pretty wonky) and provide APIs for its functions. Of course that's what Babel (http://babel.pocoo.org http://babel.pocoo.org) does.
  - thrownblown 10y ago
    and here I thought babel was for es2015
ben_jones 10y ago
People are asking why this is needed compared to Arrow and a few other libraries. Personally I was never happy using the python time libraries, it always felt like my use cases were slightly different then the library, and I always found myself getting frustrated over little things here and there. I think there are a LOT of use cases for time in python applications and there is plenty of room for small libraries to satisfy these conditions, versus one monolithic time library that attempts to solve all.
fnord123 10y ago
Python datetimes used with timezones, even UTC, are ridiculously slow and bloated. It's puzzling why this doesn't wrap np.datetime64 instead. Or wrap boost.datetime. There are many good options to claw some performance back so it's really head-scratch inducing that someone would recognize that stdlib datetime is a dog and then wrap it instead of scrapping it.
- aangjie 10y ago
  > It's puzzling why this doesn't wrap np.datetime64 instead. Perhaps to avoid the numpy dependency. I've found numpy a bit heavy and wouldn't want to depend on it simply for the sake of some datetime syntax. May be boost.datetime or even numpy's implementantion on datetime, but that sounds like a lot of maintenance.
  - guitarbill 10y ago
    Agree, 100%. E.g. numpy in a virtual environment can be a real pain. Best case, it takes a bit longer to install. Worst case, it's on a system that's missing some OS specific dependencies. "Works for me" is not okay for stuff on PyPI.
    - aangjie 10y ago
      Hmm.. that's making me rethink that 'datascienceutils' project I put up on pypi.. May be work on getting it to stable version before pushing to pypi..
- deleted 10y ago
  [deleted]
- daenney 10y ago
  > It's puzzling why this doesn't wrap np.datetime64 instead. Or wrap boost.datetime. I don't find it that puzzling. It's meant to help deal with timezones, efficiency isn't necessarily part of that equation. And if I were to write a simple CLI or web app I would avoid anything that would pull in a dependency like numpy or require a C-library plus its bindings. If you need to manipulate a huge amount of date times and this is something your app is constantly doing then sure, optimise for that.
joaoqalves 10y ago
Yet another library to handle dates in Python. Oh, boy... https://xkcd.com/927/ https://xkcd.com/927/
partycoder 10y ago
[offtopic] I do not like the "for humans" thing. What could it be for instead? for kangaroos or giraffes?
- frou_dh 10y ago
  Popo: An Artisinal Handcrafted Curated Simple-yet-Powerful Library (for Waterfowl). By L'Wren Scott.
jMyles 10y ago
What's up buddy? My only real question: > rand_day = maya.when('2011-02-07', timezone='US/Eastern') This returning an object representing a DateTime on the 6th (in UTC time) strikes me as perhaps "not for humans." If I just see that line casually, I think I expect to get a Date and for it to be the 7th. It looks like, in order to get this (arguably expected) object, I need to take the resulting MayaDT epoch and run its `datetime` method, passing naive=True? And I also see that _tz can only ever be pytz.timezone('UTC') - is this the result of some belief that timezones are illusions or something? :-) For a while, I have kinda thought that timezones foment a confused mental model of time and teamwork. I prefer to think in terms of the astronomy - it's not actually a different time anywhere else, it's just that the sun is at a different position relative to the rest of the earth (and thus, ones faraway teammates and loved ones). Anyway, thanks for yet another set of interesting ideas. Hope you are well.
- guitarbill 10y ago
  It's worth thinking of it the same as strings. Strings can have several encodings, e.g. ASCII, UTF-8 and a hundred others. These are external representation. You don't want to parse the string every time you have to deal with it, so you parse it and store it using (ideally) a single internal representation (e.g. UCS-2, UCS-4). Then, when you need to pass the string to something else, you encode again. Same with dates. Instead of trying to deal with and store all possible representations (format, timezones, etc), you convert all dates to a single representation (hence UTC). When you need to output the date, encode it into whatever format and timezone you need. It's much cleaner this way, because there's less chance you'll mix up representations. So the "for humans" part is more "for developers, which are also humans but like to pretend they aren't and never make mistakes".
aibottle 10y ago
Great job, Mr. Reitz. I think I will use this in every project from now on. Thanks!
japhyr 10y ago
I'm competent with strptime(), but this looks really nice: # Automatically parse datetime strings and generate naive datetimes. >>> scraped = '2016-12-16 18:23:45.423992+00:00' >>> maya.parse(scraped).datetime(to_timezone='US/Eastern', naive=True) datetime.datetime(2016, 12, 16, 13, 23, 45, 423992) I'm happy not to have to write formatting arguments to strptime() anymore. Do the other datetime libraries have similar parsing functions?
- detaro 10y ago
  Maya actually uses the dateutil.parser.parse function from dateutils under the hood, which as far as I remember is the most comprehensive such function among the typical python datetime libs.
  - japhyr 10y ago
    Wow, I've been using datetime for years, without ever noticing the dateutil library. I'll use this from now on.
Goopplesoft 10y ago
Kenneth Reitz is a testament to how important good interfaces are in the developer community. There are literally 0 interesting things in the code (all dependency driven, https://git.io/v15i3 https://git.io/v15i3). It does have a nice interface and because of this it will probably become one of the more popular python datetime libs.
- vegabook 10y ago
  Arguably bringing multiple dependencies under one roof, with a nice, human-readable interface, is exactly what the value is here. This SO answer from Pandas lead Wes Mckinney says it better than anything I could comment. Python datetime/timestamp management: "Welcome to Hell" http://stackoverflow.com/questions/13703720/converting-between-datetime-timestamp-and-datetime64 http://stackoverflow.com/questions/13703720/converting-betwe... I run batch and real time financial feeds into Python from multiple sources, inevitably from different programming language paradigms, and with different time/date/timezone conventions, and then I allow user interaction with it in their own lingo for dates and times, and my experience is that it is indeed hell consolidating all this into a common tongue. The "slang time" idea suggest to me that this library understands the need for flexibility and malleability in this very disjointed and often frustrating part of the Python ecosystem.
  - dom0 10y ago
    I think there are two separate issues here. One thing is that some libraries bring their own not-really-different-just-different-enough-to-break-things date/time types to the table. That can be worked out over time, not that big of a problem, just inconvenient. The other is that I've grown to think that the abstraction commonly used, seeing date/time as something like (year, month, day, hour, minute, second, subsecond, timezone) is in itself not really suitable, because the derived interfaces (like accessing and modifying days, months etc. separately) mean that most code working with these will inevitably be bug-ridden. The root cause for this is imho that date and time are extremely complicated and it's usually unclear what the correct behaviour would be in any of the many, many edge cases. Some examples: 1.) One week after `this` date. Does this mean, add seven days? What about leap days? Should the result be the same weekday, just in the next week instead? 2.) Start and end time of a process should be recorded. The process starts and finished. All good. The duration of the process is -1 hour and 5 minutes. Should instead a start timestamp be recorded and a monotonic duration? Or perhaps we should rather note the begin of a monotonic period and record an end timestamp, deriving the start timestamp? Which is correct? 3.) One ^W two words: recurrence rules 4.) DST 5.) Combining 3.) and 4.)! dom's rule of thumb: if code does addition or substraction with some measure of time, it's probably wrong.
d0m 10y ago
Yeah.. datetime is one of the rare part of the Python API that I really hate using. It's just badly designed, e.g. simple things are hard and confusing.
jchassoul 10y ago
I don't understand the author's effort on solving this "problem"... I wonder if he considered at some point make a contribution to stable open-source existing libraries like arrow that even claim inspiration on one of his projects for the API. What's not for humans on the arrow API? what are the arguments for basically start yet another time library from scratch?, why is making a contribution not an option? probably the author of arrow will love your contributions and will love to hear the arguments on the changes you propose.
- scrollaway 10y ago
  As rich as it is to tell Kenneth to "consider contributing to existing open source libraries", I have to agree: what's so exceptional about this library when Arrow is already a thing? http://crsmithdev.com/arrow/ http://crsmithdev.com/arrow/
- detaro 10y ago
  > what are the arguments for basically start yet another time library from scratch? It's not exactly a time library from scratch, if you look at the amount of existing libraries it imports and the small code size (which is a good thing!). Why should he not offer a differently flavored API on top of existing parts? My impression is that most of these libs start with a strong idea what the API should be like, so large changes there are unlikely to happen.
niftich 10y ago
The link to its github was posted on HN when it was brand new, and I left a comment [6], which I'm reproducing here -- since most of my points still apply. (...) take my very early comments with a grain of salt -- they refer to the progress as of this commit [1]. I love Requests -- its API design is fantastic, and manages to distill down most of a complex problem domain to a clean, dare-I-say, elegant API. So I can eagerly anticipate this design applied to datetimes. But the progress being shown so far is definitely not it. >>> tomorrow = maya.when('tomorrow') <MayaDT epoch=1481919067.23> Why is "tomorrow" a precise-to-centisecond, infinitesimally small point on a giant cosmic timeline? I'm reasonably sure it's an abstract concept that describes the calendar day that starts after the current calendar day ends. At least, Pendulum normalizes tomorrow() and its ilk to represent midnight on the given day [2], while Delorean's natural language methods [3] like next_day() advance the day while leaving the time-of-day unchanged, but the method name makes this fairly clear. Even Arrow, which is heavily inspired by Moment.js to the point of conflating every single datetime idea into the same class, opts for mutators that are still more clear [4]. > Timezones fit in here somewhere... Yeah, this needs more work. Java 8 / Threeten, and its predecessor Joda-Time took the approach of clearly modeling each and every concept that humans actually use to describe time; even if you take issue with their API, the designers have clearly done their homework, and their data model is solid. Formats like TOML wrestled with datetimes and realized [5] that datetimes aren't just woefully underspecified in most other specs and APIs, but that they're frequently mis-modeled, so they adopted large portions of Threeten's data model. Cases like this should merit strong consideration from anyone trying to propose new datetime APIs today. [1] https://github.com/kennethreitz/maya/commit/ecd0166ba215c1a5.. https://github.com/kennethreitz/maya/commit/ecd0166ba215c1a5.... [2] https://pendulum.eustace.io/docs/#instantiation https://pendulum.eustace.io/docs/#instantiation [3] http://delorean.readthedocs.io/en/latest/quickstart.html#nat.. http://delorean.readthedocs.io/en/latest/quickstart.html#nat.... [4] http://crsmithdev.com/arrow/#replace-shift http://crsmithdev.com/arrow/#replace-shift [5] https://news.ycombinator.com/item?id=12364805 https://news.ycombinator.com/item?id=12364805 [6] https://news.ycombinator.com/item?id=13190314#13190657 https://news.ycombinator.com/item?id=13190314#13190657
batbomb 10y ago
Kenneth: It would be really cool if you could put in support for TAI and MJD.
aivosha 10y ago
i dont trust any piece of software that has all these made-up, unrelated, non-reflecting names. I mean come on, arrow, maya ? the main challenge about software engineering is correct naming. If you fail there you pretty much fail in the rest. Same goes for actually builtin name, "datetime". There is no such thing in real life as datetime. There is date and there is time. They are very separate notions and the root of the problem IMO is in trying to pile them together.
ianamartin 10y ago
I think its fascinating that the community has no consensus about a datetime library. A lot of Python is really solved. We don't argue about using requests (a not-coincidental example). If you're using Python, and you need to deal with http, you use requests. Everyone knows this. There are basically 3 platforms for web frameworks. Flask, Pyramid, and Django. Maybe we're a little more dissolute than C# or Ruby folks, but that's pretty impressive considering how much we Python people like to roll our own. The fact that there is real disagreement about this among ourselves about this particular issue says to me that this is more about the difficulty of the problem than it is anything else.
- NelsonMinar 10y ago
  Related: the datetime library in pretty much every language is terrible. The only one I've ever used that I remember not actively hating was MomentJS. Python's history of time types was terrible, as well. Particularly the era when we just used tuples of length 8 or 9 that weren't timezone aware.
  - bjacobel 10y ago
    Funny that JavaScript would find community alignment on something (anything!) where Python can't. I guess that's a testament to the work of the Moment creators.
    - niftich 10y ago
      It is. Datetime is an unpleasant problem domain, and Tim was active both on SO [1] and Github, arguably so much so that it was to the detriment of his health [2]. In the case of Moment, a little promotion went a very long way, and the community was eager to take what's given so they wouldn't have to think too much about datetime, and focus on debating how to arrange and design their applications instead. Moment's design is a testament to "let's not think too hard about datetime", and I mean that both as a compliment and a critique. [1] http://stackoverflow.com/users/272034/timrwood?tab=summary http://stackoverflow.com/users/272034/timrwood?tab=summary [2] https://news.ycombinator.com/item?id=12175369 https://news.ycombinator.com/item?id=12175369
diyseguy 10y ago
But does it get DST correct. I haven't found any python time libraries that do.
- sdispater 10y ago
  You can take a look at pendulum: https://github.com/sdispater/pendulum https://github.com/sdispater/pendulum A lot of effort has been put into getting DST right so I hope it will be what you are looking for. Disclaimer: I am the author of Pendulum :-)
- robochat 10y ago
  Interestingly, Python 3.6 has just tried to fix the DST problem in the datetime library itself. See PEP 495 https://www.python.org/dev/peps/pep-0495/ https://www.python.org/dev/peps/pep-0495/
ak217 10y ago
I like Kenneth's work, but "I wrote a new datetime library" is a cliche now. We have datetime, dateutil, pytz, babel, arrow, pendulum, delorean, a bunch of lesser known stuff, and now this. I have yet to see the need for anything but the first four.
- toyg 10y ago
  Seems to be a case of "Python Web Framework"-itis: it's relatively simple to hash up something from scratch that covers one's particular use-case (in this case, it seems, a slightly smarter parser), but it's actually hard to properly cover all use-cases. This is what happened with Python web frameworks, a scene which was heavily fragmented before Django and Flask basically solidified the two main communities of users ("I need everything and the kitchen sink" / "I need the bare minimum to get going"). The same happened with urllib2/urllib3/httplib/ etc before Requests appeared. Nobody seems to have pulled this trick for datetime libraries yet, so here's a new contestant.
Animats 10y ago
Bikeshedding. The Python "datetime" module does almost all of this. (Although I did try to get ISO8660 parsing put in, after finding eight libraries for it, all of which were broken in some way.)
Walkman 10y ago
This is a joke. The whole "library" is 200 lines, has 9 tests, Kenneth probably wrote it in a couple of hours. It is totally not necessary, because, there are a gazillion datetime libraries and still, this is on the front page? Also: >>> dt = maya.now() >>> dt.datetime() datetime.datetime(2016, 12, 18, 19, 24, 50, 212663, tzinfo=<UTC>) >>> dt.datetime('Europe/Budapest') datetime.datetime(2016, 12, 18, 20, 24, 50, 212663, tzinfo=<UTC>) I would not use it...
- deleted 10y ago
  [deleted]
- deleted 10y ago
  [deleted]
- tedmiston 10y ago
  Just wanted to remind you that the entire Node community depends on one 11-line module. Lines of code is still a pretty meaningless metric.
  - jasoncchild 10y ago
    Idk, pointing to issues like deep dependency within the Node community isn't the best argument. It's healthy to be skeptical of dependencies imo, especially in light of issues like the infamous left pad problem.
    - tedmiston 10y ago
      My comment wasn't about deep dependencies; it was about parent suggesting 200 lines is too few to be a legitimate library.
      - deleted 10y ago
        [deleted]
      - jasoncchild 10y ago
        Ah, I see. Noted!
foxhop 10y ago
My first commit to ago.py was 'Fri Jun 29 19:25:55 2012'. >>> import ago >>> import dateutil >>> ago.human(dateutil.parser.parse('Fri Jun 29 19:25:55 2012')) '4 years, 172 days ago' The current implementation is 66 lines of code including docstrings: * https://bitbucket.org/russellballestrini/ago/src/tip/ago.py https://bitbucket.org/russellballestrini/ago/src/tip/ago.py * https://pypi.python.org/pypi/ago https://pypi.python.org/pypi/ago
smilekzs 10y ago
Slightly off topic: > Maya never panics, and always carrys a towel. Nice reference!
krick 10y ago
Uh, Arrow? I'm not a big fan of it, really, as I've seen it to break silently on some edge cases API is not totally awesome, but this one doesn't seem to be any better. Still not sure if I should use this or that or write one of my own.
hclivess 10y ago
why

350 points by kenneth_reitz 10y ago

I have found that no matter what language/platform I use, the one thing that is always supported is UNIX timestamp. That makes date+time operations much easier: 1) Whenever dealing with users, use local tz. 2) Always save and manipulate in utc.

masklinn 10y ago
Except (2) doesn't necessarily work when events are local and you factor in timezone variations (both DST and actual TZ changes). There are classes of events where you're much better off with zoned local dates e.g. local meetings. Example: in early 2011, Samoa announced that on December 29th at midnight local they would switch their timezone offset from -11 to +13. Before that announcement (or at least before your timezone database has been updated), store as UTC a meeting in local Samoa time, and the user will miss their meeting by a day. Of course storing a meeting on December 30th local would also have been fraught as there is no December 30th 2011 in Pacific/Apia but that is a common occurrence historically due to the unsynchronised julian/gregorian switches e.g. none of the dates between February 16th and February 28th 1923 (included) exist in Greece, and the US doesn't have a 9/11 in 1752.
- appleflaxen 10y ago
  I think the recommendation still holds. You want a meeting at 1100 local time. Meeting is stored for that time in UTC, on the correct date (before Samoa changed, we all understood what date you meant) Samoa changes the rules. The calendar doesn't change. This stuff is mind-bending, so I could be missing something, but a more detailed walk through your mental debugger might clarify.
  - achamayou 10y ago
    Classic counter example: storing business hours. If the store/exchange opens at 8:00AM every weekday in Somethingania/Foocity, you want to store 8:00 and the name/reference of the timezone. Because the store/exchange will still open at 8:00AM local time, even across DST, government-mandated timezone changes etc (largely, I'm sure there are exceptions but for stock exchanges there definitely aren't). School times, times for religious services, lots of times are relative to local time, which itself is subject to change against UTC (sometimes predictable, sometimes not, the government gets to update the timezone and the DST scheme arbitrarily). So only ever storing UTC and using local time purely for display isn't a sustainable one-size-fits-all option.
    - deleted 10y ago
      [deleted]
  - masklinn 10y ago
    > You want a meeting at 1100 local time. On a specific day. > The calendar doesn't change. Of course it does, you've stored a UTC datetime, the timezone database is updated, the UTC datetime now maps to the wrong time (and because Samoa changed its offset by 24h it actually maps to the wrong day entirely every single time). > This stuff is mind-bending, so I could be missing something, but a more detailed walk through your mental debugger might clarify. * while Pacific/Apia is UTC-11, you create a meeting for January 5th at 11 local * this is stored as 2012-01-05T22:00:00+0000 * tzinfo gets updated with Pacific/Apia at UTC+13 * your calendar now tells you your meeting is on January 6th at 11 local * you're a day late
    - mjevans 10y ago
      You're both correct. The scheduling library should be more upfront about /what/ is being agreed to and force the user to pick. P) The event is at a precise internationally recognized moment (better for co-ordination globally). R) The event is in local time (like a lunch date) and expected to remain colloquially fixed. In the case of the first you store a precise UTC timestamp and drop the timezone (it's UTC). In the case of the second you store /how/ to pick a UTC timestamp based on a time in a timezone.
  - tedunangst 10y ago
    Samoa did change the calendar. The local <-> utc mapping changed. In order to update the stored database, you need to convert all times to local with the old mapping, then convert to utc again with the new mapping.
- vacri 10y ago
  Using one of the most extreme edge cases is not a good argument. How often is Samoa (or another small dateline-neighbour country) going to 'jump sides'?
  - masklinn 10y ago
    > Using one of the most extreme edge cases is not a good argument. 1. How is it not a good argument? It's an actual historical fact which is not yet 5 years old rather than some intellectual exercise, you can hardly go better than "this stuff happened not 5 years ago". 2. The only "extreme" part here is the magnitude of the drift, it's a clear example and demonstration of the issue. > How often is Samoa going to 'jump sides'? How is that relevant to the problem specifically and clearly existing? Does it matter if your calendar is 24h off or 6h off? Your system is FUBAR either way. > or another small dateline-neighbour country The dateline isn't even relevant, are you somehow trying to win a prize in missing the point?
    - vacri 10y ago
      > It's an actual historical fact It's actual historic fact that the gregorian calendar we use skipped 11 days, but I don't see your suggestion solving that. Again, it was "just magnitude". > Does it matter if your calendar is 24h off or 6h off? Your system is FUBAR either way. Your theoretical appointment-haver might not blink at an appointment at a normal time + 1 day in the future, but might blink when their appointments start in the early afternoon and continue until midnight. The fact is that if you skip days, then it doesn't matter if you use local or UTC time. > are you somehow trying to win a prize in missing the point? I'm not the cretin who thinks that timekeeping is a binary good/bad state. Some applications have troubles when they're milliseconds out. Others have quite a bit of lag time that they can handle. A daily to-do list just needs to appear on the right day, for example; DST means little to it.
  - toyg 10y ago
    Timezones change all the time. There are almost 200 countries in the world, some so large that even their sub-states can and do change timezone rules on their own (US, Russia, India, China...). This is why Linux distributions have a dedicated package listing timezone info that is updated almost weekly, and Windows has similar arrangements. Any solid datetime implementation better consider that time is not, in fact, immutable - even UTC can and will shift here and there.
paulddraper 10y ago
Sure, if you limit yourself to timestamps. But dates/times have much more notions than an instant. Like, that I have a recurring event every weekday at 9am local time. Or that the weekend is two days (hello, DST). And even if you only need timestamps, "Whenever dealing with users, use local tz" isn't exactly a magic wand either. Also, "unix timestamp" isn't exactly one data type. Sometimes you need greater precision than one second.
PeterisP 10y ago
"Always manipulate in UTC" fails for datetime distance computation. Two very common examples: For a recurring scheduled event, "Same time tomorrow" may require adding 24, 23 or 25 hours to the UTC time, depending on the TZ. The question "how many full days have elapsed between two timestamps?" may need to return 1 for 23.5 hour distance or 0 for 24.5 hour distance depending on the TZ. I'd be so happy if we all could agree that a day always has 24 hours, but that doesn't seem likely any time soon.

charlex815 10y ago

I suppose this beats breaking out time delta, but I think it'll be hard for me to see an actual use in my projects that I couldn't accomplish with maybe just a couple extra lines

kenneth_reitz 10y ago
you'd be surprised, the amount of headache-inducing code that timezones can cause, especially when dealing with servers with different locales than your development machine when doing timezone algebra. This API avoids that problem entirely.
- SFJulie 10y ago
  It does not: timezone (TZ) definitions are politicial, inaccurate, and stupid and non versioned. The problem with TZ is TZ in its core definition. 1) they always change: if you have not updated your TZ since 3 months they are probably inaccurate 2) your TZ definition maybe accurate but they may not have been applied for real in the concerned zone 3) you can have different local time for the same longitude 4) you can have different days on the same longitude ... 5) TZ are not versioned, if the TZ changed between 2 records you may have made, you have inaccurate intervals stored. We do NOT have an API to take TZ change over time in consideration. 6) CEST/DST is breaking the axiom that time is a growing monotonic function References 8.5.3 TZ in postgresql man https://www.postgresql.org/docs/9.2/static/datatype-datetime.html https://www.postgresql.org/docs/9.2/static/datatype-datetime... Computerphile what's wrong with timezones https://www.youtube.com/watch?v=-5wpm-gesOY https://www.youtube.com/watch?v=-5wpm-gesOY
  - DasIch 10y ago
    The TZ database has versions and you could at least in theory expose that at a higher level in libraries.
    - nicpottier 10y ago
      Pretty sure pytz will hand you the appropriate timezone for a given date. There aren't version numbers after all but it can do the right thing based on when you say it is. There is some weird behaviour due to this when trying to get a timezone without an associated date as it doesn't default to now. Then again you probably have all sorts of DST bugs if you have places in your code that do that.

bndr 10y ago

How does it differ from Arrow which was made several years ago? [1] [1] http://crsmithdev.com/arrow/ http://crsmithdev.com/arrow/

smegel 10y ago
That would be nice to know. Arrow still seems active, and has been consistently so over the last 3 years. I have a bit of doubt about a time library that defaults to UTC but uses human phrases like "tomorrow". Who thinks about UTC tomorrow?
- kenneth_reitz 10y ago
  that's why there's `maya.when('yesterday', timezone='US/Eastern')` :)
  - Drdrdrq 10y ago
    That's great, but why this then: >>> tomorrow = maya.when('tomorrow') <MayaDT epoch=1481919067.23> >>> tomorrow.slang_date() 'tomorrow' >>> tomorrow.slang_time() '23 hours from now' Huh? Given that we never specified timezone, I would expect 24h (unless DST change happens). This is actually my biggest gripe with date+time libs. Imho API should always be explicit in what its default TZ is. EDIT: still, appreciate what you are doing. Requests rule, and there are many other areas (including datetime) that need libs with better APIs. Thumbs up!
    - StavrosK 10y ago
      I'm pretty sure that it's 23 because some time passed between keystrokes, so the delta was 23:59:50, and he only keeps the most significant digit. I had to do this exact thing yesterday for timetaco.com.
      - prashnts 10y ago
        Shouldn't it be immutable? I'd expect this to be a default behaviour.

SFJulie 10y ago

It is heartbreaking to see such a contributor like Kenneth Reitz in conferences. I really enjoy using Kenneth Reitz's creation : I would like to see him smile. He is probably in the top 10% of the developers, he does not profit of his talent as much as an idiot that just code shit. When I sample over the vast majority of free software coder I know (100s) over the non free one (~same) and I compare the wealth/recognition I can't help but notice that for some of the 1% top FOSS devs you do have a breakthrough, but then, blind obedience seems to give more results in terms of social and wealth status. Something is rotten in the state of coding. My advice: don't make free software enjoy life, and close your ears to the voice of the voice of the crowd open them to the one you care about. And if you really want to make FOSS code, do it like others do knitting, pastry, moonshining, brewery. Keep the fun in your life.

kenneth_reitz 10y ago
Do I look unhappy at conferences or something?
- deleted 10y ago
  [deleted]
- cthalupa 10y ago
  I mean, they can get pretty smelly.
- SFJulie 10y ago
  At pycon montréal you were looking a tad tired. Maybe I mistook jetlag for something else?
  - orf 10y ago
    Something else being a void within his soul, representative of 'something rotten at the heart of programming'? That's quite a leap to make from seeing someone tired.
- jMyles 10y ago

djoser 10y ago

I understand the link with the maya believes but for a "digital" feature, this sound a lot like an Autodesk product... That said, nice API!

krautsourced 10y ago

I wonder if the naming isn't a bit unfortunate, seeing that Maya is one of the major 3d packages out there and googling for Maya and Python will almost always lead there (also, not sure whether Autodesk might object...)

duncanawoods 10y ago
Yep. Picking cute short brand names for libraries as though they were startups clutters up the namespace of programmer's conversational language causing ambiguity and obfuscation. It would make life easier if we adopted an informal naming convention for small libraries e.g. <lang><brand><purpose>. Lang is useful when libraries get ported so you get e.g. PyMayaTime, GoMayaTime etc. Still sounds cute but you know what its for when you pick up someone else's code and its easy to google.
- Senji 10y ago
  Picking names which collide with stuff seems to be the standard these days. Don't get me started on Google(Chrome, Go)
agumonkey 10y ago
Years ago Maya had Python, now Python has Maya. The circle is complete. ps: note that the Maya in Alias Wavefront (now Autodesk but nope) Maya is not that Maya, it's Sanskrit for illusion, which is what Maya was all about considering what it did to your dollars.
gohrt 10y ago
MayaTime would be a responsible name for this project. Brand Name + Hint about is functionality. Like "JodaTime" and "Golang"
dagmx 10y ago
Also unfortunate because the python api in Autodesk Maya is in the maya namespace. So this library would need to be installed as a separate name to be usable there.

RubyPinch 10y ago

<MayaDT epoch=1481850660.9> not the most human readable is it? I guess it would help with like, "X happened before Y" situations, but I don't think I'd trust my eyes for that! I think I would of preferred the "this happened in X month" case instead, I find it easier to trust my eyes for that, instead of trying to look for differing digits! - - - UTC default is a godsend though

jvdh 10y ago
Fortunately it is really easy to print it in other formats. Which is certainly not the case for the standard datetime library.

meltingwax 10y ago

This is designed for programmers of user facing applications. For science and engineering, it does not address the problems, eg lack of leap seconds.

cgvgffyv 10y ago
You need to wait for another rewrite with a cute name for these.
masklinn 10y ago
> For science and engineering, it does not address the problems, eg lack of leap seconds. What do you mean by "lack of leap seconds"? A non-unix-timestamp-based time library?
- meltingwax 10y ago
  When the mapping between a time_t and a "human readable format" (hours, minutes, seconds, etc) takes into account extra leap seconds. You could use time_t underneath, if that's what you really wanted to do. As my co-worker once put it, "time is a four letter word." At the very least, there is a need for showing the occasional 61st second of a minute, and adding a time and timedelta where leap seconds is taken into account.
  - masklinn 10y ago
    > When the mapping between a time_t and a "human readable format" (hours, minutes, seconds, etc) takes into account extra leap seconds. I still don't understand what you mean. > You could use time_t underneath, if that's what you really wanted to do. Well you've got unix timestamp (UTC) which "skips" leap seconds but as a result is very easy to map to "human time", or you've got TAI[0] which includes leap seconds but doesn't allow dates in the future since you don't know where and when new leap seconds will be added long in advance, and now you need regular updates/permanent connectivity so you can remap TAI onto human time. And you need an NTP replacement, though I guess having a GPS unit in everything would do. [0] and GPS time which is just TAI + 19s

sametmax 10y ago

I wish Kenneth would have contributed to an existed project for once. Arrow and pendulum (my current favorite) have a very decent API. The later one is especially well tested for the numerous corner cases of date handling, which I doubt Kenneth got right on first try. For request, a full rewrite made sense because urllib sucked so much and we had no good alternatives. But for date time, alternative exists and they are good. Do not split the open source effort, join forces!

KODeKarnage 10y ago
Who wishes that Requests didnt exist? That guy!!! ^^^^^^^
- ergo14 10y ago
  You comment makes no sense...
  - KODeKarnage 10y ago
    He wishes the guy was the sort of guy who contributed to other projects rather than doing his own thing. But if he WAS that sort of guy we wouldn't have Requests. I thought you guys were supposed to be smart.
    - stable-point 10y ago
      No, he doesn't: > For request, a full rewrite made sense because urllib sucked so much and we had no good alternatives.
      - KODeKarnage 10y ago
        > I wish Kenneth would have contributed to an existed project for once. Yes, he does. He wishes Kenneth was not the same as the existing Kenneth. He emphasizes this with the "for once". Kenneth went his own way with Requests, and everyone is better off for it. You do not want a guy with that track record abandoning his work patterns. I say, let the man be. Whining about "fracturing" in open source ignores those important things which makes progress possible; conflict, competition, and triumph. I see the original comment as being little more than a "yeah, it's nice, but you know, we already formed a committee and working group to do this thing".

dom0 10y ago

> Datetimes are a headache to deal with in Python, especially when dealing with timezones, especially when dealing with different machines with different locales. Anything with date/time calculations is always a pain, probably doesn't really have much to do with the library/language itself, but that the abstraction level that's used (and typically used in other libraries) means that the complexities of calendar and time systems are sprinkled all over application code. I do have to notice here that always using UTC is not always the right thing to do. For example, evaluating rrules in UTC is rather error-prone (DST).

mjevans 10y ago
If you're mixing UTC and DST you are doing something wrong.
- deathanatos 10y ago
  I think what dom0 means is a situation like the following. Say you want to schedule, on a calendar, an event that happens daily at 2PM in your local timezone, say America/New_York. If you record that even as happening at 2200 UTC each day, when DST rolls around, it'll shift to 1PM or 3PM in the local timezone. This is not what the user wants. Here, despite storing everything in UTC and attempting to apply DST as late as possible as if it were a "display issue" — normally the right thing — here results in the wrong outcome.
  - mjevans 10y ago
    In your example what I said is still true. The user didn't want a precise coordinated time (DST is not a factor for storage/recall); the user wanted a fuzzy reference to local time (DST is a factor for evaluation).

BoppreH 10y ago

Awesome! Timekeeping is hard and I'm glad we now have one more tool do deal with it. One thing that is bothering me is that when you ask for `maya.when('tomorrow')`, or give only a date, you get back a timestamp with millisecond precision, representing 00:00 of that day. I understand this simplifies the implementation, but shouldn't `tomorrow` be a range, from 00:00 to 23:59? Treating imprecise dates as ranges would allow for stuff like # Did this event happen "yesterday" according to the US/Eastern time zone? timestamp in maya.when('yesterday', to_timezone='US/Eastern') # Get me all events that happened last Friday (UTC) [event for event in events if event.time in maya.when('2016-12-16')] Maybe I'm being naive, and there's a reason why this won't work, but this seems the way most humans deal with time. PS: it failed to install on Windows, so I opened an issue at https://github.com/kennethreitz/maya/issues/10 https://github.com/kennethreitz/maya/issues/10

tokenizerrr 10y ago
How would you serialize that range to, for example, ISO8601?
- cheapsteak 10y ago
  From wiki: ISO8601 has four ways to express a time interval: 1. Start and end, such as "2007-03-01T13:00:00Z/2008-05-11T15:30:00Z". 2. Start and duration, such as "2007-03-01T13:00:00Z/P1Y2M10DT2H30M" 3. Duration and end, such as "P1Y2M10DT2H30M/2008-05-11T15:30:00Z" 4. Duration only, such as "P1Y2M10DT2H30M", with additional context information http://en.wikipedia.org/wiki/ISO_8601#Time_intervals http://en.wikipedia.org/wiki/ISO_8601#Time_intervals
- BoppreH 10y ago
  Excellent question. ISO 8601 has features to encode time intervals ( https://en.wikipedia.org/wiki/ISO_8601#Time_intervals https://en.wikipedia.org/wiki/ISO_8601#Time_intervals ). If that's too verbose, and we only need to represent integer intervals, one could store a truncated representation, where the omitted parts mean "any" (like * in cron). For example, `2016-11` could mean the entire month of November. But those are just ideas for the representation. The important feature is that not all objects are precise points in time, but may be a block/interval/range.

sebastibe 10y ago

Hopefully we will have a "API for Humans" for each of the standard library modules.

etanol 10y ago

Not to undermine its merit, but most of the dirty work and heavy lifting is done by its dependencies: https://github.com/kennethreitz/maya/blob/d57a78c6bc6b5295f7d7207be0ee7cf0d62f063f/setup.py#L14-L20 https://github.com/kennethreitz/maya/blob/d57a78c6bc6b5295f7... And i18n support in humanize is a bit lacking, as it only translates to French, Korean and Russian. Given that most of the translations needed to render human dates can be found in the CLDR database, maintaining their own looks like a bit of a wasted effort. Reference: http://cldr.unicode.org/ http://cldr.unicode.org/

masklinn 10y ago
> Given that most of the translations needed to render human dates can be found in the CLDR database, maintaining their own looks like a bit of a wasted effort. You still need to parse the CLDR (the formats are pretty wonky) and provide APIs for its functions. Of course that's what Babel (http://babel.pocoo.org http://babel.pocoo.org) does.
- thrownblown 10y ago
  and here I thought babel was for es2015

ben_jones 10y ago

People are asking why this is needed compared to Arrow and a few other libraries. Personally I was never happy using the python time libraries, it always felt like my use cases were slightly different then the library, and I always found myself getting frustrated over little things here and there. I think there are a LOT of use cases for time in python applications and there is plenty of room for small libraries to satisfy these conditions, versus one monolithic time library that attempts to solve all.

fnord123 10y ago

Python datetimes used with timezones, even UTC, are ridiculously slow and bloated. It's puzzling why this doesn't wrap np.datetime64 instead. Or wrap boost.datetime. There are many good options to claw some performance back so it's really head-scratch inducing that someone would recognize that stdlib datetime is a dog and then wrap it instead of scrapping it.

aangjie 10y ago
> It's puzzling why this doesn't wrap np.datetime64 instead. Perhaps to avoid the numpy dependency. I've found numpy a bit heavy and wouldn't want to depend on it simply for the sake of some datetime syntax. May be boost.datetime or even numpy's implementantion on datetime, but that sounds like a lot of maintenance.
- guitarbill 10y ago
  Agree, 100%. E.g. numpy in a virtual environment can be a real pain. Best case, it takes a bit longer to install. Worst case, it's on a system that's missing some OS specific dependencies. "Works for me" is not okay for stuff on PyPI.
  - aangjie 10y ago
    Hmm.. that's making me rethink that 'datascienceutils' project I put up on pypi.. May be work on getting it to stable version before pushing to pypi..
deleted 10y ago
[deleted]
daenney 10y ago
> It's puzzling why this doesn't wrap np.datetime64 instead. Or wrap boost.datetime. I don't find it that puzzling. It's meant to help deal with timezones, efficiency isn't necessarily part of that equation. And if I were to write a simple CLI or web app I would avoid anything that would pull in a dependency like numpy or require a C-library plus its bindings. If you need to manipulate a huge amount of date times and this is something your app is constantly doing then sure, optimise for that.

joaoqalves 10y ago

Yet another library to handle dates in Python. Oh, boy... https://xkcd.com/927/ https://xkcd.com/927/

partycoder 10y ago

[offtopic] I do not like the "for humans" thing. What could it be for instead? for kangaroos or giraffes?

frou_dh 10y ago
Popo: An Artisinal Handcrafted Curated Simple-yet-Powerful Library (for Waterfowl). By L'Wren Scott.

jMyles 10y ago

What's up buddy? My only real question: > rand_day = maya.when('2011-02-07', timezone='US/Eastern') This returning an object representing a DateTime on the 6th (in UTC time) strikes me as perhaps "not for humans." If I just see that line casually, I think I expect to get a Date and for it to be the 7th. It looks like, in order to get this (arguably expected) object, I need to take the resulting MayaDT epoch and run its `datetime` method, passing naive=True? And I also see that _tz can only ever be pytz.timezone('UTC') - is this the result of some belief that timezones are illusions or something? :-) For a while, I have kinda thought that timezones foment a confused mental model of time and teamwork. I prefer to think in terms of the astronomy - it's not actually a different time anywhere else, it's just that the sun is at a different position relative to the rest of the earth (and thus, ones faraway teammates and loved ones). Anyway, thanks for yet another set of interesting ideas. Hope you are well.

guitarbill 10y ago
It's worth thinking of it the same as strings. Strings can have several encodings, e.g. ASCII, UTF-8 and a hundred others. These are external representation. You don't want to parse the string every time you have to deal with it, so you parse it and store it using (ideally) a single internal representation (e.g. UCS-2, UCS-4). Then, when you need to pass the string to something else, you encode again. Same with dates. Instead of trying to deal with and store all possible representations (format, timezones, etc), you convert all dates to a single representation (hence UTC). When you need to output the date, encode it into whatever format and timezone you need. It's much cleaner this way, because there's less chance you'll mix up representations. So the "for humans" part is more "for developers, which are also humans but like to pretend they aren't and never make mistakes".

aibottle 10y ago

Great job, Mr. Reitz. I think I will use this in every project from now on. Thanks!

japhyr 10y ago

I'm competent with strptime(), but this looks really nice: # Automatically parse datetime strings and generate naive datetimes. >>> scraped = '2016-12-16 18:23:45.423992+00:00' >>> maya.parse(scraped).datetime(to_timezone='US/Eastern', naive=True) datetime.datetime(2016, 12, 16, 13, 23, 45, 423992) I'm happy not to have to write formatting arguments to strptime() anymore. Do the other datetime libraries have similar parsing functions?

detaro 10y ago
Maya actually uses the dateutil.parser.parse function from dateutils under the hood, which as far as I remember is the most comprehensive such function among the typical python datetime libs.
- japhyr 10y ago
  Wow, I've been using datetime for years, without ever noticing the dateutil library. I'll use this from now on.

Goopplesoft 10y ago

Kenneth Reitz is a testament to how important good interfaces are in the developer community. There are literally 0 interesting things in the code (all dependency driven, https://git.io/v15i3 https://git.io/v15i3). It does have a nice interface and because of this it will probably become one of the more popular python datetime libs.

vegabook 10y ago
Arguably bringing multiple dependencies under one roof, with a nice, human-readable interface, is exactly what the value is here. This SO answer from Pandas lead Wes Mckinney says it better than anything I could comment. Python datetime/timestamp management: "Welcome to Hell" http://stackoverflow.com/questions/13703720/converting-between-datetime-timestamp-and-datetime64 http://stackoverflow.com/questions/13703720/converting-betwe... I run batch and real time financial feeds into Python from multiple sources, inevitably from different programming language paradigms, and with different time/date/timezone conventions, and then I allow user interaction with it in their own lingo for dates and times, and my experience is that it is indeed hell consolidating all this into a common tongue. The "slang time" idea suggest to me that this library understands the need for flexibility and malleability in this very disjointed and often frustrating part of the Python ecosystem.
- dom0 10y ago
  I think there are two separate issues here. One thing is that some libraries bring their own not-really-different-just-different-enough-to-break-things date/time types to the table. That can be worked out over time, not that big of a problem, just inconvenient. The other is that I've grown to think that the abstraction commonly used, seeing date/time as something like (year, month, day, hour, minute, second, subsecond, timezone) is in itself not really suitable, because the derived interfaces (like accessing and modifying days, months etc. separately) mean that most code working with these will inevitably be bug-ridden. The root cause for this is imho that date and time are extremely complicated and it's usually unclear what the correct behaviour would be in any of the many, many edge cases. Some examples: 1.) One week after `this` date. Does this mean, add seven days? What about leap days? Should the result be the same weekday, just in the next week instead? 2.) Start and end time of a process should be recorded. The process starts and finished. All good. The duration of the process is -1 hour and 5 minutes. Should instead a start timestamp be recorded and a monotonic duration? Or perhaps we should rather note the begin of a monotonic period and record an end timestamp, deriving the start timestamp? Which is correct? 3.) One ^W two words: recurrence rules 4.) DST 5.) Combining 3.) and 4.)! dom's rule of thumb: if code does addition or substraction with some measure of time, it's probably wrong.

d0m 10y ago

Yeah.. datetime is one of the rare part of the Python API that I really hate using. It's just badly designed, e.g. simple things are hard and confusing.

jchassoul 10y ago

I don't understand the author's effort on solving this "problem"... I wonder if he considered at some point make a contribution to stable open-source existing libraries like arrow that even claim inspiration on one of his projects for the API. What's not for humans on the arrow API? what are the arguments for basically start yet another time library from scratch?, why is making a contribution not an option? probably the author of arrow will love your contributions and will love to hear the arguments on the changes you propose.

scrollaway 10y ago
As rich as it is to tell Kenneth to "consider contributing to existing open source libraries", I have to agree: what's so exceptional about this library when Arrow is already a thing? http://crsmithdev.com/arrow/ http://crsmithdev.com/arrow/
detaro 10y ago
> what are the arguments for basically start yet another time library from scratch? It's not exactly a time library from scratch, if you look at the amount of existing libraries it imports and the small code size (which is a good thing!). Why should he not offer a differently flavored API on top of existing parts? My impression is that most of these libs start with a strong idea what the API should be like, so large changes there are unlikely to happen.

niftich 10y ago

The link to its github was posted on HN when it was brand new, and I left a comment [6], which I'm reproducing here -- since most of my points still apply. (...) take my very early comments with a grain of salt -- they refer to the progress as of this commit [1]. I love Requests -- its API design is fantastic, and manages to distill down most of a complex problem domain to a clean, dare-I-say, elegant API. So I can eagerly anticipate this design applied to datetimes. But the progress being shown so far is definitely not it. >>> tomorrow = maya.when('tomorrow') <MayaDT epoch=1481919067.23> Why is "tomorrow" a precise-to-centisecond, infinitesimally small point on a giant cosmic timeline? I'm reasonably sure it's an abstract concept that describes the calendar day that starts after the current calendar day ends. At least, Pendulum normalizes tomorrow() and its ilk to represent midnight on the given day [2], while Delorean's natural language methods [3] like next_day() advance the day while leaving the time-of-day unchanged, but the method name makes this fairly clear. Even Arrow, which is heavily inspired by Moment.js to the point of conflating every single datetime idea into the same class, opts for mutators that are still more clear [4]. > Timezones fit in here somewhere... Yeah, this needs more work. Java 8 / Threeten, and its predecessor Joda-Time took the approach of clearly modeling each and every concept that humans actually use to describe time; even if you take issue with their API, the designers have clearly done their homework, and their data model is solid. Formats like TOML wrestled with datetimes and realized [5] that datetimes aren't just woefully underspecified in most other specs and APIs, but that they're frequently mis-modeled, so they adopted large portions of Threeten's data model. Cases like this should merit strong consideration from anyone trying to propose new datetime APIs today. [1] https://github.com/kennethreitz/maya/commit/ecd0166ba215c1a5.. https://github.com/kennethreitz/maya/commit/ecd0166ba215c1a5.... [2] https://pendulum.eustace.io/docs/#instantiation https://pendulum.eustace.io/docs/#instantiation [3] http://delorean.readthedocs.io/en/latest/quickstart.html#nat.. http://delorean.readthedocs.io/en/latest/quickstart.html#nat.... [4] http://crsmithdev.com/arrow/#replace-shift http://crsmithdev.com/arrow/#replace-shift [5] https://news.ycombinator.com/item?id=12364805 https://news.ycombinator.com/item?id=12364805 [6] https://news.ycombinator.com/item?id=13190314#13190657 https://news.ycombinator.com/item?id=13190314#13190657

batbomb 10y ago

Kenneth: It would be really cool if you could put in support for TAI and MJD.

aivosha 10y ago

i dont trust any piece of software that has all these made-up, unrelated, non-reflecting names. I mean come on, arrow, maya ? the main challenge about software engineering is correct naming. If you fail there you pretty much fail in the rest. Same goes for actually builtin name, "datetime". There is no such thing in real life as datetime. There is date and there is time. They are very separate notions and the root of the problem IMO is in trying to pile them together.

ianamartin 10y ago

I think its fascinating that the community has no consensus about a datetime library. A lot of Python is really solved. We don't argue about using requests (a not-coincidental example). If you're using Python, and you need to deal with http, you use requests. Everyone knows this. There are basically 3 platforms for web frameworks. Flask, Pyramid, and Django. Maybe we're a little more dissolute than C# or Ruby folks, but that's pretty impressive considering how much we Python people like to roll our own. The fact that there is real disagreement about this among ourselves about this particular issue says to me that this is more about the difficulty of the problem than it is anything else.

NelsonMinar 10y ago
Related: the datetime library in pretty much every language is terrible. The only one I've ever used that I remember not actively hating was MomentJS. Python's history of time types was terrible, as well. Particularly the era when we just used tuples of length 8 or 9 that weren't timezone aware.
- bjacobel 10y ago
  Funny that JavaScript would find community alignment on something (anything!) where Python can't. I guess that's a testament to the work of the Moment creators.
  - niftich 10y ago
    It is. Datetime is an unpleasant problem domain, and Tim was active both on SO [1] and Github, arguably so much so that it was to the detriment of his health [2]. In the case of Moment, a little promotion went a very long way, and the community was eager to take what's given so they wouldn't have to think too much about datetime, and focus on debating how to arrange and design their applications instead. Moment's design is a testament to "let's not think too hard about datetime", and I mean that both as a compliment and a critique. [1] http://stackoverflow.com/users/272034/timrwood?tab=summary http://stackoverflow.com/users/272034/timrwood?tab=summary [2] https://news.ycombinator.com/item?id=12175369 https://news.ycombinator.com/item?id=12175369

diyseguy 10y ago

But does it get DST correct. I haven't found any python time libraries that do.

sdispater 10y ago
You can take a look at pendulum: https://github.com/sdispater/pendulum https://github.com/sdispater/pendulum A lot of effort has been put into getting DST right so I hope it will be what you are looking for. Disclaimer: I am the author of Pendulum :-)
robochat 10y ago
Interestingly, Python 3.6 has just tried to fix the DST problem in the datetime library itself. See PEP 495 https://www.python.org/dev/peps/pep-0495/ https://www.python.org/dev/peps/pep-0495/

ak217 10y ago

I like Kenneth's work, but "I wrote a new datetime library" is a cliche now. We have datetime, dateutil, pytz, babel, arrow, pendulum, delorean, a bunch of lesser known stuff, and now this. I have yet to see the need for anything but the first four.

toyg 10y ago
Seems to be a case of "Python Web Framework"-itis: it's relatively simple to hash up something from scratch that covers one's particular use-case (in this case, it seems, a slightly smarter parser), but it's actually hard to properly cover all use-cases. This is what happened with Python web frameworks, a scene which was heavily fragmented before Django and Flask basically solidified the two main communities of users ("I need everything and the kitchen sink" / "I need the bare minimum to get going"). The same happened with urllib2/urllib3/httplib/ etc before Requests appeared. Nobody seems to have pulled this trick for datetime libraries yet, so here's a new contestant.

Animats 10y ago

Bikeshedding. The Python "datetime" module does almost all of this. (Although I did try to get ISO8660 parsing put in, after finding eight libraries for it, all of which were broken in some way.)

Walkman 10y ago

This is a joke. The whole "library" is 200 lines, has 9 tests, Kenneth probably wrote it in a couple of hours. It is totally not necessary, because, there are a gazillion datetime libraries and still, this is on the front page? Also: >>> dt = maya.now() >>> dt.datetime() datetime.datetime(2016, 12, 18, 19, 24, 50, 212663, tzinfo=<UTC>) >>> dt.datetime('Europe/Budapest') datetime.datetime(2016, 12, 18, 20, 24, 50, 212663, tzinfo=<UTC>) I would not use it...

deleted 10y ago
[deleted]
deleted 10y ago
[deleted]
tedmiston 10y ago
Just wanted to remind you that the entire Node community depends on one 11-line module. Lines of code is still a pretty meaningless metric.
- jasoncchild 10y ago
  Idk, pointing to issues like deep dependency within the Node community isn't the best argument. It's healthy to be skeptical of dependencies imo, especially in light of issues like the infamous left pad problem.
  - tedmiston 10y ago
    My comment wasn't about deep dependencies; it was about parent suggesting 200 lines is too few to be a legitimate library.
    - deleted 10y ago
      [deleted]
    - jasoncchild 10y ago
      Ah, I see. Noted!

foxhop 10y ago

My first commit to ago.py was 'Fri Jun 29 19:25:55 2012'. >>> import ago >>> import dateutil >>> ago.human(dateutil.parser.parse('Fri Jun 29 19:25:55 2012')) '4 years, 172 days ago' The current implementation is 66 lines of code including docstrings: * https://bitbucket.org/russellballestrini/ago/src/tip/ago.py https://bitbucket.org/russellballestrini/ago/src/tip/ago.py * https://pypi.python.org/pypi/ago https://pypi.python.org/pypi/ago

smilekzs 10y ago

Slightly off topic: > Maya never panics, and always carrys a towel. Nice reference!

krick 10y ago

Uh, Arrow? I'm not a big fan of it, really, as I've seen it to break silently on some edge cases API is not totally awesome, but this one doesn't seem to be any better. Still not sure if I should use this or that or write one of my own.

hclivess 10y ago

why