8 ms·
Arrow v1.0: After 8 years, even better dates and times for Python
- dataflow 6y agoNot to be confused with Apache Arrow's Python bindings (pyarrow)... https://arrow.apache.org/docs/python/ https://arrow.apache.org/docs/python/
- folkrav 6y agoAfter a quick search seems like this lib actually predates `pyarrow` by quite a bit.
- visarga 6y agoDoes arrow have support for multi-language date parsing like dateparser? I like dateparser but it's slow if you allow many languages.
- chupasaurus 6y ago[0]There is for month name and AM/PM, but not for day of week. [0]https://arrow.readthedocs.io/en/stable/#supported-tokens https://arrow.readthedocs.io/en/stable/#supported-tokens
- jeffrogers 6y agoWhat's the best .js equivalent?
- scott_w 6y agoI’ve usually used moment.js in the past (https://momentjs.com/ https://momentjs.com/).
- Brendinooo 6y agoEither that or https://date-fns.org/ https://date-fns.org/
- folkrav 6y agomoment.js is in maintenance mode and the library authors themselves try to push people away to other libs[1]. Seems like most people have moved to date-fns or day.js. Moment authors seem to point to Luxon, which I didn't know about. [1] https://momentjs.com/docs/#/-project-status/ https://momentjs.com/docs/#/-project-status/
- scott_w 6y agoYep, I was aware that they moved to maintenance mode but I mistakenly thought they were still good for new projects.
- fiddlerwoaroof 6y agoI’ve evaluated a bunch and there’s no clear winner: luxon is probably my favorite that I’ve used, but it’s missing some crucial locale stuff. I’ve never used it, but I think js-joda looks the best in paper: it’s based on the API for a fairly popular JVM date library which I used a lot. One issue, though, is that the “correct” date-time representation is often use-case dependent.
- LaundroMat 6y agoI like dayjs best. https://day.js.org/ https://day.js.org/
- minitoar 6y agoNice, love Arrow. We use it in Interana to build calendar-aware buckets for time series aggregations.
- boxed 6y agoI've recommended people stay far away from arrow for many years now. The basic problem is on their "features" list: > [the standard library has] too many types No. That's just not at all true. In fact I would argue it's the opposite. You can make some very bad mistakes in python with dates because they silently coerce into each other in nonsensical ways. Arrow makes this much worse. A date is not the same as the millisecond at the start of thst date!
- xapata 6y agoMind giving an example?
- dmix 6y agoI wonder if this is Arrow or working with datetimes in programming in general. These such as Arrow, like moment.js, are good for UIs. But if you require plenty of date manipulation then I’d keep it native as possible with plenty of tests. An example would be good as well. I think dates have been my worst experience working in programming across a multitude of languages.
- symlinkk 6y agoJust seems like the age old problem of Python not having types built in.
- dragonwriter 6y agoWhat does the absence of (static) types have anything to do with the issue here?
- alanfranz 6y ago> You can make some very bad mistakes in python with dates because they silently coerce into each other in nonsensical ways The Big Problem in Python's datetime library is the naive/tzaware thing. The same type is used for two very different concepts.
- xapata 6y ago
- throwaway81523 6y agoDJB approach: https://cr.yp.to/proto/utctai.html https://cr.yp.to/proto/utctai.html
- lifthrasiir 6y agoFirst, your link should be to libtai. Second, DJB never adequately addressed the complex problem of parsing and formatting human-readable dates (only providing the fixed ISO 8601 format), nor time zones (it doesn't support the local time zone, which is fine [1], but it doesn't support the specified time zone as well). It represents an incredibly naive understanding of date and time processing. [1] This needs elaboration. The local time zone is typically implemented by at least one of two ways: you read the central repository (`/etc/localtime` in Linux for example) every time local time is requested, or you listen to the time zone change event (`WM_SETTINGCHANGE` event in Windows for example). This is no different from your typical pan-process synchronization problem, which is complex and costly.
- pood 6y agoHow does it compare to pendulum?
- jsilence 6y agoHow does Arrow compare to Pendulum?
- neolog 6y agoPendulum is better
- stigz 6y agoI have only used Pendulum and skimmed Arrow's API just now. Pendulum does more. It has tools for manipulating a date. Add days, compare to other dates, etc. Arrow appears primarily focused on parsing and formatting.
- mazatta 6y agoNo, Arrow is much more than parsing/formatting. Each Arrow object has a `shift` method, which is how you can add or subtract units of time. Greater/less/equal comparisons work exactly as you'd expect them to.
- qwertox 6y agoThanks for bringing this onto my radar. I've already settled with arrow, but when I can go without it, I combine datetime with pytz, which is so much faster at applying/changing timezones than arrow. I still need to check out 3.9's built-in timezones.
- Animats 6y agoToo many modules: datetime, time, calendar, dateutil, pytz and more Yes. [1] Back in 2012, I filed a Python bug report that "datetime" has a formatter for ISO 8601 format strings, but no parser. I'd found four parsers, all with serious bugs. After six years of discussion, it was fixed.[1] [1] https://bugs.python.org/issue15873 https://bugs.python.org/issue15873
- xapata 6y agoIf four popular parsers all had serious bugs, 6 years seems not too shabby.
- mixmastamyk 6y agoNeat, I had an impact on that. I found your bug after also needing a matching parser and went on the mailing list to ask for it. Had to come back twice more to complain and get people to focus and get the ball rolling. Perfect is the enemy of the good in a nutshell. (8601 is a huge spec, and we only needed a matching parser to start with. It is now free to grow.)
- Animats 6y agoThis subject does seems to attract bikeshedding.
- mixmastamyk 6y agoIIRC it was, what about weeks? Time periods? Offsets vs zones? (Other esoteric features.) "Just have it parse .isoformat() and ship it!!"
- 1wd 6y agoIt still does not parse all ISO 8601 format strings: "ISO 8601, RFC 3339 and datetime.isoformat() are three slightly different and in some senses incompatible datetime serialization formats" https://bugs.python.org/issue35829 https://bugs.python.org/issue35829 For example it rejects typical strings produced by Javascript using Z instead of +00:00 as the timezone suffix.
- ridaj 6y agoPet peeve: people should stop saying "even better" in their announcements as if simply saying "better" implied that your previous version was crappy. Be more lucid in your self assessments, more humble in your comms and just say you improved it.
- googletron 6y agoI prefer Delorean much more. https://delorean.readthedocs.io/en/latest/quickstart.html https://delorean.readthedocs.io/en/latest/quickstart.html
- Lammy 6y agoThe library looks nice, but the name doesn't sit well with me since I don't endorse the namesake concept :p > The arrow of time, also called time's arrow, is the concept positing the "one-way direction" or "asymmetry" of time.
- alanfranz 6y ago> Timezone-aware and UTC by default This is the main problem with python's builtin datetime module: the default is a kind of "localdate" or "localdatetime", and when you add a timezone it becomes an instant. It's a bad idea to conflate such two concepts. OTOH arrow seems to think that "one size fits all". Which is bad as well :-/ I think joda time / java time design is good!
- kuschku 6y agoThe only working datetime solution is JSR310. It has separate types for • LocalTime (milliseconds after midnight) • LocalDate (julian day), • LocalDateTime (julian day and milliseconds after midnight), • Instant (nanoseconds since EPOCH), • ZonedDateTime (which is a point in time with a timezone, and exposes both the localdatetime and the instant APIs) All of these types have a reason to exist, and definitely shouldn't be mixed together (like python does). • If I program an alarm clock, I'll want to use LocalTime. Even if DST starts/ends, you still wanna get woken up at 8am. • My birthday is a LocalDate. It usually has no time information, and usually isn't depending on a timezone. An anniversary is the same. • The moment when a loggable event happened is an Instant. The event doesn't care about timezones, only the exact time it happened. • A calendar event is a ZonedDateTime: the meeting will happen at 10am CET, regardless of which timezone I'm in at that point. All these types describe different concepts, and they shouldn't be mixed together. Every language should adopt JSR310. Yes, it's relatively complicated, but it's extremely precise and accurate. And you can convert them all! •LocalDate + LocalTime becomes LocalDateTime • LocalDateTime.atZone(ZoneId or ZoneOffset) becomes ZonedDateTime • Instant.atZone(ZoneId or ZoneOffset) becomes ZonedDateTime • ZonedDateTime.toInstant() or ZonedDateTime.toLocalDateTime() exposes local datetime and instant. • LocalDate can easilybe converted to JapaneseDate, HebrewDate, ArabicDate.
- dthul 6y agoIf I'm not mistaken "chrono"[1] behaves almost the same. It calls Local[Date|Time|DateTime] Naive[Date|Time|DateTime]. ZonedDateTime is just called DateTime. It doesn't have a separate type for Instants but can convert (zoned) DateTimes to and from UNIX timestamps. [1] https://docs.rs/chrono/0.4.19/chrono/index.html https://docs.rs/chrono/0.4.19/chrono/index.html Edit: it seems like chrono's design took lessons learned from other time libraries, including JSR-310, into account, so that explains why they are so similar.
- kuschku 6y agoYeah, Chrono is also really nice, indeed. Didn’t know it was inspired by JSR310, but it makes sense now that I think about it :) The new ES Temporal API is also inspired by the same API.
- kdeldycke 6y agoArrow is the only Python library with date-aware ceiling and flooring methods. I wouldn't have been able to design and implement Scaleway’s billing pipeline in 2013 without those. More details at: https://kevin.deldycke.com/2020/10/billing-pipeline-critical-time-sensitive-system/ https://kevin.deldycke.com/2020/10/billing-pipeline-critical...
- krisfremen 6y agoThat's a nice write up, Antoine's post is insightful as well! Spice must flow!
- jwow_mat 6y agoAny thoughts on comparisons with Pandas Timestamp? It parses from ISO8601, it has appropriate guards against coercing tz naive/aware instances and, unlike datetime, it behaves like ZonedDateTime, so tz aware instances are tied to instants. I would guess that it's faster than arrow-py or Pendulum, too, since it's part of Pandas, but don't have proof.
- nojito 6y agoPandas silently converts to different dates within the same column https://github.com/pandas-dev/pandas/issues/12585 https://github.com/pandas-dev/pandas/issues/12585
- dec0dedab0de 6y agoI always forget what arrow is called when I need it, but back when it came out it was the only way I could do anything useful with time. I eventually got good enough with the standard library to do what I need to do, but it's so ugly. I have to remember arrow, i wonder if the name is a reference to the tng episode? I'm just going to try to associate Python time with Data's head, And the snake cane. I really wish improvements to python focused on things like this. Maybe an official higher level standard library for wrappers that focus on being nice to use like arrow, requests, envelopes, etc. (I know envelopes is not maintained but it was so nice)
- tzs 6y ago> Too many modules: datetime, time, calendar, dateutil, pytz and more Worse...too many modules included in the standard Python library with slightly incompatible implementations of the same functions. I spent quite a while trying to understand why strptime was not working like I thought it should, before figuring out that I was using time.strptime when I should have been using datetime.strptime. The difference that got me was that time.strptime was ignoring timezones in the format string and input string and assuming the local timezone. It was not an error to include %z in the format string and give it a time string that includes a timezone. It would simply silently be ignored and the local timezone used instead.
- BrandoElFollito 6y agoI am an amateur programmer who started with Python some 10 years ago. I had two revelations when it comes to modules: - requests - arrow Dates in Python are so fucked up that it is beyond imagination. I understand that there are philosophical reasons for that but I do not care. I need to compare dates, add dates, do sometime < now < someothertime, etc. All this without a PhD in Time Pythonlology. When I discovered arrow I fell in love and now I use it for anything. Need to output an ISO date? arrow. Need to output a timestamp? arrow. Need to compare two dates? arrow. arrow. arrow. Yes, this is a hammer for any date nail *but it works*. There are some great programs (such as Borg) that were hit by the monstruosities in pythin dates and eneded up with some half-baked semi-ISO output without timezones. Has the author used arrow by default he would not have gone wrong. The main dev is great and a bit stubborn (he closed a Very important Issue Of Mine and I had to fix it myself :/) but I will be forever admirative about his work (and the work of other contributors). I am waiting now for the next announce about requests. EDIT: I also tried pendulum and delorean but always ended up back with arrow. I now need to understand how to move from moment.js to luxon.js and my life with dates will be complete.