23 ms·
datetime.utcnow() is now deprecated
- deleted 3y ago[deleted]
- nemothekid 3y ago>I may be missing something here, but I don't really follow this logic. Isn't the logic that throwing an error on naive timestamps would break existing code?
- rtpg 3y agoIt’s pretty disappointing to see this article make claims like “web developers just use naive datetimes”. I think anyone who does anything nontrivial with datetimes should always use non-naive datetimes (unless you have a really really really good reason to) Even when I started doing Python web dev 10 years ago this knowledge was bestowed upon me and has saved me many headaches (and helped me find bugs in third party code). The fact this article claims otherwise makes me doubtful of other advice given. Unfortunately, Pythons date/datetime APIs make it way too easy to do the wrong thing on this topic.
- ape4 3y agoStackoverflow on this https://stackoverflow.com/questions/15940280/how-to-get-utc-time-in-python https://stackoverflow.com/questions/15940280/how-to-get-utc-...
- tzs 3y agoThe timezone thing in Python that bit me was when I needed to parse a time string. Googling told me Python had strptime and it would do what I wanted. So I found it in the Python docs and used it. What I failed to notice was that Python has two strptime functions, one in time and one in datetime, and they are not the same. I was losing timezone information that was in my time strings, and when I wrote a test program to just test timezone parsing I happened to pick the other strptime and everything worked fine, which left me quite confused.
- NelsonMinar 3y agoPython could be better but really, does any language handle date and time types well? After 30+ years of this I mostly just use seconds since epoch everywhere like some sort of caveman banging rocks together. But at least it works clearly. Last I looked Pendulum was the best choice for a fancy but humane Python library for dates and times. Install size is over 4MB :-( https://pendulum.eustace.io/ https://pendulum.eustace.io/
- tgsovlerkhgsel 3y agoEvery time I see news like this, the voice in me that is saying "timestamps should be represented as epoch-time integers" grows louder. The amount of pain I've seen "better" solutions bring feels like the benefits aren't worth it unless you need some very specific feature (e.g. you're not trying to express an absolute time but repeating calendar events).
- globular-toast 3y agoTimestamps yes, but you can't represent meaningful future times that way. You should also record the timezone with the timestamp, though. An epoch-time integer will unambiguously tell you what time something was recorded at, but it can't tell you what the time on the clock was at that time. If you're doing some kind of data analysis on user habits, for example, you almost certainly want to work with user local times, not UTC.
- maegul 3y agoHmmm. Can’t help but wonder if it would have been better to have the function utcnow simply return a time zone aware object instead. Would any usages of the function break?
- netvl 3y agoAfter Java’s JSR-310 almost any other API in almost any other language looks broken or insufficient. Threeten’s modeling of different time types seems so powerful and natural - specifically, it has Instant (which is literally a nanosecond-precision UNIX epoch timestamp), then it has ZonedDateTime/OffsetDateTime, which are full timestamps with time zone or offset information, and LocalDateTime/LocalDate/LocalTime, which are basically these “naive” timestamps from Python. These are all different types, with well-defined conversion methods between them, so you can’t easily use an inappropriate type. My personally most hated piece of date-related API is Go’s formatting API, with its asinine number-based patterns instead of something reasonable and conventional like strftime-like or Java-like symbols. What’s especially maddening in it is that it is so US-centric, with month coming before day in these number patterns, which does not follow unit size increase or decrease sequence.
- vsnf 3y ago> What’s especially maddening in it is that it is so US-centric, with month coming before day in these number patterns, Whether this particular bit of design is good or not, you'll certainly be hard pressed to find someone who has a reasonable argument in favor of the absurdity that is the Go date-time format string. > 2006-01-02 (aka 'yyyy-MM-dd') > January 02, 2006 (aka 'MMMM dd, yyyy') > Jan-06 (aka 'MMM-yy') > Monday (aka 'EEEE') It's insanity start to finish and it boggles the mind how it was ever accepted in the language spec.
- deleted 3y ago[deleted]
- aftbit 3y agoThis is a bad change. Many applications work with naive datetimes that are implicitly UTC. The current way to get these objects is: datetime.datetime.utcnow() Now the easiest way to get those will be this soup: datetime.datetime.now(tz=datetime.timezone.utc).replace(tzinfo=None) Why not just use .now()? Sometimes your code gets run on machines that have varying local times, but you still want to work with implicit UTC times. For example, when testing server code, or when writing a client application.
- Dylan16807 3y agoBut why not use explicitly UTC datetimes? You know they're UTC. Why do you want them to be naive?
- aftbit 3y agoBecause the existing codebase already uses naive datetimes. For example, perhaps there is a feature flag component that includes checks like: input_dt > datetime.datetime(2023, 1, 2, 3, 4, 5) As mentioned elsewhere, it's common to serialize datatimes without timezone because it's a huge waste of space and speed in databases and elsewhere to constantly include the timezone info. Of course you could handle this at the edge like many other kinds of serialization, but many apps exist which instead choose to keep all datetimes naive and handle TZ conversions at the edge instead.
- raminf 3y agoOne of the unusual things Swift did during its transition to a real programming language was to promise that there would be breaking changes. This way, everyone knew what to expect. Customers, co-workers, contractors, and management knew where things stood. If every programming language and API/ABI did this from the get-go, there would be so much less angst and hair-pulling each time some old, bad design was deprecated.
- Kab1r 3y ago> Note that if you are using Python 3.11 or newer, you can replace datetime.timezone.utc with a shorter datetime.UTC Pretty sure I did that!
- t_durden 3y agoAll these breaking changes coming from Redhat engineers. All while Guido works for Microsoft. Curious.
- pacifika 3y agoYou don’t need timezones until you did need them. Good change this avoids a whole class of subtle bugs. We can’t change Python 3.3 now.
- deleted 3y ago[deleted]
- kstenerud 3y agoI look at the arguments in this post and see echoes of Y2K: - It's a clever hack based on an assumption that dismisses the fact that this assumption can change (that the time zone will never be changed, or that everyone will follow the same rules once it does). - The argument for doing it is: It "uses less space" and "takes less time" These are similar to the reasons why we used 2-digit years for so long. Timestamps without time zone data are BUGS (and I mean REAL time zones, not time offsets like CET and PST). They are landmines just waiting to go off once the assumptions they're based upon change. They require your host system to be configured just-so, making your software brittle. They push the ugly details of handling many time issues that could be handled centrally into the user-space (and even worse, they make it OPTIONAL). The Python maintainers are right to remove this, and really they should have removed naive datetime when they released Python 3. Time is one of the most notoriously difficult things to get right. I've yet to see software using time calculations (or anything more complex than recording timestamps of past events - the only thing for which UTC is useful) that is bug-free. Time data ALWAYS REQUIRES time zone data. The only "exception" is timestamps of past events, but only because we always assume them to be UTC (and so there's an implied time zone so long as nobody bucks the trend).
- globular-toast 3y agoNot quite. Local time is very important. "Tomorrow at 9:00" is a meaningful local time for everyone, but doesn't refer to the same global time. It would be wrong to attach a time zone to such data. Past events, on the other hand, do require timezone data if they are recording human activity, which is basically everything. "Just use UTC" is data loss. You now don't know whether at event took place at midday or midnight. Two very different results for many activities.
- kstenerud 3y ago"Local time" is time zone metadata. I've written a fair bit about timekeeping, because the context of what you're capturing becomes very important: https://github.com/kstenerud/concise-encoding/blob/master/ce-structure.md#appendix-b-recording-time https://github.com/kstenerud/concise-encoding/blob/master/ce...
- al_be_back 3y agofor me, it's UTC everywhere by default, let clients convert UTC to local time. dbs/upstream systems do that already (these days). computing is predominantly distributed these days
- redandblack 3y agoIs there a suggested local date and time implementation? Simple example is parsing a csv that has date and timestamps data - and are indifferent to any timezone
- mharig 3y agoUTC does not have a timezone, so it is not a 'naive' timestamp. It just coincidentally has the same format as naive timestamps. Deprecating the utcnow(), because some folk does not think enough, instead of teaching them the correct way, is plain stupid. Timezones are jurisdictional. Their exact definition may change (in the future). So store datetimes as UTC, depending on your requirements either as Unix epoch or ISO8601 string.
- omnicognate 3y agoIt's not a timezone but the timezone mechanism in python is used for time standards (which it is) as well as zones so the distinction is irrelevant. You certainly can have a python datetime with its timezone set to UTC and the naming of utcnow() strongly suggests that's what it returns. Besides, this isn't a question about what is right in some abstract sense. It's a usability issue, and this accidental misuse of utcnow() is common. Python tries to avoid containing APIs that frequently trip people up, even if there's a "well akshewally" argument that they are in some sense correct. Not that there is, in this case.
- mharig 3y ago> You certainly can have a python datetime with its timezone set to UTC and the naming of utcnow() strongly suggests that's what it returns. No. You can have a datetime with timezone +00:00. UTC IS NOT A TIMEZONE. And utcnow() definitely suggests that you get an UTC. And this is right in a very practical sense. If it trips people up, they should be taught the right thing.
- omnicognate 3y agoWriting it in caps doesn't make it true. In real life UTC is a time standard, not a zone, but in python and in most other date/time mechanisms it is treated the same as one. It exists in the IANA tz database. A python "aware" datetime carries a "tzinfo" object. This can be a timezone object which just uses a fixed offset from UTC or it can be a more sophisticated subclass of tzinfo such as the one provided in the zoneinfo module (since Python 3.9), which uses the IANA timezone database and so can express things like "America/Los_Angeles" (and, indeed UTC). The UTC timezone object provided in the datetime module (datetime.timezone.utc) is the former kind, an instance of timezone, which is sufficient because of course UTC can be expressed as an offset (of +00:00) from UTC. However it isn't just the offset. Try creating a datetime using tzinfo=timezone.utc and call tzname() on it. It returns "UTC". It knows that it is called UTC, not just that it is an offset of +00.00. Now try calling datetime.utcnow().tzname(). It will return None. Also try calling datetime.utcnow().utcoffset(). It will also return None (not 0). This is because the value returned by utcnow() is a "naive" datetime: it doesn't have a tzinfo at all. That doesn't mean it represents UTC (or an offset of +00:00 from it) as you appear to think. It means it has no timezone or offset specified. It is just a nominal time. Plain nominal times with no timezone information are useful in some circumstances, but they do not represent an instant in time. A "naive" datetime of 12pm 1st jan 2023 means just that. It's a valid concept in its own right but it doesn't denote a single point in linear physical time. For that you have to add more information. The information that, in python, goes in the tzinfo. You say yourself that "utcnow() definitely suggests that you get an UTC" but you then seem to be under the impression that it does so. It does not, because it returns a naive datetime, and a naive datetime is not UTC or offset zero. A naive datetime has unspecified timezone/offset. It could represent UTC or CST or BST or all three at the same time or none of them. It is unspecified. This is very misleading behaviour from a function that has "utc" in its name. All of the above is easily verified by consulting the docs. I don't know why I took the time out of a busy day to write it, given the laws of ego on the internet mean you're unlikely to even read it all let alone thank me for writing it. Nonetheless, I hope it might help you avoid at some point making the exact mistake this deprecation is trying to prevent.
- Nonosfe 3y ago[flagged]
- Nonosfe 3y agoTimestamp is always utc, unless something is messed up. On it's own, Timestamp are already timezoned, to utc-0. Not the most difficult thing to master if you keep that in mind. Honestly i dont see the benefit in the change, just confusion.
- Belphemur 3y agoThey should have gone further like .NET 8.0 with a full class abstraction for datetime provider: https://learn.microsoft.com/en-us/dotnet/core/whats-new/dotnet-8#time-abstraction https://learn.microsoft.com/en-us/dotnet/core/whats-new/dotn... This is 10x better and helps to assure the code can be unit tested with providing the exact date we want in the exact format/timezone
- lallysingh 3y agoThat's a lot of complexity. How's that better than requiring an arg with the timezone?
- rjbwork 3y agoBecause you can now fully simulate arbitrary times during testing by providing your own implementation of the time provider. If you just take in a timezone as an argument, how will you simulate it being 2041-08-23T21:17:05.023743Z?
- SpaghettiCthulu 3y agoUh, just assign your mock function to `<insert name here>`? Python is really the wrong language to be complaining about a lack of mockability.
- rjbwork 3y agoTrue. When you can just make any function be whatever you want whenever you want that sort of abstraction does seem a big much. I was responding to the specific question in reference to the time provider in .net 8.
- SpaghettiCthulu 3y agoAh, fair enough. C# certainly presents more of a need for that sort of abstraction, being a statically typed language.
- talboren 3y agoPaul Ganssle released a great article[1] explaining why you shouldn't use utcnow in 2019. [1] https://blog.ganssle.io/articles/2019/11/utcnow.html https://blog.ganssle.io/articles/2019/11/utcnow.html
- m12k 3y agoTime handling seems like one of those cases where a type system could really come to the rescue - times with and without time zones in them shouldn't be the same type, and using one in a place where the other is expected should ideally just give you a compile error.
- physicles 3y agoThis would be fantastic. As someone who just occasionally touches Python, I can’t count the number of times I’ve been bitten by bugs caused by unwittingly mixing tz-naive and aware datetimes.
- vore 3y agoTimezoneless time I think is just a mistake in general – all times inherently have some timezone, even if the timezone is UTC.
- IggleSniggle 3y agoAll timestamps even have a position in space that will experience discrepancies relative to all other timestamps, even if they are UTC, we just usually don't care about it that much. Since our time is at least all Earth-based, we could do even better than timezone and make timestamps have a lat/long attached for the place in which it was produced. Maybe a linked list for any timestamps that are calculated from that based type. Are we having fun yet?
- vore 3y agoThe social solution is to abolish timezones (mine is of course objectively the right one, other people must deal with breakfast at 9pm wherever they are).
- gcbirzan 3y agoThey don't, not always. I want to wake up at 7am every morning, regardless of where I am.
- vegabook 3y agoI suspect a lot of financial data code is going to break. I've worked on a number of codebases where the explicit instructions were "everything is UTC, all the time", and enforced at the entry edges of the code. Bringing in timezones was a definite NO for reasons of the unholy mess this would inevitably create: * India has half-hour timezones * Countries change their DST rules all the time. * Conversion between timezones is an utter mess. As Wes McKinney once said about python datetime: "Welcome to Hell"[0] [0] https://stackoverflow.com/a/13753918 https://stackoverflow.com/a/13753918
- Buttons840 3y agoHow do you express "our stores, across 6 different timezones, all open at 9:30 AM local time" in UTC?
- pelorat 3y agoyou don't, since they are different times?
- deleted 3y ago[deleted]
- sillysaurusx 3y agoThat code runs in the context of humans. Therefore it needs to take humans into account when running. This is a bit like asking "how do you represent a string length in utf-8?" The answer is, you don’t. It’s almost never a problem in practice. Just like there is almost never a reason to care about the number of characters in a string, it’s very difficult to imagine that this code comes up in practice. But if it did, the code is simple: for a given timezone, is the current time in utc past 9:30am? If so, the store is open.
- Buttons840 3y agoLet's say you host a store front for many businesses and it's decided that your customers, who are businesses, some with many stores, want to store a default opening time. Something along the lines of "All Home Depot's open a 6 AM, all Lowes open at 7 AM."
- hanniabu 3y agoThis is just dumb... > I would have made it more clear that these functions work with naive time, maybe by calling them naive_utcnow() and naive_utcfromtimestamp(). > But their names are not the problem here. The specific issue is that some Python date and time functions accept naive timestamps and assume that they represent local time, according to the timezone that is configured on the computer running the code. Uhhh, no I would not expect local time because as the name entails we're dealing in UTC. Sigh.....
- vore 3y agoYou might not, but the standard library sure does. And that's the exactly the problem! How can functions in the standard library know if the time is local or UTC without any indication on the value itself? (spoiler: they can't!)
- BugsJustFindMe 3y agoThe less stupid solution is to just make those functions produce zone aware datetimes because you have literally asked for a specific time zone.
- seanp2k2 3y agoAs the article author correctly points out, Python should have just thrown when a method is called that requires a TZ with a datetime that doesn’t include one. Between this, the Unicode circus, and the rejection of PEP582, Python seems more and more like the HOA that isn’t earning their monthly fees IMO.
- BugsJustFindMe 3y agoI don't understand why the functions weren't just made to return non-naive results. You've literally asked for UTC. There's no excuse for allowing the result of that to be interpreted as something that isn't UTC. Set a separate flag that governs conversion to string to avoid different ISO output for shitty string parsers if you want, but there's no great reason to not just fix the functions so that they continue to work better than before and do the thing that people actually expect them to do, which is produce UTC. Intentionally naive storage or transfer is irrelevant. That's just serialization. And when you explicitly tell the computer to "deserialize this as UTC" it should fucking do what you told it to and give a zone aware result.
- ak217 3y agoLet's imagine that I have a perfectly reasonable looking piece of code in my app: db.write(Event(timestamp=int(datetime.datetime.utcnow().timestamp())) After the change that you propose, the timestamps written by this code will jump by an amount equal to the offset of the local timezone this machine runs on (and also lose any DST adjustment they might have had). Now, let's imagine another independent process is consuming events from the database and compares an event written before this change to one written after. Python is a popular language, used in production by scheduling applications around the world. The potential downside of this kind of change is measured in billions of dollars. The safe solution is of course to not silently change the API in a breaking way, and instead introduce a new API and get someone to look at why the code stopped working.
- wodenokoto 3y agoOn my machine, a naive timestamp from utcnow() and a utc timezone aware datetime object from datetime.now(tz=ZoneInfo('utc')) produces the same timestamp. datetime.now() does not return utc time (but does return a naive datetime object)
- ak217 3y agoOK, but that's not relevant to the scenario that I described.
- ryandrake 3y agoI'm not a Python guy so am not familiar with the culture, but I'd assume many, many programs use the deprecated function, given that it made "the news." It seems not right for backward compatibility to deprecate something many developers are using, simply because the old one wasn't thought out very well, and "developers should switch" to the explicit functions. Especially since, as the author demonstrates, the deprecated one can be easily re-implemented. It seems like a developer inconvenience was added without changing the state of the world. Things like this--I don't know--make me take a development environment less seriously. In C, we all know we shouldn't use things like strcpy() and gets(), and you'd be a total fool to add them to new code. But for the sake of backward compatibility with programs that aren't going to change, they should be kept in and documentation should nudge people to do it the right way. Maybe that's the Python way of nudging: "This function is deprecated!"
- jkrejcha 3y agoFwiw, gets() was actually removed in C11 because of it's inherit security problems. :) But aside from that, I think it's just a different style. Waiting 10 years from deprecation to remove a deprecated thing might be required with C just because of how prevalent C is and how much the developers in that have to think about basically every platform, but removing something on a major release isn't inherently bad, it's just a different tradeoff (being able to move a bit faster and fix some design issues vs. being very very good at backwards compat) The deprecated function can obviously be reimplemented pretty easily, but so can gets(). With utcnow(), it had some unfortunate effects in that it didn't really do what people thought it did, and because of that, most of the uses were basically bugs to begin with. (Having experience with .NET for example, you'd think that DateTimeOffset.UtcNow and datetime.utcnow() are semantically similar, but the problem is that utcnow() doesn't actually have a timezone attached to it. Adding one would also be a breaking change, and one that is more subtle.)
- akira2501 3y ago> Fwiw, gets() was actually removed in C11 because of it's inherit security problems. :) It always exists, it's just that the import gets controlled by a macro. Which, hints at a way Python could have solved this same problem without requiring auditing and rewriting code.
- sillysaurusx 3y agoOne advantage of getting older is that you see a lot of change. In the long run, simple ideas tend to win. And there have been lots of cases where the programming world collectively agreed that some more complex idea was the right way, then a decade later everyone has reverted to the simple idea. UTC is simple and unambiguous. Time is a number representing a fixed point in time. When displaying it for humans, convert it into human form. When this principle is applied everywhere, it’s hard to make mistakes. My gut says that deprecating utc as the native time representation will be one of those ideas fervently embraced by a few, ignored by most, and then quietly forgotten about in a decade.
- p5a0u9l 3y agoThey’re not deprecating UTC. Did you read the link? They’re deprecating functions that read as authoritative, but are actually naive about timezones, which can less to non-UTC results.
- sillysaurusx 3y agoIf that were true, the recommended fix would be to continue using utc (as numbers). Instead, the recommendation is to use datetime objects. This, I think, is the idea that will be quietly ignored in the long run.
- 38 3y agoGo seems to work as expected here: package main import ( "fmt" "time" ) func main() { dt := time.Unix(0, 0).UTC() fmt.Println(dt) // 1970-01-01 00:00:00 +0000 UTC fmt.Println(dt.Unix()) // 0 }
- skitter 3y agoIt does, which is great. Note however that Go times can be either wall-time or (wall-time, monotonic time), which have different semantics and are both represented by the same type.
- forgotpwd16 3y agoDoes Go also has a distinction between naive/timezone-aware times? If not don't see why would expect something else. This is an issue because the distinction exists. Note it isn't bad that it does. There're actually arguments in favor of it. It's bad that can freely mix them.
- 38 3y ago> Does Go also has a distinction between naive/timezone-aware times? Why would you ever want or need that?
- forgotpwd16 3y agoSuppose you want to run a routine after 12h. That is now is 7 PM and want it to run tomorrow at 7 AM. Like if simply print times you get: t1 = datetime.now() t2 = t1 + datetime.timedelta(days=0.5) print(str(t1)) # '2023-11-19 19:00:00.000000' print(str(t2)) # '2023-11-20 07:00:00.000000'
- 38 3y ago> That is now is 7 PM and want it to run tomorrow at 7 AM. Like if simply print times you get is this English? also I am still not seeing a problem: package main import ( "fmt" "time" ) func main() { t1 := time.Now() t2 := t1.Add(12 * time.Hour) fmt.Println(t1) // 2023-11-19 15:14:16 fmt.Println(t2) // 2023-11-20 03:14:16 }
- slotrans 3y agoThis was an aside and not the point of the article but I can't let it go: > It is also a best practice to store naive datetimes representing UTC in databases. It is most definitely NOT a best practice and you SHOULD NOT DO THIS. If your database supports TIMESTAMP WITH TIME ZONE -- which the "aware" type per the article's terminology -- you should be using it. If your database doesn't, you should get a better one.
- klysm 3y agoThis is a good idea for Postgres but only because of its specific semantics
- spsesk117 3y agoWould you care to elaborate? I'm not an expert in this area so I'm probably missing something, but my thought has always been the 'portability' of storing everything in the backend as UTC is more flexible/less complex in the long run, dealing with conversation at the 'last mile' so to speak. Happy to be corrected here but it's not intuitive to me that dealing with timezones as the 'base unit' makes anything easier.
- toyg 3y agoIt's the usual divide between people who actually know how databases work, and hence want to do more stuff there (because it will likely be faster and more robust); and people who don't know (and don't want to know) anything about databases, and hence would rather use them as dumb stores of basic data. I've been in each camp at one time or another, since being db-agnostic makes sense in some cases (libraries, products) and not in others (websites, services).
- int_19h 3y agoIt's not an either-or. You can store everything in UTC while also using TIMESTAMP WITH TIME ZONE. The timezone will always be UTC, but that way at least anyone who will be dealing with this data later knows that to be the case, instead of assuming.
- Denvercoder9 3y ago
- FObersteiner 3y agoI think naive datetime would be fine, even if it resembles UTC - as long as you don't mix in a serial representation. The module documentation says, "Whether a naive object represents Coordinated Universal Time (UTC), local time, or time in some other timezone is purely up to the program, [...]". That definition goes down the drain imho once you call `.timestamp()`, which gives you Unix time as a serial representation (epoch time), which must be referenced against aware datetime, namely 1970-01-01 00:00 UTC. Same trouble vice versa (`.fromtimestamp()`).
- BiteCode_dev 3y agoThe snippets the article give are good to get an UTC datetime or timestamp now. I'd like to add that if you have any fancy datetime manipulation to do in Python, pendulum (https://pendulum.eustace.io/ https://pendulum.eustace.io/) is probably your best bet.
- nooorofe 3y ago>pendulum is probably your best bet It is best until you work with Pandas DataFrame (https://stackoverflow.com/questions/47849342/making-pandas-work-with-pendulum https://stackoverflow.com/questions/47849342/making-pandas-w...)
- BiteCode_dev 3y agoNumpy and pandas are their own little island. It's not just dates and time, it's everything. If you use numpy and pandas, you should also not use Python datetime, generators, most stdlib mathematical functions, the itertools module, random, etc. It's the first thing you learn if you read any good pandas book, and the first thing I teach in my numpy/pandas trainings. It has pretty much nothing to do with pendulum. Basically, half the Python ecosystem is "well, except with numpy/pandas of course".
- nooorofe 3y agofrom zoneinfo import ZoneInfo import pendulum import datetime def pendulum_to_datetime(pendulum_dt: pendulum.DateTime) -> datetime.datetime: return datetime.datetime.fromtimestamp(pendulum_dt.timestamp(), ZoneInfo(pendulum_dt.timezone_name)) # test df = pd.DataFrame([[1, 2], [1, 2]], columns=['a', 'b']) df["time_column"] = pendulum_to_datetime(pendulum.now()) print(df) output a b time_column 0 1 2 2023-11-19 18:14:16.027777-05:00 1 1 2 2023-11-19 18:14:16.027777-05:00 >> df.dtypes a int64 b int64 time_column datetime64[ns, America/New_York] dtype: object
- contravariant 3y agoYou know what would help people adopt timezone aware timestamps? Just make now and utcnow return timezone aware timestamps! What's the point creating a naive timestamp when you know the time?
- oblvious-earth 3y agoDiscussion on Python discuss: https://discuss.python.org/t/deprecating-utcnow-and-utcfromtimestamp/26221 https://discuss.python.org/t/deprecating-utcnow-and-utcfromt... I'm the one arguing against this move, I've already made my thoughts clear that it's a lot of churn, e.g. the PR to remove it from Pip literally caused Pip to have to do a bug release, and has valid use cases, e.g. modelling datetime data types from spreadsheets and databases that do not have timezones. But I also accept I'm not the one maintaining Python, so I've updated my codebase appropriately (`datetime.datetime.now(datetime.UTC).replace(tzinfo=None)`). Python is not a backwards compatability language, and it's clearly not a strong motivation for most users as I don't see anyone attempting to write a backwards compatibility shim over the standard library, so no point complaining too much.
- deleted 3y ago[deleted]
- tomp 3y agoBad idea. datatime.utcnow() should be the default way of recording & storing time, e.g. in logs or in the database. All other options are strictly inferior. >>> str(datetime.datetime.utcnow()) '2023-11-19 16:46:04.049705' >>> str(datetime.datetime.now(datetime.UTC)) '2023-11-19 16:46:35.924671+00:00' There's no reason for `+00:00` visual noise in the UTC-aware datetime.
- forgotpwd16 3y ago>datetime.datetime.now(datetime.UTC) There're many things to complain but this is certainly not one. datetime.datetime.now(datetime.UTC).replace(tzinfo=None) Though this should be `datetime.timezone.utc` rather `datetime.UTC` if not mistaken.
- tomp 3y agonot sure, I tried this in Python 3.11.6 and `datetime.UTC` works whereas `datetime.timezone.utc` doesn't exist in any case, just compare datetime.datetime.utcnow() and datetime.datetime.now(datetime.UTC).replace(tzinfo=None) verbosity is not a virtue in this case
- remram 3y agoThis is verbose because it is dissuaded. The whole point of this change is to not have users get naive datetime objects, of course it gets more verbose.
- int_19h 3y ago`datetime.timezone.utc` is the original way of doing it and should work just fine on a stock Python 3.11 install. `datetime.UTC` is a relatively recent addition, and if you look at the actual type of the object it returns, it's still `datetime.timezone.utc`
- forgotpwd16 3y agoSo my installation was outdated. Still not fan of having two different ways to refer to the same thing.
- amluto 3y agoIt would be nice if there were separate types for naive and aware datetimes. Then mypy and similar tools could actually check code for correctness.
- karmakaze 3y agoIsn't Python adding gradual typing? This is the sort of confusion that becomes apparent when we know and write the types (TZ aware or not) coming out of into functions.
- jes5199 3y agoI’ve definitely been burned by assuming that “no time zone” means UTC when my python environment assumed I wanted it to mean Pacific time - it’s like the APIs all prefer naive datetimes, and I had to learn a new trick for each one to convince it to do the right thing
- neillyons 3y agoPython should use different types to represent datetimes with a timezone and those without. class Datetime: pass class NaiveDatetime: pass That way you can annotate functions properly. def timezone(datetime: Datetime) -> ...: This is how Elixir does it. So much easier to understand as a newcomer. https://hexdocs.pm/elixir/DateTime.html https://hexdocs.pm/elixir/DateTime.html https://hexdocs.pm/elixir/NaiveDateTime.html https://hexdocs.pm/elixir/NaiveDateTime.html
- llbeansandrice 3y agoThis seems like the blatantly obvious path to me. Perhaps that needs to be dealt with in a major version release tho. TZ naive and TZ datetimes are needed and it’s important to chose the correct one.
- dikei 3y agoSo is Java with `LocalDateTime` and `ZonedDateTime`, a very useful example of making your typesystem work for you.
- usrbinbash 3y ago> To me it is clear that the Python maintainers behind this deprecation have a problem with naive datetimes and are using this supposed problem as an excuse to cripple them. I am not a python maintainer, and I also use naive datetimes a lot, and I can absolutely understand the maintainers rationale behind this. Let me explain. What's a "naive datetime" suposed to be conceptually? There is no such thing. Datetimes only make sense in the context of a timezone. UTC is just another timezone, with an UTC-Offset of +0 So why do we have this strange thing that may be a UTC timestamp, or a local timestamp, or god knows what in the first place? It makes sense as an implementation detail of the datetime library to have timezones as separate objects, but for consumers of that library? It can cause confusion, and hard to track bugs, when suddenly one of the functions that accept aware or naive is used and spits out something completely unexpected. I have spent many an hour tracking bugs, where systems spat out complete nonsense timestamps, only to discover that the reason they do that, is a mixed usage of aware and naive dateteime objects. In my opinion, making explicit use of aware datetime the norm as much as possible, is a good thing. And removing prominently named functions that spit out naive ones, is a good first step. Let's look at the listed usecases: > An application may be designed in such a way that all dates and times are in a single timezone that is known in advance. Okay...so put that TZ info somewhere it's easy to find, like a configuration Singleton, and use it. > It is also a best practice to store naive datetimes representing UTC in databases. Okay...read them from, and into datetime objects using UTC as their timezone information. The bottom line is: There shouldn't be "naive" datetime objects, or at least they shouldn't be a common thing. Conceptually, datetimes always have a timezone attached, they are worthless without one.
- vegabook 3y agoall true but you miss that naive UTC has become a _de facto_ standard in many (particularly finance) codebases. Sometimes conceptual beauty must unfortunately reckon with long-established reality. It's basically very late to be doing this.
- usrbinbash 3y agoI certainly didn't miss that, as I pointed out myself: I use naive datetimes in my codebases. Alot of them. And this change generates quite a bit of work for me. But I'm okay with that. The problem with de facto standards is: The fact that something became de facto standard, doesn't make it a better idea. The only thing it means is that it is now even harder to get rid of it. > It's basically very late to be doing this. Yes, it is. But not yet impossible. Languages and libraries that wait forever instead of ripping off the bandaid at some point, eventually become so crusted with such early problems, they become problems themselves.
- fishywang 3y agoThe naive time thing in python really is (was) a nightmare. I once noticed that a db library we use when storing a timestamp in the db would calculate that timestamp from 1970-01-01 00:00:00 at _local time_ instead of utc and had to patch that to make sure we never store a wrong timestamp in our db (luckily all our production servers run on UTC as the local timezone). Glad that it's finally being addressed at the language level.
- orf 3y agoI’ve been using Python for going on 14 years now, and I had no idea utcnow() returned a naive date time. I happen to live and deploy to a (mostly) UTC time zone, so i expect a bunch of my code is subtly wrong. One of my projects added all of the code in PyPI to GitHub, it would be interesting to see how prevalent the use of “utcnow()” is.
- arp242 3y ago> An application may be designed in such a way that all dates and times are in a single timezone that is known in advance. In this case there is no need for individual datetime instances to carry their own timezones, since this uses more memory and processing power for no benefit, since all these timezones would be the same and it would never be necessary to perform timezone math or conversions. The memory and processing differences between the two seem marginal, at best. Especially when it's at UTC ("plus zero" doesn't take too much processing). So I'm gonna have to ask for benchmarks on this one. And even if it was slower one must ask if having a confusing concept like a naïve datetime is worth it. Obviously Python is all about the "programmer productivity at the expense of some performance" trade-off.
- remram 3y agoYeah, both None and UTC are singletons so there should be exactly 0 difference in memory usage between a datetime with tzinfo=None or tzinfo=UTC.
- blincoln 3y agoAm I correct that the datetime.datetime.now(datetime.timezone.utc) syntax is backwards-compatible all the way to Python 3.2?[1] If so, this seems like a good move. The edge cases like compatibility with data sources that don't indicate the time zone seem like less of an issue than data corruption due to developers not realizing that they were working with "UTC" timestamps that weren't actually UTC. I'd be more concerned if this was one of those changes where there isn't a good way to write reasonably straightforward code that will run in relatively old versions of Python as well as newer ones. [1] https://peps.python.org/pep-0615/ https://peps.python.org/pep-0615/
- lowbloodsugar 3y agoHahaha. Fucking python. This kind of shit is why you’d be mental to use it for anything that can’t be rewritten in a day.
- drexlspivey 3y agoHaha yes you'd have to be crazy to use the most popular language in the world.
- qwertox 3y agoI was recently working with stock data from a Django shop, half of their Unix timestamps were correct timestamps, the other half had a local timezone offset baked right into the integer. At least this time finding problematic code will be a grep away, so I'm ok with this change, even though I've used it a lot. Edit: Wow. So Paul Ganssle writes in April "Previously, we have documented that utcnow and utcfromtimestamp should not be used , but we didn’t go so far as to actually deprecate them [...]" The linked to issue where this was decided is from Jul-Sep 2019. And I've never heard about this. Obviously this is my fault, but: where do I need to inform myself to know about these decisions? I've been "happily" using them all the time (quoted because we're all aware of the challenges these conversions pose each and every time). [0] https://github.com/python/cpython/issues/81669 https://github.com/python/cpython/issues/81669
- pierotofy 3y agoPython breaking stuff with another minor version update? Shocker...
- phendrenad2 3y agoThe problem with powerful languages is you can misuse them. The maintainers are deprecating a perfectly-good API, which is likely used by many people, simply because someone, somewhere, might not have read the (very clear) documentation and misused it. Maybe they should deprecate strftime() and strptime() because someone might serialize a datetime and drop the timezone.
- kerkeslager 3y agoI've noted before and will note again that the amount of hate Python got for the 2-to-3 deprecations has led to a much worse problem: piecemeal low-visibility updates which are much harder to plan for. As a contractor, I singlehandedly migrated 4 different codebases from 2 to 3; a total of just under a million lines of code, and none of these codebases took more than 40 hours. This was not some horrific burden for most companies using Python. The people who flamed Python for these breaking changes were either extreme outliers or the sort of people who would be happy with the language never making breaking changes, ever, no matter how bad of mistakes in design were discovered. I would strongly prefer that datetime.utcnow were not deprecated in a 3.x release, but instead deprecated along with the GIL and any other breaking changes, in Python 4.0. Yes, it would get them flamed. And the flamers are wrong. At least that way people can plan for the transition at their leisure instead of constantly worrying that a 3.x release will unexpectedly break their codebase if they don't read the release notes before every update.
- dheera 3y agoDeprecation shouldn't break peoples' code all of a sudden. Code should still work, and just get more annoying over time. Initially, spit out warnings that deprecation is going to happen. Then, start changing the user's desktop background. Then, start playing nice cat meowing sounds out of the speaker. Then, start playing ghastly wailing sounds out of the speaker. Then, start sending e-mails. Then, scan for IoT light bulbs and mess with the colors. Then, scan for bluetooth headsets within a 50 meter radius and play wailing noises out of those. Then, impose a 0.001 second delay. Then a 0.01 second delay. Then a 0.1 second delay. Then a 1.0 second delay. Then start scraping LinkedIn for engineer resumes and e-mail them to any sysadmin e-mails you can find in the various config files in /etc/ and /home/. Don't suddenly deprecate stuff and break people's code on a moment's notice. Most likely the person who wrote the code using the deprecated function was laid off 2 years ago and the company still depends on it.
- panarky 3y ago> Don't suddenly deprecate stuff and break people's code on a moment's notice. "Deprecated" doesn't mean "break people's code on a moment's notice". FTA: "datetime.datetime’s utcnow() and utcfromtimestamp() are deprecated and will be removed in a future version" "likely a few years out" "once you switch to Python 3.12 or newer you will start seeing deprecation messages on your console and your logs, and these can get annoying"