30 ms·

I suspect a lot of financial data code is going to break. I've worked on a number of codebases where the explicit instructions were "everything is UTC, all the

I suspect a lot of financial data code is going to break. I've worked on a number of codebases where the explicit instructions were "everything is UTC, all the time", and enforced at the entry edges of the code. Bringing in timezones was a definite NO for reasons of the unholy mess this would inevitably create: * India has half-hour timezones * Countries change their DST rules all the time. * Conversion between timezones is an utter mess. As Wes McKinney once said about python datetime: "Welcome to Hell"[0] [0] https://stackoverflow.com/a/13753918 https://stackoverflow.com/a/13753918

Buttons840 3y ago
How do you express "our stores, across 6 different timezones, all open at 9:30 AM local time" in UTC?
- pelorat 3y ago
  you don't, since they are different times?
  - deleted 3y ago
    [deleted]
- sillysaurusx 3y ago
  That code runs in the context of humans. Therefore it needs to take humans into account when running. This is a bit like asking "how do you represent a string length in utf-8?" The answer is, you don’t. It’s almost never a problem in practice. Just like there is almost never a reason to care about the number of characters in a string, it’s very difficult to imagine that this code comes up in practice. But if it did, the code is simple: for a given timezone, is the current time in utc past 9:30am? If so, the store is open.
  - Buttons840 3y ago
    Let's say you host a store front for many businesses and it's decided that your customers, who are businesses, some with many stores, want to store a default opening time. Something along the lines of "All Home Depot's open a 6 AM, all Lowes open at 7 AM."
    - wayfinder 3y ago
      That’s not date data though. That’s just storing local time - 6 AM or 7 AM.
    - sillysaurusx 3y ago
      The store property would be "6 AM". Anywhere it’s used, it’s running in the context of humans. When displaying on a website, you don’t need to do any conversions: the human viewing the store knows where it’s located, and they know the current time, so computers don’t need to answer the question of whether it’s open. If computers do need to know, then they’ll need to know a specific store with a physical location. That gives you the time zone.
      - macintux 3y ago
        > If computers do need to know, then they’ll need to know a specific store with a physical location. That gives you the time zone. Mapping a location to a time zone seems fraught with peril. I’d store the local time zone explicitly.
        amluto 3y ago
        Indeed. And anyone near a time zone boundary looking at an online map of store locations doesn’t want to see “opens at 6AM” or whatever — they want to know whether the store is open.
        JJMcJ 3y ago
        Indeed. It's not as bad as mapping address => sales tax (no ZIP Code isn't good enough), but only because it doesn't change as often and tends to follow county boundaries when it doesn't follow state boundaries. Another good example. Arizona doesn't follow Daylight Savings, but the Navajo Nation, which includes the Four Corners area and is in three states, does observe the change. However the Hopi reservation, completely surrounded by Navajo territory, is entirely in Arizona, and does not observe DST. So it can get quite complicated and it's not enough to say "Arizona doesn't have Daylight Savings Time".
  - deleted 3y ago
    [deleted]
- joshspankit 3y ago
  If someone expresses it that way it will be a huge hassle the moment they decide one store opens at a different time.
  - Buttons840 3y ago
    So maybe individual stores can customize their opening time, but corporate still wishes to store a default. How do you represent the default? How do you handle the DST time changes for individual stores? I'm trying to poke holes in the "always use UTC and you're good" advice. I have actually encountered this scenario, and I didn't know how to apply that advice.
    - bpfrh 3y ago
      I don't get your problem, couldn't you store it like this: table timetable: open | close | store 6am | 6am | default 7am | 12pm | store1 table stores: name | address store1 | xyx NYC If you can't find the store in the timetable: 1. Get the timezone of the target store by looking up the address from the table stores 2. Set Hours from the default row in the timetable table on the object with the timezone
    - Izkata 3y ago
      > I'm trying to poke holes in the "always use UTC and you're good" advice. There's a much simpler one - recurring meetings/events (in-person or local-ish like within one country) that cross a DST boundary. You always want these to be attached to a local time zone, so it doesn't suddenly happen an hour earlier/later than you intended.
      - fragmede 3y ago
        Where that gets even more fun is with international meetings, where the host of the meeting and the attendees are in different zones.
- bofaGuy 3y ago
  Not sure if this is a serious question. Each store will have an opening time, each will store the local time that the store opens.. just as utc.
- d0mine 3y ago
  utc is for a specific moment in time (deterministic with great precision for everyday use). "9:30am local time" may be ambiguous/unknown (the rules may not exist yet) Using the same programming type for both may cause confusion. How do you express geographic coordinates? Noon in London and in NY are different moments in time (it may be important if you want to have a remote meeting).
  - samus 3y ago
    No need for something as brittle as geographic coordinates or addresses. Time Zone names are the right abstraction. A given location very rarely [edit: adopts another] time zone (unless in case of annexations, handovers of territory, etc.), but time zone rules (zone offset from UTC, DST, etc.) change all the time. A consequence of this is that the time zone database of applications should be updated every few months!
    - d0mine 3y ago
      "brittle" ? Your suggestion to define places by their timezone name is ok (it may work in many cases).
      - samus 3y ago
        "Complicated" then. One would have to consult a map to find out the time zone from geographical coordinates. The time zone is really everything needed to properly perform timezone-aware arithmetic.
- Footkerchief 3y ago
  Postgres stores this as TIMESTAMP WITHOUT TIME ZONE, sometimes called wall-clock time. IMO this is a good convention -- treating "ideal times" and specific epoch moments as two different types -- and more languages should support this (ideally with a less confusing label).
  - kevindamm 3y ago
    Elixir does as well, with NaiveDatetime
- dumah 3y ago
  Tuples of the local time zone (not utc offset) and a time, along with a utc localization conditional on date.
- JJMcJ 3y ago
  Reference opening time to local time on each day of the year, and then refer that back to UTC. Like California is -8 in Standard, -7 in Daylight, so a 9:30 AM opening in the Winter is at 17:30 UTC, in the Summer, 16:30 UTC. It's a hassle but programmers are supposed to model reality. As an example of the problems that can arise, I remember when the new Japanese Emperor was to take office upon his father's abdication, there was a big fuss because the Era Name was customarily only revealed after the new Emperor's reign started but all the computer people wanted to be ready in advance. Not sure what finally happened, if it was revealed in advance or not.
  - tsimionescu 3y ago
    What if the date for DST changes tomorrow, but only in some locations? How will you know which stores times need to be adjusted and which don't? UTC to local time is not a function, it is not somethung that can be computed once and stored for the future. The two are fundamentally different concepts, and can only be related to one another in certain contexts, and only reliably for the past.
    - JJMcJ 3y ago
      > How will you know which stores times need to be adjusted and which don't? There are a number of libraries for this, usually updated by the unpaid volunteer in Nebraska. Or else your company has staff whose job it is to follow this for any place you operate. If you record opening times as UTC, well, then you have to change to UTC when the store switches standard <=> daylight. This is what Fred Brooks called an essential complexity. You can poke the pain from one part of your system to another part, but it's never going away. The function is complicated. There is no avoiding that. One Pacific island nation duplicated a calendar day when it switched International Dateline sides to more closely align with Australia instead of Hawaii, reflecting a shift in its economic ties. No idea how they handled it exactly but pretty hairy to model.
- konschubert 3y ago
  store_opening_hour_and_minute_of_day=(9,0) It’s not a date time.
- remram 3y ago
  "9:30 AM every day" is not a datetime. It's a time.
smooc 3y ago
Financial data deals with cutoff times and those happen in a timezone. Thus particularly in finance timezones are important. You are right though that internally everything should be UTC internally and only when required it gets transposed to the timezone. However many legacy financial systems assume a local time zone and not UTC and they do not expose that information in their data. Happy reconciliation :-).
- steveBK123 3y ago
  One edge case to this is I've worked on financial systems where we kept the Asian market systems in JST, while EU/MENA/Americas ran on UTC. As long as you aren't dealing with FX/Futures, markets are generally open no earlier than 7am and no later than 6pm. This worked nicely since it meant that the DATE in each regions system&local clocks agreed.
- midasuni 3y ago
  If the market opens at 9am New York time then that’s not the same as storing 1400 utc. An law could be passed to change the offset on a given day You have to store time in the relevant geographic time zone. Something occurring at 1200UTC on dec 1st is not the same as something occurring at 1200Europe/London on dec 1st. The two times may coincide today, but if the U.K. government decides to implement daylight saving next week as a fuel rationing measure, then your 1200London event will have to be displayed at 1300UTC, but your 1200UTC event would be at 1200UTC.
  - Kamq 3y ago
    I think you mean 1300GMT, I don't think the UK owns enough of the world these days to futz with UTC.
    - midasuni 3y ago
      1200 London time, so 1100UTC with daylight saving yes. Point remains, many events are linked to local civil time, which can change - and not just to daylight saving. The Line Islands in Kiribati shifted a whole day in 1994, same with Samoa in 2011. Have an event set for 0800 local on the first Monday of the year in Samoa which you stored as UTC, and your events would have broken. Unless you store the relevant data (it occurs on first monday according to the timezone that Samoa is in), then you will be wrong. Now of course you can still get problems if a new timezone is created, as time is stored as Europe/London, if Scotland creates its own timezone in the future a meeting in Aberdeen would no longer be in Europe/London but in a new area Europe/Edinburgh, with different behaviour, but that’s far less common than DST changes or other date shifts. All in all you should store the relevant datetime and zone, not just “normalise” it to UTC.
1letterunixname 3y ago
Using UTC as a universal time standard is stupid because it contains leap-seconds while TAI does not. TAI64* should be the preferred computational monotonic time format.
Spivak 3y ago
You still keep everything in UTC in this new world, it's just the object you use internally now knows that it's UTC and makes it so you can't accidentally do calculations on two naive datetimes which happen to be in different timezones. You get to have the typing help you enforce that everything is in UTC.
- konschubert 3y ago
  And you have to treat the timezone offset as information-free. As in, 02:00+00:00 and 04:00+02:00 are absolutely equivalent representations of the same data. If you need to know the timezone in which to display a time, you save that timezone separately
plibither8 3y ago
> India has half-hour timezones If that's messy, consider Nepal's timezone that's on the 45-minute mark (UTC+5:45).
- hprotagonist 3y ago
  or one island of new zealand (chatham), which is +12:45
  - slyall 3y ago
    The Chatham Islands are an Island group, two of which are inhabited (Chatham Island and Pitt Island). https://en.wikipedia.org/wiki/Chatham_Islands https://en.wikipedia.org/wiki/Chatham_Islands Right now is it on +13:45 due to Daylight Saving https://en.wikipedia.org/wiki/Chatham_Standard_Time_Zone https://en.wikipedia.org/wiki/Chatham_Standard_Time_Zone
  - scheme271 3y ago
    Nepal is UTC+5:45. I think that's the largest population on a 15/45 minute offset.
    - physicles 3y ago
      Fun fact: there are currently no time zones with 15 minute offsets.
      - iso8859-1 3y ago
        Why does Nepal not count as one?
  - angrygoat 3y ago
    or along the highway near the Western Australia/South Australia border, which informally uses +8:45 https://en.wikipedia.org/wiki/UTC%2B08:45 https://en.wikipedia.org/wiki/UTC%2B08:45
    - 3y ago
Hamuko 3y ago
I'd say that wouldn't enforcement be rather simple going forwards since you could just enforce that dt.tzinfo == datetime.timezone.utc, but as far as I know, timezone information checking in Python can be a bit of a pain. >>> utc1 = datetime.timezone.utc >>> utc2 = zoneinfo.ZoneInfo("UTC") >>> utc1 == utc2 False >>> dt1 = datetime.datetime.now(utc1) >>> dt2 = datetime.datetime.now(utc2) >>> dt1.tzinfo == dt2.tzinfo False
maxerickson 3y ago
They aren't being removed, they will just toss up a warning.
- tripflag 3y ago
  The warning is also opt-in; you'll have to run python with PYTHONWARNINGS=always or similar to see it. Which is scary, because you will be taking the performance hit from generating the warning regardless, which can easily amount to 30% of your total CPU time if you use utcnow/utcfromtimestamp a lot.
- tumult 3y ago
  They will be removed: datetime.datetime’s utcnow() and utcfromtimestamp() are deprecated and will be removed in a future version.
  - jagged-chisel 3y ago
    Aren’t now, “will be” in a future version. So the can will be kicked down the road as many generations of devs as needed until no one cares any longer.
    - tumult 3y ago
      So, they remove it in 12 months and your code wasn't ready for the change, and people mock you for having not been prepared by updating your code. The Python people have told you to remove it, so nobody will have any sympathy for you if you don't do it. As you demonstrated, people are sticklers for quoting exact wording when it comes to criticizing others.
      - Hamuko 3y ago
        Has anything in Python been removed in 12 months? TestCase.assertEquals was deprecated in Python 3.2 (February 2011) and was finally removed in Python 3.12 (October 2023). And that was 12 years for just a simple rename from "assertEquals" to "assertEqual".
jagged-chisel 3y ago
For sharing between systems and representing dates to humans debugging a system, the following has worked well … Planned local events: future date and time string including local time zone Planned global events: future date and time string in UTC, with UTC timezone Past events: date and time string in UTC with UTC time zone, unless there is a strong case to be made that these should be localized (see “planned local events” above) Strings in the format specified by ISO-8601. Of course, internal representation can be whatever it needs to be for efficiency. For example, maybe your database stores as integer microseconds since an epoch. Fine, but as soon as that leaves the database, ISO-8601 that date.
- theK 3y ago
  I like your premise but disagree with your guidance. You should not convert the integer early, you should actually wait until the very last moment. Nowadays a timestamp will be marshalled and demashalled by (at the very least) two runtimes until it reaches human eyeballs opening up a wide spectrum of bug potential, difficult to understand code and development slowdowns. Keeping it simple and unambiguous is very important. Time libraries have a habit of coming and going, ints are always there and 99% of the date arithmetic is trivial any way. Don't use a lib, just pretty print numbers and you'll be fine.
  - fluoridation 3y ago
    An int is not unambiguous, though. "2023-11-19T14:22:34-03:00" is clearly a timestamp and represents an unambiguous point in time, while a number by itself doesn't mean anything. You have to know that it's a timestamp (probably from context) and you have to know what timezone it's in.
    - Dylan16807 3y ago
      > You have to know that it's a timestamp So what? That's true of all formats. How is it relevant to the choice of which format to use at which stage of the process? > and you have to know what timezone it's in. You don't. > clearly a timestamp Not very helpful. It's easy to set up types in advance. Don't figure out if things look like timestamps at parse time, or you'll be causing the same problems that turn gene names into dates and choke on people named Null.
albert_e 3y ago
> India has half-hour timezones Not plural though; The whole of India -- even though it is pretty vast in area -- has a single time-zone country-wide. I am very thankful for this. We already have a lot of other challenges -- including dozens of different languages -- that a single time zone is a relief. But yes -- the +5:30 offset usually makes for a less-than-ideal factor to plan meetings or follow non local event schedules. Did I mention we also hate the Summertime / Daylight Saving time adjustments we have to accommodate twice a year when working with western teams -- more adjustment even after most people have already sacrificed on a sane 9-5 schedule to ensure they overlap with EU/US teams for daily work meetings. We cant imagine why US that has multiple time-zones already in a single country, and EU whose countries are often smaller than states in India ... need so much adjustment. Why can't they, for instance, simply declare that school/offices open at 10am instead of 9am for six designated months every year instead of imposing this adjustment on the entire world.
- steveBK123 3y ago
  Brazil's stock market does/used to do something along these lines when US & Brazil changed their clocks in opposite direction, to keep the NY time hours of the Brazilian exchange from drifting as far. Given that we all also observed DST at the time, this made things even more complicated for systems.
- screye 3y ago
  Having lived in both India and far North in the USA, I'd take half-hour time zones over daylight savings (switches) every time. Shifting time-zones increases car accidents by about 6-16% for the entire week (studies vary). Just that is a massive enough impact on safety and productivity to discourage the switch. At a personal level, leaving work at 5.30-ish with it being pitch black outside, is among the most depressing things in the world. Even more wasteful is that the average American wakes up at 7.20am. So, they don't gain any daylight but lose an entire hour. I'm not usually the type to make such accusations, so I apologize in advance for sounding culture-warry. But, switching-time zones is "white person thing". Large percent of the Chinese, Indian & the Middle Eastern population live above the 25th parallel (Florida) and large populations of Korea & Japan live north of the Bay Area without daylight-switching. Permanent daylight time with later sunrise would be a huge win.
master_crab 3y ago
It definitely will break financial firms. I just did a code search in our repos for utcnow and other relevant functions, and the indexer was definitely wheezing pulling in the hits.
- andrewprock 3y ago
  Deprecation should not break anything. At some point the code will be removed and the org will have to pin the version until the deprecations are resolved.
StableAlkyne 3y ago
> I suspect a lot of financial data code is going to break. Reminds me of when Numpy removed a lot of types like np.bool a few releases ago, because they were aliases of the standard library types. Which broke a ton of libraries having Numpy versions pinned to ">=1.0" or even worse, "~=1.0" expecting them to follow the standard practice of incrementing the major version when they break compatibility. It's not terrible because you just pin your project to 1.23 or whatever, but it was very annoying when everything I was using just started breaking. I wonder if Python plans to do the same with this - the 2 to 3 transition was painful and dragged on for a decade, nobody wants to do that again for 3 to 4. When they remove this function and break back compatibility with old code, is that going to happen in a future 3.x release?
- hexane360 3y ago
  It's worth noting that numpy and scipy have a consistent policy now, although it's not semver. Deprecations will always happen at least 2 minor versions before removal: https://numpy.org/neps/nep-0023-backwards-compatibility.html https://numpy.org/neps/nep-0023-backwards-compatibility.html https://numpy.org/doc/stable/dev/depending_on_numpy.html#runtime-dependency-version-ranges https://numpy.org/doc/stable/dev/depending_on_numpy.html#run... So, when using numpy in a package, and you've tested e.g. versions 1.12 to 1.22, you should require 'numpy>=1.12,<1.25', because something deprecated in version 1.23 could be removed in 1.25.
- jkrejcha 3y ago
  Yes, they're almost certainly going to do it a 3.x release. I think in some senses these backward compatibility breaks have already happened. I remember that Google got bit by async becoming a keyword in I think Tensorflow. Generally a lot of old Python 3 code still works today if it doesn't do anything really weird like that, but I do definitely think that the pain of Python 2 to 3 was painful so they don't really want to repeat it. I think it is probably prudent to treat point releases as major releases at least with regards to Python.
notatoad 3y ago
Removing utcnow() doesn't force you to use tz-aware dates though. That's just a leap that the author of this article has made. If you're working in a system where all dates are utc, then datetime.now() gets you a date in utc and you don't need datetime.utcnow(). The only reason to call utcnow is if you're working on a system where the local timezone isn't utc, so you inherently have to deal with timezones at some level. Which means that using naive timestamps is a bad idea.
- nerdponx 3y ago
  datetime.now(tz=timezone.utc) is what you probably want most of the time anyway. You can always manually strip off the time zone component if you really want to use datetimes without associated time zones.
ahoka 3y ago
Then accept that UTC is a timezone and just use that everywhere. There are also a lot of things that can’t be easily stored in UTC, like opening hours.
- rtpg 3y ago
  Something like “9AM-5PM EST” can’t be stored in a datetime object even with a timezone slot! Datetimes with a timezone represent specific points in time, not vague “5PM” like concepts.
  - MrJohz 3y ago
    I feel like every time a discussion on timezones comes up, people compete to come up with different situations where "see, your perfect system can't work, we should give up on timezones entirely". Here, I'm not even sure what your point is. A datetime object cannot capture a range, no, just like a number can't capture a range of values. But through the magic of having two of them, we can get easily create a data structure containing two datetimes to represent a range. 5pm is not a vague concept, it is the time 5pm, the hour that typically comes after 4pm and before 6pm. If your point is that you can't store only a time in a datetime object, that's true, that's why the standard library also provides the `time` object which represents a time. Where things might get a bit more complex is if you want to store the time of a recurring event that should occur at the same time on multiple days for a given timezone. In this situation, you can typically use a naive time object to represent a wall clock time, plus the timezone that the user has requested. This way you still have all the raw information to decide when the event will happen in the future. (Note however, that the same time can occur multiple times on the same day, for example during daylight savings changes.)
    - rtpg 3y ago
      "every day, from 9AM to 5PM EST" is a thing that can be represented through calendar RRULEs. Usually libraries exist to help with this (Python has one, conveniently called rrule!). This is a thing to describe recurring events, has a spec and everything. So there is a thing. And you're totally right about `time` existing, it slipped my mind that time has a tzinfo slot. I do think you should still take care and I would avoid having any naive time objects just floating around beyond the serialization layer though.
divbzero 3y ago
If “everything is UTC, all the time” is enforced at the entry edges of the code, couldn’t the entry edges be refactored to always specify the UTC timezone?
lern_too_spel 3y ago
That doesn't work for recurring events in local time.

Buttons840 3y ago

How do you express "our stores, across 6 different timezones, all open at 9:30 AM local time" in UTC?

pelorat 3y ago
you don't, since they are different times?
- deleted 3y ago
  [deleted]
sillysaurusx 3y ago
That code runs in the context of humans. Therefore it needs to take humans into account when running. This is a bit like asking "how do you represent a string length in utf-8?" The answer is, you don’t. It’s almost never a problem in practice. Just like there is almost never a reason to care about the number of characters in a string, it’s very difficult to imagine that this code comes up in practice. But if it did, the code is simple: for a given timezone, is the current time in utc past 9:30am? If so, the store is open.
- Buttons840 3y ago
  Let's say you host a store front for many businesses and it's decided that your customers, who are businesses, some with many stores, want to store a default opening time. Something along the lines of "All Home Depot's open a 6 AM, all Lowes open at 7 AM."
  - wayfinder 3y ago
    That’s not date data though. That’s just storing local time - 6 AM or 7 AM.
  - sillysaurusx 3y ago
    The store property would be "6 AM". Anywhere it’s used, it’s running in the context of humans. When displaying on a website, you don’t need to do any conversions: the human viewing the store knows where it’s located, and they know the current time, so computers don’t need to answer the question of whether it’s open. If computers do need to know, then they’ll need to know a specific store with a physical location. That gives you the time zone.
    - macintux 3y ago
      > If computers do need to know, then they’ll need to know a specific store with a physical location. That gives you the time zone. Mapping a location to a time zone seems fraught with peril. I’d store the local time zone explicitly.
      - amluto 3y ago
        Indeed. And anyone near a time zone boundary looking at an online map of store locations doesn’t want to see “opens at 6AM” or whatever — they want to know whether the store is open.
      - JJMcJ 3y ago
        Indeed. It's not as bad as mapping address => sales tax (no ZIP Code isn't good enough), but only because it doesn't change as often and tends to follow county boundaries when it doesn't follow state boundaries. Another good example. Arizona doesn't follow Daylight Savings, but the Navajo Nation, which includes the Four Corners area and is in three states, does observe the change. However the Hopi reservation, completely surrounded by Navajo territory, is entirely in Arizona, and does not observe DST. So it can get quite complicated and it's not enough to say "Arizona doesn't have Daylight Savings Time".
- deleted 3y ago
  [deleted]
joshspankit 3y ago
If someone expresses it that way it will be a huge hassle the moment they decide one store opens at a different time.
- Buttons840 3y ago
  So maybe individual stores can customize their opening time, but corporate still wishes to store a default. How do you represent the default? How do you handle the DST time changes for individual stores? I'm trying to poke holes in the "always use UTC and you're good" advice. I have actually encountered this scenario, and I didn't know how to apply that advice.
  - bpfrh 3y ago
    I don't get your problem, couldn't you store it like this: table timetable: open | close | store 6am | 6am | default 7am | 12pm | store1 table stores: name | address store1 | xyx NYC If you can't find the store in the timetable: 1. Get the timezone of the target store by looking up the address from the table stores 2. Set Hours from the default row in the timetable table on the object with the timezone
  - Izkata 3y ago
    > I'm trying to poke holes in the "always use UTC and you're good" advice. There's a much simpler one - recurring meetings/events (in-person or local-ish like within one country) that cross a DST boundary. You always want these to be attached to a local time zone, so it doesn't suddenly happen an hour earlier/later than you intended.
    - fragmede 3y ago
      Where that gets even more fun is with international meetings, where the host of the meeting and the attendees are in different zones.
bofaGuy 3y ago
Not sure if this is a serious question. Each store will have an opening time, each will store the local time that the store opens.. just as utc.
d0mine 3y ago
utc is for a specific moment in time (deterministic with great precision for everyday use). "9:30am local time" may be ambiguous/unknown (the rules may not exist yet) Using the same programming type for both may cause confusion. How do you express geographic coordinates? Noon in London and in NY are different moments in time (it may be important if you want to have a remote meeting).
- samus 3y ago
  No need for something as brittle as geographic coordinates or addresses. Time Zone names are the right abstraction. A given location very rarely [edit: adopts another] time zone (unless in case of annexations, handovers of territory, etc.), but time zone rules (zone offset from UTC, DST, etc.) change all the time. A consequence of this is that the time zone database of applications should be updated every few months!
  - d0mine 3y ago
    "brittle" ? Your suggestion to define places by their timezone name is ok (it may work in many cases).
    - samus 3y ago
      "Complicated" then. One would have to consult a map to find out the time zone from geographical coordinates. The time zone is really everything needed to properly perform timezone-aware arithmetic.
Footkerchief 3y ago
Postgres stores this as TIMESTAMP WITHOUT TIME ZONE, sometimes called wall-clock time. IMO this is a good convention -- treating "ideal times" and specific epoch moments as two different types -- and more languages should support this (ideally with a less confusing label).
- kevindamm 3y ago
  Elixir does as well, with NaiveDatetime
dumah 3y ago
Tuples of the local time zone (not utc offset) and a time, along with a utc localization conditional on date.
JJMcJ 3y ago
Reference opening time to local time on each day of the year, and then refer that back to UTC. Like California is -8 in Standard, -7 in Daylight, so a 9:30 AM opening in the Winter is at 17:30 UTC, in the Summer, 16:30 UTC. It's a hassle but programmers are supposed to model reality. As an example of the problems that can arise, I remember when the new Japanese Emperor was to take office upon his father's abdication, there was a big fuss because the Era Name was customarily only revealed after the new Emperor's reign started but all the computer people wanted to be ready in advance. Not sure what finally happened, if it was revealed in advance or not.
- tsimionescu 3y ago
  What if the date for DST changes tomorrow, but only in some locations? How will you know which stores times need to be adjusted and which don't? UTC to local time is not a function, it is not somethung that can be computed once and stored for the future. The two are fundamentally different concepts, and can only be related to one another in certain contexts, and only reliably for the past.
  - JJMcJ 3y ago
    > How will you know which stores times need to be adjusted and which don't? There are a number of libraries for this, usually updated by the unpaid volunteer in Nebraska. Or else your company has staff whose job it is to follow this for any place you operate. If you record opening times as UTC, well, then you have to change to UTC when the store switches standard <=> daylight. This is what Fred Brooks called an essential complexity. You can poke the pain from one part of your system to another part, but it's never going away. The function is complicated. There is no avoiding that. One Pacific island nation duplicated a calendar day when it switched International Dateline sides to more closely align with Australia instead of Hawaii, reflecting a shift in its economic ties. No idea how they handled it exactly but pretty hairy to model.
konschubert 3y ago
store_opening_hour_and_minute_of_day=(9,0) It’s not a date time.
remram 3y ago
"9:30 AM every day" is not a datetime. It's a time.

smooc 3y ago

Financial data deals with cutoff times and those happen in a timezone. Thus particularly in finance timezones are important. You are right though that internally everything should be UTC internally and only when required it gets transposed to the timezone. However many legacy financial systems assume a local time zone and not UTC and they do not expose that information in their data. Happy reconciliation :-).

steveBK123 3y ago
One edge case to this is I've worked on financial systems where we kept the Asian market systems in JST, while EU/MENA/Americas ran on UTC. As long as you aren't dealing with FX/Futures, markets are generally open no earlier than 7am and no later than 6pm. This worked nicely since it meant that the DATE in each regions system&local clocks agreed.
midasuni 3y ago
If the market opens at 9am New York time then that’s not the same as storing 1400 utc. An law could be passed to change the offset on a given day You have to store time in the relevant geographic time zone. Something occurring at 1200UTC on dec 1st is not the same as something occurring at 1200Europe/London on dec 1st. The two times may coincide today, but if the U.K. government decides to implement daylight saving next week as a fuel rationing measure, then your 1200London event will have to be displayed at 1300UTC, but your 1200UTC event would be at 1200UTC.
- Kamq 3y ago
  I think you mean 1300GMT, I don't think the UK owns enough of the world these days to futz with UTC.
  - midasuni 3y ago
    1200 London time, so 1100UTC with daylight saving yes. Point remains, many events are linked to local civil time, which can change - and not just to daylight saving. The Line Islands in Kiribati shifted a whole day in 1994, same with Samoa in 2011. Have an event set for 0800 local on the first Monday of the year in Samoa which you stored as UTC, and your events would have broken. Unless you store the relevant data (it occurs on first monday according to the timezone that Samoa is in), then you will be wrong. Now of course you can still get problems if a new timezone is created, as time is stored as Europe/London, if Scotland creates its own timezone in the future a meeting in Aberdeen would no longer be in Europe/London but in a new area Europe/Edinburgh, with different behaviour, but that’s far less common than DST changes or other date shifts. All in all you should store the relevant datetime and zone, not just “normalise” it to UTC.

1letterunixname 3y ago

Using UTC as a universal time standard is stupid because it contains leap-seconds while TAI does not. TAI64* should be the preferred computational monotonic time format.

Spivak 3y ago

You still keep everything in UTC in this new world, it's just the object you use internally now knows that it's UTC and makes it so you can't accidentally do calculations on two naive datetimes which happen to be in different timezones. You get to have the typing help you enforce that everything is in UTC.

konschubert 3y ago
And you have to treat the timezone offset as information-free. As in, 02:00+00:00 and 04:00+02:00 are absolutely equivalent representations of the same data. If you need to know the timezone in which to display a time, you save that timezone separately

plibither8 3y ago

> India has half-hour timezones If that's messy, consider Nepal's timezone that's on the 45-minute mark (UTC+5:45).

hprotagonist 3y ago
or one island of new zealand (chatham), which is +12:45
- slyall 3y ago
  The Chatham Islands are an Island group, two of which are inhabited (Chatham Island and Pitt Island). https://en.wikipedia.org/wiki/Chatham_Islands https://en.wikipedia.org/wiki/Chatham_Islands Right now is it on +13:45 due to Daylight Saving https://en.wikipedia.org/wiki/Chatham_Standard_Time_Zone https://en.wikipedia.org/wiki/Chatham_Standard_Time_Zone
- scheme271 3y ago
  Nepal is UTC+5:45. I think that's the largest population on a 15/45 minute offset.
  - physicles 3y ago
    Fun fact: there are currently no time zones with 15 minute offsets.
    - iso8859-1 3y ago
      Why does Nepal not count as one?
- angrygoat 3y ago
  or along the highway near the Western Australia/South Australia border, which informally uses +8:45 https://en.wikipedia.org/wiki/UTC%2B08:45 https://en.wikipedia.org/wiki/UTC%2B08:45
  - 3y ago

Hamuko 3y ago

I'd say that wouldn't enforcement be rather simple going forwards since you could just enforce that dt.tzinfo == datetime.timezone.utc, but as far as I know, timezone information checking in Python can be a bit of a pain. >>> utc1 = datetime.timezone.utc >>> utc2 = zoneinfo.ZoneInfo("UTC") >>> utc1 == utc2 False >>> dt1 = datetime.datetime.now(utc1) >>> dt2 = datetime.datetime.now(utc2) >>> dt1.tzinfo == dt2.tzinfo False

maxerickson 3y ago

They aren't being removed, they will just toss up a warning.

tripflag 3y ago
The warning is also opt-in; you'll have to run python with PYTHONWARNINGS=always or similar to see it. Which is scary, because you will be taking the performance hit from generating the warning regardless, which can easily amount to 30% of your total CPU time if you use utcnow/utcfromtimestamp a lot.
tumult 3y ago
They will be removed: datetime.datetime’s utcnow() and utcfromtimestamp() are deprecated and will be removed in a future version.
- jagged-chisel 3y ago
  Aren’t now, “will be” in a future version. So the can will be kicked down the road as many generations of devs as needed until no one cares any longer.
  - tumult 3y ago
    So, they remove it in 12 months and your code wasn't ready for the change, and people mock you for having not been prepared by updating your code. The Python people have told you to remove it, so nobody will have any sympathy for you if you don't do it. As you demonstrated, people are sticklers for quoting exact wording when it comes to criticizing others.
    - Hamuko 3y ago
      Has anything in Python been removed in 12 months? TestCase.assertEquals was deprecated in Python 3.2 (February 2011) and was finally removed in Python 3.12 (October 2023). And that was 12 years for just a simple rename from "assertEquals" to "assertEqual".

jagged-chisel 3y ago

For sharing between systems and representing dates to humans debugging a system, the following has worked well … Planned local events: future date and time string including local time zone Planned global events: future date and time string in UTC, with UTC timezone Past events: date and time string in UTC with UTC time zone, unless there is a strong case to be made that these should be localized (see “planned local events” above) Strings in the format specified by ISO-8601. Of course, internal representation can be whatever it needs to be for efficiency. For example, maybe your database stores as integer microseconds since an epoch. Fine, but as soon as that leaves the database, ISO-8601 that date.

theK 3y ago
I like your premise but disagree with your guidance. You should not convert the integer early, you should actually wait until the very last moment. Nowadays a timestamp will be marshalled and demashalled by (at the very least) two runtimes until it reaches human eyeballs opening up a wide spectrum of bug potential, difficult to understand code and development slowdowns. Keeping it simple and unambiguous is very important. Time libraries have a habit of coming and going, ints are always there and 99% of the date arithmetic is trivial any way. Don't use a lib, just pretty print numbers and you'll be fine.
- fluoridation 3y ago
  An int is not unambiguous, though. "2023-11-19T14:22:34-03:00" is clearly a timestamp and represents an unambiguous point in time, while a number by itself doesn't mean anything. You have to know that it's a timestamp (probably from context) and you have to know what timezone it's in.
  - Dylan16807 3y ago
    > You have to know that it's a timestamp So what? That's true of all formats. How is it relevant to the choice of which format to use at which stage of the process? > and you have to know what timezone it's in. You don't. > clearly a timestamp Not very helpful. It's easy to set up types in advance. Don't figure out if things look like timestamps at parse time, or you'll be causing the same problems that turn gene names into dates and choke on people named Null.

albert_e 3y ago

> India has half-hour timezones Not plural though; The whole of India -- even though it is pretty vast in area -- has a single time-zone country-wide. I am very thankful for this. We already have a lot of other challenges -- including dozens of different languages -- that a single time zone is a relief. But yes -- the +5:30 offset usually makes for a less-than-ideal factor to plan meetings or follow non local event schedules. Did I mention we also hate the Summertime / Daylight Saving time adjustments we have to accommodate twice a year when working with western teams -- more adjustment even after most people have already sacrificed on a sane 9-5 schedule to ensure they overlap with EU/US teams for daily work meetings. We cant imagine why US that has multiple time-zones already in a single country, and EU whose countries are often smaller than states in India ... need so much adjustment. Why can't they, for instance, simply declare that school/offices open at 10am instead of 9am for six designated months every year instead of imposing this adjustment on the entire world.

steveBK123 3y ago
Brazil's stock market does/used to do something along these lines when US & Brazil changed their clocks in opposite direction, to keep the NY time hours of the Brazilian exchange from drifting as far. Given that we all also observed DST at the time, this made things even more complicated for systems.
screye 3y ago
Having lived in both India and far North in the USA, I'd take half-hour time zones over daylight savings (switches) every time. Shifting time-zones increases car accidents by about 6-16% for the entire week (studies vary). Just that is a massive enough impact on safety and productivity to discourage the switch. At a personal level, leaving work at 5.30-ish with it being pitch black outside, is among the most depressing things in the world. Even more wasteful is that the average American wakes up at 7.20am. So, they don't gain any daylight but lose an entire hour. I'm not usually the type to make such accusations, so I apologize in advance for sounding culture-warry. But, switching-time zones is "white person thing". Large percent of the Chinese, Indian & the Middle Eastern population live above the 25th parallel (Florida) and large populations of Korea & Japan live north of the Bay Area without daylight-switching. Permanent daylight time with later sunrise would be a huge win.

master_crab 3y ago

It definitely will break financial firms. I just did a code search in our repos for utcnow and other relevant functions, and the indexer was definitely wheezing pulling in the hits.

andrewprock 3y ago
Deprecation should not break anything. At some point the code will be removed and the org will have to pin the version until the deprecations are resolved.

StableAlkyne 3y ago

> I suspect a lot of financial data code is going to break. Reminds me of when Numpy removed a lot of types like np.bool a few releases ago, because they were aliases of the standard library types. Which broke a ton of libraries having Numpy versions pinned to ">=1.0" or even worse, "~=1.0" expecting them to follow the standard practice of incrementing the major version when they break compatibility. It's not terrible because you just pin your project to 1.23 or whatever, but it was very annoying when everything I was using just started breaking. I wonder if Python plans to do the same with this - the 2 to 3 transition was painful and dragged on for a decade, nobody wants to do that again for 3 to 4. When they remove this function and break back compatibility with old code, is that going to happen in a future 3.x release?

hexane360 3y ago
It's worth noting that numpy and scipy have a consistent policy now, although it's not semver. Deprecations will always happen at least 2 minor versions before removal: https://numpy.org/neps/nep-0023-backwards-compatibility.html https://numpy.org/neps/nep-0023-backwards-compatibility.html https://numpy.org/doc/stable/dev/depending_on_numpy.html#runtime-dependency-version-ranges https://numpy.org/doc/stable/dev/depending_on_numpy.html#run... So, when using numpy in a package, and you've tested e.g. versions 1.12 to 1.22, you should require 'numpy>=1.12,<1.25', because something deprecated in version 1.23 could be removed in 1.25.
jkrejcha 3y ago
Yes, they're almost certainly going to do it a 3.x release. I think in some senses these backward compatibility breaks have already happened. I remember that Google got bit by async becoming a keyword in I think Tensorflow. Generally a lot of old Python 3 code still works today if it doesn't do anything really weird like that, but I do definitely think that the pain of Python 2 to 3 was painful so they don't really want to repeat it. I think it is probably prudent to treat point releases as major releases at least with regards to Python.

notatoad 3y ago

Removing utcnow() doesn't force you to use tz-aware dates though. That's just a leap that the author of this article has made. If you're working in a system where all dates are utc, then datetime.now() gets you a date in utc and you don't need datetime.utcnow(). The only reason to call utcnow is if you're working on a system where the local timezone isn't utc, so you inherently have to deal with timezones at some level. Which means that using naive timestamps is a bad idea.

nerdponx 3y ago
datetime.now(tz=timezone.utc) is what you probably want most of the time anyway. You can always manually strip off the time zone component if you really want to use datetimes without associated time zones.

ahoka 3y ago

Then accept that UTC is a timezone and just use that everywhere. There are also a lot of things that can’t be easily stored in UTC, like opening hours.

rtpg 3y ago
Something like “9AM-5PM EST” can’t be stored in a datetime object even with a timezone slot! Datetimes with a timezone represent specific points in time, not vague “5PM” like concepts.
- MrJohz 3y ago
  I feel like every time a discussion on timezones comes up, people compete to come up with different situations where "see, your perfect system can't work, we should give up on timezones entirely". Here, I'm not even sure what your point is. A datetime object cannot capture a range, no, just like a number can't capture a range of values. But through the magic of having two of them, we can get easily create a data structure containing two datetimes to represent a range. 5pm is not a vague concept, it is the time 5pm, the hour that typically comes after 4pm and before 6pm. If your point is that you can't store only a time in a datetime object, that's true, that's why the standard library also provides the `time` object which represents a time. Where things might get a bit more complex is if you want to store the time of a recurring event that should occur at the same time on multiple days for a given timezone. In this situation, you can typically use a naive time object to represent a wall clock time, plus the timezone that the user has requested. This way you still have all the raw information to decide when the event will happen in the future. (Note however, that the same time can occur multiple times on the same day, for example during daylight savings changes.)
  - rtpg 3y ago
    "every day, from 9AM to 5PM EST" is a thing that can be represented through calendar RRULEs. Usually libraries exist to help with this (Python has one, conveniently called rrule!). This is a thing to describe recurring events, has a spec and everything. So there is a thing. And you're totally right about `time` existing, it slipped my mind that time has a tzinfo slot. I do think you should still take care and I would avoid having any naive time objects just floating around beyond the serialization layer though.

divbzero 3y ago

If “everything is UTC, all the time” is enforced at the entry edges of the code, couldn’t the entry edges be refactored to always specify the UTC timezone?

lern_too_spel 3y ago

That doesn't work for recurring events in local time.