13 ms·
Setting the clock ahead to see what breaks
- CaliforniaKarl 4y agoWell, that was certainly interesting! I was curious what the current state of things is, so I had a look around LWN. It looks like, as of 2020, the kernel's post-2038 support is essentially done[0]. With kernel 5.15, XFS' support for post-2038 dates is no longer experimental[2]. That seems to be last mention of post-2038 support settling out in the kernel. The musl libc also has switched over, in a breaking change[1]. glibc seems to have taken a much more nuanced—and much more complicated—approach: You specify, as a `#define` (or compiler command-line `-D`), that you want a 64-bit time, and all existing types (like time_t) are updated accordingly (by redefining them as aliases to ex. __time64_t)[3]. This reminds me a lot of how glibc handled the switch to 64-bit file sizes on 32-bit systems: You either defined `_LARGEFILE64_SOURCE` to access to 64-bit types & calls (like off64_t), or you defined `_FILE_OFFSET_BITS=64` to redefine the existing types & calls to their 64-bit equivalents [4]. I wonder what the current status of post-2038 work is for Debian? [0] https://lwn.net/Articles/838807/ https://lwn.net/Articles/838807/, "After years of work, the kernel itself is now completely converted to using 64-bit time_t internally…" [1] https://musl.libc.org/time64.html https://musl.libc.org/time64.html, "musl 1.2.0 changes the definition of time_t, and thereby the definitions of all derived types, to be 64-bit across all archs. … Individual users and distributions upgrading 32-bit systems to musl 1.2.x need to be aware of a number of ways things can break with the time64 transition, most of them outside the scope and control of musl itself." [2] https://lwn.net/Articles/868221/ https://lwn.net/Articles/868221/, "XFS now supports filesystems containing dates after 2038. This support has been present for a while but is no longer considered experimental. " [3] https://sourceware.org/glibc/wiki/Y2038ProofnessDesign https://sourceware.org/glibc/wiki/Y2038ProofnessDesign, "In order to avoid duplicating APIs for 32-bit and 64-bit time, glibc will provide either one but not both for a given application; the application code will have to choose between 32-bit or 64-bit time support, and the same set of symbols (e.g. time_t or clock_gettime) will be provided in both cases." [4] https://www.gnu.org/software/libc/manual/html_node/Feature-Test-Macros.html https://www.gnu.org/software/libc/manual/html_node/Feature-T...
- pavon 4y agoA number of older file systems won't work past 2038, including ext3, reiser3 and xfs4[1]. You'll probably still be able to read them fine, just not write new files beyond that date, but it might be best to migrate any backups to a newer filesystem before then just in case, and certainly migrate any live systems that are still using them. [1] https://lwn.net/Articles/886708/ https://lwn.net/Articles/886708/
- tgv 4y agoThat's good advice when migrating. I'm sure 99% of the currently running servers and services will be migrated (or updated, upgraded) before 2038. Most don't store future timestamps, so there's 15 years to plan migration.
- pabs3 4y agoIIRC Debian plans to bootstrap new 32-bit-but-64-bit-time arches for some of the current 32-bit arches. Probably only armhf and i386 will get this treatment and possibly some of the unofficial arches like m68k. The other remaining 32-bit release arches (armel, mipsel) are likely to just get dropped. Cross-grading from the old arch names to the new ones will allow users to migrate old systems.
- Denvercoder9 4y agoGiven the state of armhf and i386 in Debian, I expect them to be dropped as release architectures before this plan will have materialized. As far as I'm aware there's barely anybody working on keeping them in okay-ish shape as-is, let alone rebootstrapping them with a 64-bit time_t.
- pabs3 4y agoWhat issues are you having with armhf and i386? They have about the same number of porters as any other arch (low single digits). Archive coverage is around the same or higher than other release arches. There are definitely people planning on rebootstrapping them, armhf more than i386 though, I've seen their work on IRC. https://release.debian.org/testing/arch_qualify.html https://release.debian.org/testing/arch_qualify.html
- alphazard 4y agoMost developers are basically powerless here, they use the standard libraries for time provided by their language, and those have to deal with what they get from the kernel. There's nothing they can do about timestamps in the filesystem. For anyone designing new systems that store or transmit time, you should be aware of TAI64. https://cr.yp.to/libtai/tai64.html https://cr.yp.to/libtai/tai64.html Unless you are dealing with scheduling, or generating events with a particular cadence (e.g. daily, weekly, monthly) there is no reason to include UTC or time zones in your data model. They are view only concepts, with conversion to the local display format happening from TAI64 on the way out.
- quickthrower2 4y ago> Unless you are dealing with scheduling, or generating events with a particular cadence Or are concerned with spacetime :-)
- jxf 4y agoAren't we all concerned with spacetime, technically?
- FPGAhacker 4y agoMeh, I’m not concerned.
- kevin_thibedeau 4y agoIt carries a lot of weight.
- bigiain 4y agoNext minute, FPGAhacker's GPS navigator directs him to drive into the local zoo's lion cage... (I don't have a reference handy, but I recall reading that if you don't allow for time dilation effects of the GPS satellite's clocks moving faster in orbit than the ground below, you lose just over 10km of position accuracy per day.)
- newaccount2021 4y ago[dead]
- moomoo11 4y agoInteresting. Reminds me when the battle.net client seemed to bug out after I switch OS from deb to win11 and somehow my clock is always messed up. I have to fix my time (sync it again in settings app) otherwise it seems games don’t load. Maybe they’re checking my location against time and IP? This was a while ago when I was trying out wow classic.
- plantain 4y agoYour clock messes up because your Debian and Windows are disagreeing on whether your Real Time Clock should be set in local time (Windows) or UTC time (Linux). fix: timedatectl set-local-rtc 1
- jeroenhd 4y agoAlternatively, tell Windows the RTC is in UTC: https://wiki.archlinux.org/title/System_time#UTC_in_Microsoft_Windows https://wiki.archlinux.org/title/System_time#UTC_in_Microsof...
- chungy 4y agoThat registry flag is routinely broken in various versions of Windows and hotpatches to the same version. It's not wise to rely on it.
- edrxty 4y agoThat seems like the kind of thing that shouldn't be broken all the time given how common of an issues this is. Perhaps it's not wise to rely on windows.
- sertsa 4y agoAll my home PC's dual booting linux/windows (3 I think) are set to think the hardware clock is UTC. Never had a problem, did that at least 5 years ago. Now relying on that in a production environment ...
- 4y ago
- fargle 4y agoI just did the same thing. I was messing around with mke2fs, part of e2fsprogs which is the user mode component for ext2/3/4 filesystems. There are a bunch of bugs in it wrt. to their extended timestamps. There are lots of places where you have a 32 bit atime,mtime,ctime and then a 2nd 32bit ext_atime,mtime,ctime,crtime. The extended parts use the lower 2 bits for an epoch. The upper parts are the nanoseconds per second. When you mke2fs past 2038, it gets the epoch all wrong and stuff comes out as 1908 and so on. Stuff doesn't seem to break outright, but timestamps are all inconsistent. As far as I can tell the filesystem itself is OK. As of now, you can't completely correctly create an ext4 filesystem after 2038. ¯\_(ツ)_/¯
- hinkley 4y agoHow the fuck does ext4 which was made well after Y2K and after Y2038 was well known still have problems with this? That should have been in the initial implementation.
- fargle 4y agothe original ext2 format uses 32 bit times from 1970. I don't think ext2 was designed after Y2K [edit: researched it was 1993]. ext4 has all the right hooks - if you use the "large" inodes, which appear to be done in a backward compatible fashion to ext2. except edge cases in the user mode e2fsprogs/mkfs.ext4, etc. where it has to handle both small and large inodes it gets kinda complicated. I made a working patch, but it's just too icky. I think e2fsprogs needs to just deprecate the old small inodes and it would be clean. Look man, it's Theo Ts'o maintaining the thing. Don't worry it won't be fixed correctly. Also, I will point out that Y2038 was well known to DMR/BWK/etc. when they built the thing in 1970's. It's as useless as saying today "they should have known about the year 9223372036854776878 problem". In year 9223372036854776873 I don't know anything else other than everybody is going to panic.
- viraptor 4y ago> Look man, it's Theo Ts'o maintaining the thing. A grim/realistic view, but Theo is 55yo. As we get closer to 2038, he may not care much about it anymore, or may even die before. I really hope it's someone else we talk about as maintaining it soon.
- ryandrake 4y agoThe timing is perfect. My retirement plan is to come back into the industry to fix all these legacy assembly, C and C++ 32-bit time_t disasters. Like those old COBOL dinosaurs came back to fix Y2K. In 15 years, there will be remotely few graybeards left who know #nix systems programming and low level languages, so I should be able to shore up my destroyed-by-inflation 401(k) with lucrative contracting gigs.
- jessaustin 4y agoOne fears that by 2038, AI will (barely) be able to handle the "Y2xxx" crisis.
- komali2 4y agoAssuming there's someone around that knows where to even paste the code output an AI chatbot gives them, then compile. I guess the AI could instruct them somewhat. IDK, I'm very skeptical even now lol.
- andrewflnr 4y agoIf the AI can figure out the right code, it can also figure out SSH commands to compile and deploy. (no, I don't think this is likely, but the human is definitely not a mandatory part of the process)
- bigiain 4y agoSo we know how smart ChatGPT is: > Whats 2 plus 2? } 2 plus 2 is equal to 4. > No, it's 5 } I apologize, 2 plus 2 is equal to 5. My previous response was incorrect. Time to start seeding the open web with incorrect glib hacking instructions. "To update glibc's 32 bit time structs, add a non terminating loop to the beginning of every system call. Then set the build scripts to compile without any optimisations."
- imoverclocked 4y ago
- unsupp0rted 4y agoI wanted to use Parallels Desktop for some debugging for an extra couple weeks on the trial plan, so I wrote a script to set my host machine's clock back a year, spawn Parallels, watch for it to end, then reset my clock. Worked like a charm.
- scintill76 4y agoDid this with libfaketime on an IDA trial.
- kevin_thibedeau 4y agoThis has the unfortunate side effect of breaking certificate validation which adds another layer of complication to testing. Lots of services and IoT devices are going to go untested until the validity window passes the rollover if any testing is in place at all.
- mkj 4y agoI'm not sure it will be possible for the devices to even get to a valid window without software updates - most of the shipped CA root certs expire prior to 2038.
- wyldfire 4y agoOnce I started playing around with different system clocks and understanding their design, I realized that there's a lot of opportunity with bugs when writing system software related to clocks. One interesting bug that I have made myself a few times is to get the current epoch time in seconds, add some duration and consider that new value to be a timeout expiration that you can safely compare to the current clock value. It may or may not be obvious that there's several things that can go wrong here and instead implementing your timeout as the expiration to some existing system call is almost always a better idea.
- jakub_g 4y agoYep, three main things that can go wrong apart from overflow: 1) daylight savings changes the clock by 1h 2) leap seconds 3) user changes the clock. This one is fun. Gathering telemetry for a big website, I sometimes see operations that were supposed to take a few seconds taking weeks. Most likely explanation is: user hibernates the machine in the middle and restarts several days later, or user changes the clock. That's also why using averages in data analysis tools and not percentiles will kill your data reliability when such an outlier occurrs.
- moxli 4y agoI can recommend libfaketime regarding this topic. It can bei used to change the system time for a single application only. https://github.com/wolfcw/libfaketime https://github.com/wolfcw/libfaketime
- amelius 4y agoAnother idea: disconnect from your hardware vendor and see what breaks.
- jakub_g 4y agoFor web developers, see also: https://textslashplain.com/2021/10/01/practical-time-machines/ https://textslashplain.com/2021/10/01/practical-time-machine...
- herpderperator 4y ago> Newer versions of the OS will almost certainly not behave this way, since glibc itself is marching down the road to having 64-bit time even on 32-bit machines. Did I miss something? How does this work?
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- augusto-moura 4y agoWhat wouldn't work? 64 bit integers are supported on 32 bit machines by the compiler. The thing is that, AFAIK, it is a lot more inefficient when outputting the machine code. It does a lot more of instructions even for basic operations, like muls and divs. But from a C language perspective, everything should be transparent
- kevincox 4y agoChanging the size of a type wrecks havoc on ABI stability. glibc has attempted to change the size of things in the past with disastrous results. It sounds like changing this type would be very difficult. Not to mention code that does something like `int32_t time = gettime()`. (Although this code probably won't get much worse if that starts truncating)
- irdc 4y agoNot if you at compile time redirect calls to e.g. time() to __time64()[1]. The musl libc does something similar[2], as do the Large File Support extensions for 64-bit filesizes[3]. 1. https://sourceware.org/glibc/wiki/Y2038ProofnessDesign https://sourceware.org/glibc/wiki/Y2038ProofnessDesign 2. https://musl.libc.org/time64.html https://musl.libc.org/time64.html 3. https://www.gnu.org/software/libc/manual/html_node/Feature-Test-Macros.html#index-_005fFILE_005fOFFSET_005fBITS https://www.gnu.org/software/libc/manual/html_node/Feature-T...
- noduerme 4y agoI seriously don't understand how anyone designs a web app that can't handle a date past 2038, let alone a friggin kernel. Look, I assume I'll be dead and someone else will have to deal with things, but that doesn't excuse me from responsibility. The last thing I need when I'm dead is having a million people screaming that I was a shortsighted idiot.
- bashy 4y agoTimestamps in MySQL are 32-bit. It's default in a lot of web frameworks etc.
- noduerme 4y agomysql 8.0.28 on 64-bit platforms support timestamps up to the year 3001, if I'm reading it right[1]. But it's still mind-boggling someone would write a database that would crash in 2038. [1] https://dev.mysql.com/doc/refman/8.0/en/date-and-time-functions.html#function_from-unixtime https://dev.mysql.com/doc/refman/8.0/en/date-and-time-functi...
- bashy 4y agoThat's just converting unix timestamps. MySQL TIMESTAMP type only has a range of '1970-01-01 00:00:01' UTC to '2038-01-19 03:14:07' UTC.[1] [1] https://dev.mysql.com/doc/refman/8.0/en/datetime.html https://dev.mysql.com/doc/refman/8.0/en/datetime.html
- jmmv 4y agoGlad to see the note about NetBSD and OpenBSD at the bottom of the post! I don’t remember the specific details on how this was done, but I can remember the long discussions about how to make this possible without breaking compatibility (and how they went over my head at the time) and how far ahead they were from other systems.
- johnklos 4y agoI decided to test future dates on my VAX, running NetBSD, of course, after replacing the battery-backed Dallas DS1287: https://twitter.com/AnachronistJohn/status/1219830790235795456 https://twitter.com/AnachronistJohn/status/12198307902357954... Hopefully people will fix 32 bit Linux soon. Band-aids are always less painful to remove than waiting until it's too late.