7 ms·

I just did the same thing. I was messing around with mke2fs, part of e2fsprogs which is the user mode component for ext2/3/4 filesystems. There are a bunch of

by fargle 4y ago

I just did the same thing. I was messing around with mke2fs, part of e2fsprogs which is the user mode component for ext2/3/4 filesystems. There are a bunch of bugs in it wrt. to their extended timestamps. There are lots of places where you have a 32 bit atime,mtime,ctime and then a 2nd 32bit ext_atime,mtime,ctime,crtime. The extended parts use the lower 2 bits for an epoch. The upper parts are the nanoseconds per second. When you mke2fs past 2038, it gets the epoch all wrong and stuff comes out as 1908 and so on. Stuff doesn't seem to break outright, but timestamps are all inconsistent. As far as I can tell the filesystem itself is OK. As of now, you can't completely correctly create an ext4 filesystem after 2038. ¯\_(ツ)_/¯

hinkley 4y ago
How the fuck does ext4 which was made well after Y2K and after Y2038 was well known still have problems with this? That should have been in the initial implementation.
- fargle 4y ago
  the original ext2 format uses 32 bit times from 1970. I don't think ext2 was designed after Y2K [edit: researched it was 1993]. ext4 has all the right hooks - if you use the "large" inodes, which appear to be done in a backward compatible fashion to ext2. except edge cases in the user mode e2fsprogs/mkfs.ext4, etc. where it has to handle both small and large inodes it gets kinda complicated. I made a working patch, but it's just too icky. I think e2fsprogs needs to just deprecate the old small inodes and it would be clean. Look man, it's Theo Ts'o maintaining the thing. Don't worry it won't be fixed correctly. Also, I will point out that Y2038 was well known to DMR/BWK/etc. when they built the thing in 1970's. It's as useless as saying today "they should have known about the year 9223372036854776878 problem". In year 9223372036854776873 I don't know anything else other than everybody is going to panic.
  - viraptor 4y ago
    > Look man, it's Theo Ts'o maintaining the thing. A grim/realistic view, but Theo is 55yo. As we get closer to 2038, he may not care much about it anymore, or may even die before. I really hope it's someone else we talk about as maintaining it soon.
    - bigiain 4y ago
      I worry about any technology that has a bus factor of one.[1] And I'm sure there are several orders of magnitude more of them than _that_ xkcd cartoon imply... [1] Except when that "one" is me. You're all welcome to solve any problems in code I leave behind, I no longer will care. Whether that's a "bus factor" or a "won the lottery factor".)
      - fargle 4y ago
        agree with [1], been there. But projects that have a bus-factor ~1 are often some of the tightest, best ones. So ¯\_(ツ)_/¯, don't worry so much. Hopefully the owner leaves some keys and contingency plans, but otherwise, carry on as normal. The GPL and other open source licenses are good enough backup insurance if they didn't.
    - fargle 4y ago
      LOL, 55 years old is nearly dead? Ha! Knuth is 85 and still active. Kernighan is 81, likewise. Torvalds is 53 and he's just starting to mellow out and grow up (a little, not too much). Anyone can get hit by a bus at any age. So the age at which you stop being productive or kick it is highly individualize. I don't know Theo Ts'o, but I have no evidence of his eminent demise.
      - viraptor 4y ago
        I'm not saying that 55 is nearly dead. But 55-65 is the range when the death rate starts to really increase https://www.statista.com/statistics/241572/death-rate-by-age-and-sex-in-the-us/ https://www.statista.com/statistics/241572/death-rate-by-age... 66 is also the retirement age and while some people want to keep doing what they're doing forever, others may prefer to move into a log cabin away from civilisation. So there's a good chance he'll be around, able, and dealing with the 2038 changes, but also a reasonable change that he won't want to / be able to.
- chungy 4y ago
  ext4 isn't really a new file system, it's effectively ext2 with some more features enabled by default than you get with mkfs.ext2. (Yes, this is a simplified version, but not grossly exaggerated.) ext2 was first released in 1993 and has had impressive work on being extended far enough to keep up with growing storage needs, but I suspect the original designers figured it'd have been tossed aside well before 2038 would pose a problem. Unfortunately that assumption has proven to be wrong (unless maybe we do junk ext[24] within 15 years' time, but it would be extraordinary unlikely that every last machine running it will do so).
  - semi-extrinsic 4y ago
    Yeah, I'm pretty sure that the people who wrote ext4 assumed there would be ext5 (or something else) to take over way before 2038. If you look at the history, ext2 was released in '93, ext3 was released in '01 and ext4 in '08. If you told the developers in '08 that ext4 would not be replaced in 30 years time, they would laugh at you (or cry).
    - chungy 4y ago
      That much is basically true too. It was yet another update to the tried-and-true format to pave over storage needs until btrfs stabilized.
- josephg 4y ago
  > How the fuck does ext4 [...] still have problems with this? I'm increasingly realising that most of the code out there isn't tested anywhere near as well as we think it is. Most programmers stop when the feature works, not when the feature is bulletproof. I've been writing a database storage engine recently. It should be well behaved even in the event of sudden power loss. I'm doing "the obvious thing" to test it - and making a fake, in-memory filesystem which is configured to randomly fail sometimes when write/fsync commands are issued (leaving a spec-compliant mess). I can't be the first person to try this, but it feels like I'm walking over virgin ground. I can't find a clear definition of what guarantees modern block devices provide, either directly or via linux syscalls. And I can't find any rust crates for doing this programatically. And googling it, it looks like many large "professional" databases misunderstood all this and used fsync wrong until recently. Its like there's a tiny corner of software that works reliably. And then everything else - which breaks utterly when the clock is set to 2038 because there aren't any tests and nobody tried it. I half remember a quote from Carmack after he ran some new analysis tools on the old quake source code. He said that realising how many bugs there are in modern software, he's amazed that computers boot at all.
  - quanticle 4y ago
    Dan Luu has a post (https://danluu.com/filesystem-errors/ https://danluu.com/filesystem-errors/), which covers some of the same ground, and links to papers with more information on the failure modes of file systems in the face of errors from the underlying block device. Prabhakaran, et. al. (https://research.cs.wisc.edu/wind/Publications/iron-sosp05.pdf https://research.cs.wisc.edu/wind/Publications/iron-sosp05.p...), did a bunch of filesystem testing (in 2005!), and their paper includes discussion on how to generate "realistic" filesystem errors, as well as discussion of how the then state-of-the-art filesystems (ext3, Reiser (!), and JFS) perform in the face of these errors. I'm unaware of any research newer than Dan Luu's post on filesystem error handling.
    - db48x 4y ago
      I keep that link handy too! I wish there were newer research to quote, but I also don’t want to force anyone to do that job. It must be pretty depressing to rip all the bandaids off and really contemplate how bad the situation is.

by fargle 4y ago

hinkley 4y ago

How the fuck does ext4 which was made well after Y2K and after Y2038 was well known still have problems with this? That should have been in the initial implementation.

fargle 4y ago
the original ext2 format uses 32 bit times from 1970. I don't think ext2 was designed after Y2K [edit: researched it was 1993]. ext4 has all the right hooks - if you use the "large" inodes, which appear to be done in a backward compatible fashion to ext2. except edge cases in the user mode e2fsprogs/mkfs.ext4, etc. where it has to handle both small and large inodes it gets kinda complicated. I made a working patch, but it's just too icky. I think e2fsprogs needs to just deprecate the old small inodes and it would be clean. Look man, it's Theo Ts'o maintaining the thing. Don't worry it won't be fixed correctly. Also, I will point out that Y2038 was well known to DMR/BWK/etc. when they built the thing in 1970's. It's as useless as saying today "they should have known about the year 9223372036854776878 problem". In year 9223372036854776873 I don't know anything else other than everybody is going to panic.
- viraptor 4y ago
  > Look man, it's Theo Ts'o maintaining the thing. A grim/realistic view, but Theo is 55yo. As we get closer to 2038, he may not care much about it anymore, or may even die before. I really hope it's someone else we talk about as maintaining it soon.
  - bigiain 4y ago
    I worry about any technology that has a bus factor of one.[1] And I'm sure there are several orders of magnitude more of them than _that_ xkcd cartoon imply... [1] Except when that "one" is me. You're all welcome to solve any problems in code I leave behind, I no longer will care. Whether that's a "bus factor" or a "won the lottery factor".)
    - fargle 4y ago
      agree with [1], been there. But projects that have a bus-factor ~1 are often some of the tightest, best ones. So ¯\_(ツ)_/¯, don't worry so much. Hopefully the owner leaves some keys and contingency plans, but otherwise, carry on as normal. The GPL and other open source licenses are good enough backup insurance if they didn't.
  - fargle 4y ago
    LOL, 55 years old is nearly dead? Ha! Knuth is 85 and still active. Kernighan is 81, likewise. Torvalds is 53 and he's just starting to mellow out and grow up (a little, not too much). Anyone can get hit by a bus at any age. So the age at which you stop being productive or kick it is highly individualize. I don't know Theo Ts'o, but I have no evidence of his eminent demise.
    - viraptor 4y ago
      I'm not saying that 55 is nearly dead. But 55-65 is the range when the death rate starts to really increase https://www.statista.com/statistics/241572/death-rate-by-age-and-sex-in-the-us/ https://www.statista.com/statistics/241572/death-rate-by-age... 66 is also the retirement age and while some people want to keep doing what they're doing forever, others may prefer to move into a log cabin away from civilisation. So there's a good chance he'll be around, able, and dealing with the 2038 changes, but also a reasonable change that he won't want to / be able to.
chungy 4y ago
ext4 isn't really a new file system, it's effectively ext2 with some more features enabled by default than you get with mkfs.ext2. (Yes, this is a simplified version, but not grossly exaggerated.) ext2 was first released in 1993 and has had impressive work on being extended far enough to keep up with growing storage needs, but I suspect the original designers figured it'd have been tossed aside well before 2038 would pose a problem. Unfortunately that assumption has proven to be wrong (unless maybe we do junk ext[24] within 15 years' time, but it would be extraordinary unlikely that every last machine running it will do so).
- semi-extrinsic 4y ago
  Yeah, I'm pretty sure that the people who wrote ext4 assumed there would be ext5 (or something else) to take over way before 2038. If you look at the history, ext2 was released in '93, ext3 was released in '01 and ext4 in '08. If you told the developers in '08 that ext4 would not be replaced in 30 years time, they would laugh at you (or cry).
  - chungy 4y ago
    That much is basically true too. It was yet another update to the tried-and-true format to pave over storage needs until btrfs stabilized.
josephg 4y ago
> How the fuck does ext4 [...] still have problems with this? I'm increasingly realising that most of the code out there isn't tested anywhere near as well as we think it is. Most programmers stop when the feature works, not when the feature is bulletproof. I've been writing a database storage engine recently. It should be well behaved even in the event of sudden power loss. I'm doing "the obvious thing" to test it - and making a fake, in-memory filesystem which is configured to randomly fail sometimes when write/fsync commands are issued (leaving a spec-compliant mess). I can't be the first person to try this, but it feels like I'm walking over virgin ground. I can't find a clear definition of what guarantees modern block devices provide, either directly or via linux syscalls. And I can't find any rust crates for doing this programatically. And googling it, it looks like many large "professional" databases misunderstood all this and used fsync wrong until recently. Its like there's a tiny corner of software that works reliably. And then everything else - which breaks utterly when the clock is set to 2038 because there aren't any tests and nobody tried it. I half remember a quote from Carmack after he ran some new analysis tools on the old quake source code. He said that realising how many bugs there are in modern software, he's amazed that computers boot at all.
- quanticle 4y ago
  Dan Luu has a post (https://danluu.com/filesystem-errors/ https://danluu.com/filesystem-errors/), which covers some of the same ground, and links to papers with more information on the failure modes of file systems in the face of errors from the underlying block device. Prabhakaran, et. al. (https://research.cs.wisc.edu/wind/Publications/iron-sosp05.pdf https://research.cs.wisc.edu/wind/Publications/iron-sosp05.p...), did a bunch of filesystem testing (in 2005!), and their paper includes discussion on how to generate "realistic" filesystem errors, as well as discussion of how the then state-of-the-art filesystems (ext3, Reiser (!), and JFS) perform in the face of these errors. I'm unaware of any research newer than Dan Luu's post on filesystem error handling.
  - db48x 4y ago
    I keep that link handy too! I wish there were newer research to quote, but I also don’t want to force anyone to do that job. It must be pretty depressing to rip all the bandaids off and really contemplate how bad the situation is.