4 ms·

I can't remember the last time I lost data to a file system bug. I've certainly lost data to applications running on top of file systems ("Source Safe" -- when

by kabdib 11y ago

I can't remember the last time I lost data to a file system bug. I've certainly lost data to applications running on top of file systems ("Source Safe" -- when they put "Safe" in the product name, you know you have to run away...) All the stuff I've lost has been due to: - Hardware failure (vanilla smoke-from-the-disk, or RAID controller failure, or too many RAID volumes dying at once) - Fat fingers (rm -rf equivalents, or format the wrong drive) - Data is ancient and in a proprietary format, and the apps that can read it won't run any more (at least, not without a great deal of effort) - Butterfingers (dropped disk on a concrete floor) - Media age (drive won't turn / oxide got tired, lost adhesion and rubbed off), or treating media badly (a decade of alternating hot/cold in the garage next to the spiders is not good) - Vanishing interface standard (no EISA slot, or nowadays, no parallel port), or no drivers for modern OSes I'm happy there's a provably correct file system out there. But it ain't gonna save you :-)

amelius 11y ago
> I can't remember the last time I lost data to a file system bug. You are probably not an early adopter of new filesystem technology. If you were sitting on the edge of technology, you would strongly prefer a provable filesystem.
mwcremer 11y ago
"The hard drive is ON FIRE!"
- stephengillie 11y ago
  https://en.m.wikipedia.org/wiki/Halt_and_Catch_Fire https://en.m.wikipedia.org/wiki/Halt_and_Catch_Fire
- kabdib 11y ago
  One fine day I came into work and overnight my Gateway workstation's boot drive had developed 10,000 bad sectors (maybe more, who counts these things?) Who indeed? I called Gateway tech support and said, "Hi, overnight my hard drive got over 10,000 bad sectors and I'd like you to replace it, since it's under warranty." Gateway support: (after going through some really tiresome support nonsense, like power-cycling the machine and verifying a network connection of a computer that can barely boot). "Sorry, we're not going to do that. Your hard drive still has several hundred thousand perfectly good sectors, so it's working fine, no replacement." Me: (facepalm) So eventually through some social engineering I got them to replace the drive (a little play acting involving my manager pretending to yell at me, which got instant sympathy from the Gateway support person. This is a useful tool). Really I just went over to Fry's and bought a new hard drive. But it was the principle of the thing. And I resolvde that the next time I had to call in a hard drive failure, I would say: "Hello, Gateway? My hard drive is ON FIRE and my computer won't boot any more. What do you want me to do?"
  - A010 11y ago
    I remember the one I got years ago https://www.youtube.com/watch?v=-e9OcWIH6N8 https://www.youtube.com/watch?v=-e9OcWIH6N8
feld 11y ago
I've seen data loss due to bugs: * ext3/ext4 * reiserfs (reiserfsck used to eat data *all the time*) * xfs * jfs (can't remember kernel version, but friend lost tons of data) * ntfs Filesystems I haven't lost data on: * ufs, ufs2, ffs -- the BSD ufs variants * zfs Filesystems I haven't done much with: * hammer (would be amazing for maildir storage) * btrfs (avoiding intentionally) * reiser4 (barely used it and then the whole murdering happened) * tux (always sounded nice) There are lots of other filesystems with special use cases, but I'm ignoring those. These are just more relevant general purpose filesystems. And note that filesystems without journals, logs, or being CoW (ext2, [ex]FAT, etc) I'm ignoring because they aren't designed to handle abuse, power loss, etc. This paper about linux filesystems is worrying: https://www.usenix.org/conference/osdi14/technical-sessions/presentation/pillai https://www.usenix.org/conference/osdi14/technical-sessions/...
- kabdib 11y ago
  Yeah, avoiding file systems that don't have any transaction nature is a sound principle. I was dismayed that NTFS "was transactional" but that those transactions didn't mean that it saved user data, it just made sure that filenames and directory structure was preserved. That's entirely missing the point, guys. (Later, I got a peek at the user-data-level transactional extension to NTFS, and had a good case of the horrors. Run away; MSFT is deprecating it, last I heard, and good thing, too).
  - grogers 11y ago
    Most filesystems default to saving only metadata in the journal. Writing all file data through the journal is much more expensive (unless you are log-structured).
derefr 11y ago
Most of these problems can be averted by a "port early, port often" approach to data storage. Everything should be copied to a new piece of media every five years at most, in whatever the newest stable media format is these days. If that means the old hardware can't read the new media, then it's time to write an emulator for the old hardware that runs on new hardware. That also saves you from the "proprietary format" problem later on—if you have that emulator, you can still start up the app 30 years later in QEMU and get everything exported over the virtual serial bus. As for fat fingers—that's what creating a filesystem abstraction on top of versioned object storage is for!

by kabdib 11y ago

amelius 11y ago

> I can't remember the last time I lost data to a file system bug. You are probably not an early adopter of new filesystem technology. If you were sitting on the edge of technology, you would strongly prefer a provable filesystem.

mwcremer 11y ago

"The hard drive is ON FIRE!"

stephengillie 11y ago
https://en.m.wikipedia.org/wiki/Halt_and_Catch_Fire https://en.m.wikipedia.org/wiki/Halt_and_Catch_Fire
kabdib 11y ago
One fine day I came into work and overnight my Gateway workstation's boot drive had developed 10,000 bad sectors (maybe more, who counts these things?) Who indeed? I called Gateway tech support and said, "Hi, overnight my hard drive got over 10,000 bad sectors and I'd like you to replace it, since it's under warranty." Gateway support: (after going through some really tiresome support nonsense, like power-cycling the machine and verifying a network connection of a computer that can barely boot). "Sorry, we're not going to do that. Your hard drive still has several hundred thousand perfectly good sectors, so it's working fine, no replacement." Me: (facepalm) So eventually through some social engineering I got them to replace the drive (a little play acting involving my manager pretending to yell at me, which got instant sympathy from the Gateway support person. This is a useful tool). Really I just went over to Fry's and bought a new hard drive. But it was the principle of the thing. And I resolvde that the next time I had to call in a hard drive failure, I would say: "Hello, Gateway? My hard drive is ON FIRE and my computer won't boot any more. What do you want me to do?"
- A010 11y ago
  I remember the one I got years ago https://www.youtube.com/watch?v=-e9OcWIH6N8 https://www.youtube.com/watch?v=-e9OcWIH6N8

feld 11y ago

I've seen data loss due to bugs: * ext3/ext4 * reiserfs (reiserfsck used to eat data *all the time*) * xfs * jfs (can't remember kernel version, but friend lost tons of data) * ntfs Filesystems I haven't lost data on: * ufs, ufs2, ffs -- the BSD ufs variants * zfs Filesystems I haven't done much with: * hammer (would be amazing for maildir storage) * btrfs (avoiding intentionally) * reiser4 (barely used it and then the whole murdering happened) * tux (always sounded nice) There are lots of other filesystems with special use cases, but I'm ignoring those. These are just more relevant general purpose filesystems. And note that filesystems without journals, logs, or being CoW (ext2, [ex]FAT, etc) I'm ignoring because they aren't designed to handle abuse, power loss, etc. This paper about linux filesystems is worrying: https://www.usenix.org/conference/osdi14/technical-sessions/presentation/pillai https://www.usenix.org/conference/osdi14/technical-sessions/...

kabdib 11y ago
Yeah, avoiding file systems that don't have any transaction nature is a sound principle. I was dismayed that NTFS "was transactional" but that those transactions didn't mean that it saved user data, it just made sure that filenames and directory structure was preserved. That's entirely missing the point, guys. (Later, I got a peek at the user-data-level transactional extension to NTFS, and had a good case of the horrors. Run away; MSFT is deprecating it, last I heard, and good thing, too).
- grogers 11y ago
  Most filesystems default to saving only metadata in the journal. Writing all file data through the journal is much more expensive (unless you are log-structured).

derefr 11y ago

Most of these problems can be averted by a "port early, port often" approach to data storage. Everything should be copied to a new piece of media every five years at most, in whatever the newest stable media format is these days. If that means the old hardware can't read the new media, then it's time to write an emulator for the old hardware that runs on new hardware. That also saves you from the "proprietary format" problem later on—if you have that emulator, you can still start up the app 30 years later in QEMU and get everything exported over the virtual serial bus. As for fat fingers—that's what creating a filesystem abstraction on top of versioned object storage is for!