13 ms·
It's a travesty that so few people care about moving to more reliable filesystems. A decade later and we're still stuck with ext4 and now btrfs, which I do use
by jplayer01 8y ago
It's a travesty that so few people care about moving to more reliable filesystems. A decade later and we're still stuck with ext4 and now btrfs, which I do use nowadays but I'm absolutely aware of the high risk of data corruption because the underlying code is terrible.
Windows is even worse with NTFS, which is a complete pile of garbage. I've fixed hundreds of computers with NTFS corruption over my lifetime and it's mind-boggling that Microsoft is fine with this shit.
- zaarn 8y agoThe thing about ext4 is that it's sufficiently mediocre at what it does. It's fairly crash resistant, has well worn and battle-tested codepaths and generally seems to not loose data outside bitrot. So why pick btrfs with it's dataloss stories or ZFS with it's complex management? It's an effort to switch to a new FS as default, especially if either itself or the integration is not as well tested or completed yet. So everyone just sticks to the thing they know how it sucks.
- snuxoll 8y agoMaybe it’s because I’ve already been using it for years, but I don’t really consider ZFS any more difficult to manage than the DM/LVM/*FS stack I work with on most Linux systems. Compared to just running gdisk and using raw partitions - sure, but I’ve been using LVM for over a decade now.
- zaarn 8y agoZFS is a lot more complicated if you're expecting the average consumer to handle it. It doesn't even handle 1-disk-storage expansion. Even Synology and QNAP now have plug-and-play solutions to expand a RAID one disk at a time. You can even start plugging in higher capacity disks and once you have enough extra parity, the higher capacity becomes used magically. No need for wasteful RAID10/50/60 setups. Just plug in more drives.
- RantyDave 8y ago1 disk storage expansion? You do 'zpool add' IIRC.
- zaarn 8y agoThat doesn't give you RAID protection, sadly, with mdadm and LVM you can add a single disk into an existing array, ZFS requires you to pull up a new RAID array (vdev) and at minimum 2 devices if you want something better than a RAID 0.
- snuxoll 8y agoI don’t run Z-RAID 5/6 personally so it’s been a non issue, but that functionality is in the works right now.
- zaarn 8y agoIt is in the works but the question is how many years until it's stable and available to all end users and how well it works (IIRC from a talk on this topic, it had some limitations as well). This also affects the RAID1 setup, any storage expansion requires 2 disk to pull up a new RAID1 into the pool.
- snuxoll 8y agoMirrors, unlike RAID-Z, can be added later. They can even be detached from the pool as long as one mirror from the device remains.
- PhantomGremlin 8y agoin the works right now I don't know where you read that, but don't be fooled. That functionality has been "in the works" for quite a long time. But it hasn't yet appeared. In the past it has been said that something called "block pointer rewrite" was a prerequisite. Just google for that. Here's some 10 year old information about it. At that time it was already a "frustratingly frequent request." I stopped searching when I found something 10 years old, but I'm sure you with some effort you could go back much further than that. https://blogs.oracle.com/ahl/expand-o-matic-raid-z https://blogs.oracle.com/ahl/expand-o-matic-raid-z
- Asmod4n 8y agoZFS and "complex management"? there comes nothing close in the open source market than ZFS and its easy and straightforward to use management tools in my opinion. You don't need a handful of tools just to do one simple task, just hack away at the zfs and zpool commands and everything gets done. No need for parted, mdadm, lvm or support of your needs compiled into the kernel. No matter on which operating system you are currently, you only need those two tools and they work the same everywhere.
- zaarn 8y agoIt comes down to when you need to anything with ZFS but simply use it for data storage. Expanding a RAID vdev is not possible, instead the recommendation is to waste 50% of storage for a pool of mirrored devices to get the same protection as a RAID1 of two drives, which isn't much for some people. For the avarage home user, such a scaling option is already no longer viable. This is topped with ZFS' poor performance under low RAM conditions. FreeNAS barely managed 10MB/s when running with 2GB of RAM and an old Pentium 4 Dual Core. Not everyone can afford to throw hardware at the problem. On the other hand, Linux performs under the same conditions with 8 times the speed for simply copying files and mdadm/lvm/unraid allow me to dynamically grow the pool of harddrive space a single drive at a time. (plus mdadm/lvm is included in basically every larger distro by now, no need to install it or compile support into the kernel)
- boomboomsubban 8y ago>Expanding a RAID vdev is not possible, instead the recommendation is to waste 50% of storage for a pool of mirrored devices to get the same protection as a RAID1 of two drives, which isn't much for some people Or make a new vdev. It takes a bit of planning, but it's not some massive hurdle. >This is topped with ZFS' poor performance under low RAM conditions. FreeNAS barely managed 10MB/s when running with 2GB of RAM and an old Pentium 4 Dual Core. Not everyone can afford to throw hardware at the problem. FreeNAS is enterprise software, the default settings expect you to read and follow the system requirements. The developers aren't spending their time supporting it on less powerful systems, but ZFS (and likely FreeNAS) can run fine if you configure it for that hardware.
- laumars 8y agoZFS is no harder to use than ext4, if all you want is a file system. But then once you start needing multiple volumes or multiple disks per storage pool (eg LVM, mdadm, etc) then ZFS really does come into it's own for being by far the easiest tool for solve those problems (that I've used anyway). The hardest part on Linux if you're running a distro that doesn't officially support ZoL so you'd have to compile you're own kernel module. But even that isn't the problem it used be in the old days of Linux since we usually have bootstrapping scripts to handle the pain now (however if you can run FreeBSD then that's always a good choice as well. It's genuinely a really nice platform to use).
- zaarn 8y agoIf you only need a single disk filesystem, it's basically the same plus some minor RAM hunger (doesn't run well with 1-2GB regardless of how many disks you have). ZFS isn't quite easier to user for multiple disks or volumes, rather, there is a very tiny set of configurations that everyone recommends (RAID1 mirrors, leaving consumers unable to grow pools 1 disk at a time because ZFS is fundamentally unable to grow a RAID vdev). LVM and mdadm manage these jobs much better and are composable to your needs.
- laumars 8y ago> doesn't run well with 1-2GB regardless of how many disks you have I've read people say that but I've never had any issues when I used to run ZFS on various systems with 2GB RAM. So I wonder how many of those people actually had practical experiences running ZFS on low memory systems and how much of it is people just looking at some of the recommended memory specifications (which are high) and start making assumptions about it's performance on low memory systems. > RAID1 mirrors, leaving consumers unable to grow pools 1 disk at a time because ZFS is fundamentally unable to grow a RAID vdev You actually can grow a ZFS vdev - exactly by replacing one disk at a time (albeit you don't get the gained capacity until all the disks in the vdev have been replaced). What you cannot do is shrink it - but that's not a unique problem to ZFS and is a pretty uncommon use-case anyway. Also you cannot grow a ZFS "raidz" vdev by adding disks - but you cannot do that with any typical RAID5/6 array either. However if you want to add disks to a raidz then you can workaround the RAID5/6 striped parity problem by adding the additional disks into a new vdev and adding that new vdev into the existing ZFS volume (named "tank"). The kind of storage arrays that grow by adding solitary disks work quite different to your typical RAID. They'll often have designated parity drives instead. See solutions like "unraid". > LVM and mdadm manage these jobs much better and are composable to your needs. LVM and mdadm are not file systems though, so they only solve part of the problem. The fact ZFS handles volume management, software RAIDing, file system operations (and a bunch of other stuff that your typical fs doesn't yet support (eg snapshotting)) and does so from surprisingly intuitive command line tools is what edges ZFS for me in terms of usability. Ultimately though, "easier" is such a broad term that we're essentially just arguing personal opinion. :)
- eeperson 8y ago> So why pick btrfs with it's dataloss stories or ZFS with it's complex management? As you mentioned, the bitrot. Also, my understanding is that a COW filesystem can be more reliable than a traditional journaling filesystem in a system crash since nothing is overwritten. Plus there is compression and all of the features provided by a COW filesystem.
- zaarn 8y agoBitrot is rare. Over the average lifetime of most consumer desktops, the loss of any personal documents is very unlikely unless it makes up a majority of the disk. And if it does, it's most likely that it affects negligible parts of the file (ie, corruption in a frame of video). The risk of bitrot becomes interesting when you talk about long timeframes like a decade. Considering the capacity and price of a 2TB disk, compression isn't the immediate need of the consumer either, unless they have a NAS, where Syno and QNAP already provide integrated solutions with btrfs and ext4 both. The modern ext4 journaling is also very battle tested, it's a very simple and resilient physical journal, data loss in ext4 happens largely when the disk is already failing or has been cheap enough to lie about the state of data being written or not. For the average consumer that sums up to "it doesn't matter enough". Something like bcachefs could change the game by being btrfs+ZFS but without the dataloss risk.
- jl6 8y agoConcurring on the rarity of bit rot. I have kept and verified checksums for years, over many terabytes of data, and have never observed bit rot. Of course I’ve been using btrfs RAID1, so perhaps bit rot has been silently corrected.
- eeperson 8y ago> Bitrot is rare. Over the average lifetime of most consumer desktops, the loss of any personal documents is very unlikely unless it makes up a majority of the disk. And if it does, it's most likely that it affects negligible parts of the file (ie, corruption in a frame of video). > The risk of bitrot becomes interesting when you talk about long timeframes like a decade. I think you are making a lot of assumptions here about frequency of bitrot, especially in the face of failing hardware. Also, consumer desktops are not the only use. There are bunch usages where the risk of bitrot is much more concerning. Or were you intending your original post to be only about consumer usage? >Considering the capacity and price of a 2TB disk, compression isn't the immediate need of the consumer either, unless they have a NAS, where Syno and QNAP already provide integrated solutions with btrfs and ext4 both. This kind of feels like you are saying that it isn't needed because it is already being used when it is needed. Am I misunderstanding? Consumers may want compression for smaller hard drives they already own in order to get more out of them. Also, compression can speed up disk IO in some workloads. > The modern ext4 journaling is also very battle tested, it's a very simple and resilient physical journal, data loss in ext4 happens largely when the disk is already failing or has been cheap enough to lie about the state of data being written or not. The issue with ext4 resilience isn't about quality of the implementation. It is a fundamental limitation of no COW filesystems. Since things are overwritten, if the system crashes mid write you can end up with partial data. COW filesystems always give you the old data or the new data. This can be worked around but that requires that all of your applications do this workaround. > For the average consumer that sums up to "it doesn't matter enough". Something like bcachefs could change the game by being btrfs+ZFS but without the dataloss risk. What makes you think bcachefs is going to be any better? A lot of functionality still has to be implemented. What makes you think that the implementation of this functionality will be any less error prone? You also haven't addressed the large pile of new features provided by BTRFS/ZFS. Snapshotting alone can make it worth the price of admission for consumers. With snapshotting, you can set up a system where rollbacks after any upgrade can occur seamlessly.
- chucky_z 8y ago.... XFS? I've had a single issue across many EBS drives partitioned with only XFS, in several mdadm configurations. The data-loss occured during a maintenance window, so I'm convinced it was an EBS issue and not XFS. I've also used it on physical hardware and never experienced dataloss, even losing drives in RAID configurations (I will admit I was lucky enough to never lose 2 drives in my normal RAID10 setup). I've had several filesystems become corrupted, but the XFS community is extremely helpful, and the tooling that comes with it has never failed me in at least letting me figure out what's actually wrong.
- jplayer01 8y agoEhm, what distro supports xfs out of the box and what are the guides like for setting it up? Also, it seems to be missing features like COW or data checksums (bitrot says hello), and data deduplication seems to still be experimental. Doesn't seem like the next-gen filesystem we should be hoping for.
- kevml 8y agoRedhat and CentOS use XFS by default.
- jplayer01 8y agoReally not a fan of their package managers.
- boomboomsubban 8y agoWhen you're installing you can select something like "manually set the partitions/disks" and format XFS with basically any distro. It's part of the kernel, and I can't remember installing a distro without that option.
- zbentley 8y agoIf by that you mean "these distros support XFS, but I don't use them for other reasons (package managers)" . . . OK. Don't use them. But don't criticize a filesystem for unrelated tradeoffs that OSes that support it happen to make.
- eeperson 8y ago> btrfs, which I do use nowadays but I'm absolutely aware of the high risk of data corruption because the underlying code is terrible. Is the underlying code terrible? I haven't heard any particular concerns about code quality (aside from the original implementation of RAID 5/6). Most of the issues I've heard about were due to people using features that weren't ready yet. I've heard a lot fewer of these issue since BTRFS started providing clear statuses for individual features[1]. Do you have a different impression? [1] - https://btrfs.wiki.kernel.org/index.php/Status https://btrfs.wiki.kernel.org/index.php/Status
- zanny 8y agoI've used btrfs as my main fs (with ample backups) since 2012. Plenty of file system corruption and backup restores in the intervening years, but I went from having it happen 4x a year in 2012-2013 to once a year by 2015 and never since 2016. Its totally anecdotal to say just because my ~10 hard drives with it being stable for two years means its improved but I'd consider putting some faith in it now. A bit. With backups.