6 ms·
Reading the comments, I'm guessing I was the only fan of SGI who saw supercomputers used XFS (or variants), figured it would handle my lowly workloads, put it o
by nickpsecurity 11y ago
Reading the comments, I'm guessing I was the only fan of SGI who saw supercomputers used XFS (or variants), figured it would handle my lowly workloads, put it on my machines, worked around its few weaknesses, and just never thought of filesystem choice again aside from boot partition.
Still my favorite filesystem for what was my favorite supercomputing & NUMA company.
- grahamux 11y agoand now XFS is the default in RHEL / CentOS 7
- nickpsecurity 11y agoI was about to try out the latest versions of RHEL-based distros in the near future. You just gave me more incentive. :)
- jessaustin 11y ago...worked around its few weaknesses... Details please! What are they and what did you do?
- nickpsecurity 11y agoBelieve it or not, Wikipedia has you covered on some big ones: https://en.wikipedia.org/wiki/XFS#Performance_considerations https://en.wikipedia.org/wiki/XFS#Performance_considerations (And disadvantages section.) I mostly used lowest common denominator of it with backups, clusters, and so on. Any other issues I countered with storage design and recovery procedures. I often did recovery around once a week or month to prevent silent buildup of issues. So, there might be issues I'm not aware of but it worked fine almost every time. Another commenter mentioned audio skips which happened to maybe a handful of my own files out of GB of music. Could be one of those issues or a quirk with that commenter's setup.
- bronson 11y agoI did the same thing for the same reason and I'm still finding songs in my mp3 collection with long strings of nulls in them. They only spent a year on XFS but that, plus a flaky display driver, did quite some damage. Luckily I kept my less bulky files on an ext2 partition and they were fine. Turns out supercomputer software might be written to a different set of requirements... Who knew!
- nickpsecurity 11y agoI try not to rely on one storage device: assume it will have issues and implement accordingly. Yet, I've had audio issues with some files on XFS partitions. I didn't investigate further to see what it was but it is interesting that you said that. Might be an issue there. EDIT to add: check your hard disks and OS's you plug them into, too. I lost a ton of data once over a bug in a driver that looked like a filesystem bug but had nothing to do with it. Several layers where issues can pop up but aren't obvious from their results.
- marcosdumay 11y agoNot relying on storage is a too general advice. I've never seen in practice a backup system that would survive your filesystem silently corrupting your data at random. (Yeah, I know they exist. I've just never been in some place as big as to actually use them.)
- icebraining 11y agoYou don't need anything big; it's just a matter of saving an hash of the file, then occasionally rehashing the files and comparing. If it's different, fetch the file from another location and update the local copy. I use git annex, which has this built-in; just run "git annex repair" and it'll verify & fix any damaged files.
- nickpsecurity 11y agoYou nailed it. And there were typically utilities to do it on every 'NIX box I encountered. Along with scripts to mostly automate the process. :) Then there was my evolved scheme of using diversity where I had different OS's or filesystems with stuff handled at application layer. More complex to set up but decreases chance any one set of code is going to mess you up. Safety or security through diversity is a powerful technique. Harder to do today with so much code reuse: things might only be different on the surface with same bugs lurking underneath. All these different web security schemes using the same OpenSSL library is a good example.
- anabis 11y agoMy understanding is that XFS was designed for big media files. With the advance in SSD, HDD will be mainly used for big files, so XFS may yet end up dominating. I don't know if ext4 is equally good on the SSD or not.
- andyana 11y agoI've always used XFS when I expect lots of small files. Maybe 10 or 15 years ago, I was benchmarking filesystems to store maildirs; XFS was the fastest by far when it came to directory scans. I'm not sure if that's still the case, but I haven't had a reason to re-examine.
- nickpsecurity 11y agoXFS was designed for use by the company that made these [1] bad boys. They did it and a clustered variant. Their systems worked on both large files for media and applications where lots of small data had to be processed in parallel (or just faster). So, it was general purpose albeit tilted for the big stuff a bit. I never benchmarked it, though, so I won't speculate about what's better for small files. "With the advance in SSD, HDD will be mainly used for big files, so XFS may yet end up dominating." Good point. We're already seeing that with all the videos and games. So, there might be a resurgence of XFS use. I doubt it will dominate, though, unless it stays the default in enough distro's. [1] https://duckduckgo.com/?t=lm&q=SGI+Origin&ia=images https://duckduckgo.com/?t=lm&q=SGI+Origin&ia=images
- buster 11y agoActually, XFS is now default on RHEL and Suse :) But honestly, over the last years i have a strong feeling for the old, fast, stable (and most importantly simple!) filesystems like ext* or xfs. Just yesterday i had btrfs telling me that my disk is full (when in fact it is half full with nearly 400GB free!). Also have seen some major, hard to resolve problems with ZFS years ago. It led me to believe, why the hell we need those ultracomplicated can-do-everything filesystems?! The only thing i actually care for is fast access without some complicated compression, deduplication and what-not steps in between and data safety. And as far as i can tell, ext4 and XFS are pretty safe. Never seen those destroy files. Ever. On the other hand finding problems and debugging filesystems in kernel can be a major headache. I want my filesystem to as simple as possible, nowadays. ZFS and btrfs is all nice and so on, but next time i look for a filesystem, i'll go back to the established and simpler ones.
- XorNot 11y agoI use ZFS on every machine I directly control and could never consider going back. Definitely lost way too many files to XFS zeroing things out on a crash.
- nickpsecurity 11y agoThe things I read on it were nice. I'll give it that.
- laumars 11y agoI've had the opposite impression of ZFS - it's saved my data on a number of occasions when ext* would have failed completely. I personally think Sun's engineers did an amazing job with ZFS. Plus while the file system is hugely complex, management of it is remarkably straightforward - which leads to less user / sysadmin error when running it. Sadly though, I do agree with your judgement of Btrfs. I trialled it for about 6 months and found it to be conceptually similar to ZFS but very much it's practical polar opposite. However, like with any software decision, the right file system choice depends as much on the platform's intended purpose and the administrator, as it does on age and features. For file servers, ZFS is an excellent choice; but for low footprint appliances, you'd be better off with ext4 or xfs. And desktops is just a question of personal preference.
- lmm 11y agoXFS always had the trailing zeroes problem. And has no FSCK if your filesystem gets really messed up. I still use reiserfs when I have to run linux (these days I'm mostly FreeBSD with ZFS).
- nickpsecurity 11y agoMy application-layer file management avoided the issues. To be honest, I didn't even know what they were other than a lack of FSCK. Problems happened, they were corrected, and so on. It's an attitude I learned from research into cluster storage and safety-critical systems. That said, this does seem to be a big problem for anyone depending on XFS by itself. Sounds like filesystem developers should dig in and fix that zero's problem among any other that are known. Just seems wasteful to ditch such a good, battle-hardened filesystem over a few bugs. Better to just fix them.
- the_why_of_y 11y agoThe zeroes problem (which was mostly caused by applications not using fsync when they should) was fixed in 2007. http://www.xfs.org/index.php/XFS_FAQ#Q:_Why_do_I_see_binary_NULLS_in_some_files_after_recovery_when_I_unplugged_the_power.3F http://www.xfs.org/index.php/XFS_FAQ#Q:_Why_do_I_see_binary_... Also, there's xfs_repair(8).
- nickpsecurity 11y agoAwesome! I was having a hard time believing such a big problem would get no attention all this time. Good to know they knocked it out.
- moonbug 11y agoYes. XFS just works.