9 ms·
New tricks for XFS: support for subvolumes
- brian_herman 9y agoNice can’t wait for the next version of XFS from red hat.
- qubex 9y agoI've been having some fun with a revived SGI Octane running IRIX and it occurred to me, quite out-of-the-blue, that XFS development essentially ceased before SSDs were ever even contemplated. For a few moments, I pondered the apparent profundity of this realisation, and then I moved on, cursing the lack of package management while trying to get something to work.
- mixmastamyk 9y agoBelieve there was a "software manager" or some such.
- liveoneggs 9y agopkgsrc is still setup to work on IRIX, although few have used it recently
- KaiserPro 9y agoah, youre right up to a point. around 2014 there was a great flurry of work done, which meant its metadata speed improved >50x turning it from a great streaming file system, to a good all rounder.
- aidenn0 9y agoAh good to know; I had given up on XFS before then because the metadata speed made certain SVN operations painful
- pbh101 9y agoIt was a bit before that, iirc. LWN article summarizing the changes at [1], which are mostly delaylog, if we are referring to the same thing. Was default-enabled in 2011 on Linux 3.3. ... I’ve happened to spend much of the last three weeks learning about XFS. [1] https://lwn.net/Articles/476263/ https://lwn.net/Articles/476263/
- KaiserPro 9y agoAha, good spot. My boss sent me the video of that presentation
- jaas 9y agoI worked down the hall from the XFS team at SGI in 2005. Development had definitely not ceased, and their work did not seem heavily related to Octane systems. They were focused on XFS for Suse and RedHat Linux for enterprise and supercomputing.
- hapless 9y agoIRIX, like all SRV4, has a perfectly adequate package manager. In the SGI case it is called "inst"
- diegocg 9y agotl;dr XFS will support using filesystem images as if they were directories, kind of like an "internal loop", which will allow (with the help of copy-on-write data, which they support in recent versions) having subvolumes/snapshots.
- josteink 9y agoIt might just be how you present it, but to me that sounds like using multiple layers of hacks to implement use-cases other filesystems were carefully designed to support in the first place, and that sounds extremely unreliable and brittle. I'll stick to ZFS :)
- diegocg 9y agoYeah, these subvolumes are going to have scalability issues compared with cleanly designed subvolumes such as in ZFS. But I wouldn't describe them as a hack - it's a rather interesting feature that not other filesystem has explored before. I would describe it as "loop devices done well". I don't think reliability will be a problem, for the upper layer these embedded filesystems are in fact just files.
- viraptor 9y ago> When a CoW filesystem writes to a block of data or metadata, it first makes a copy of it Is this a really a precise description? I was sure that (for data) the actual copy only happens in case of a block with multiple references. If there was a single use of a block I expected it to be modified in place in most real-world filesystems. Summary from wikipedia seems to confirm that. Am I missing something, or is that just unfortunate wording? What they describe with the write of the data followed with write of the indexes as new elements seems more like a log filesystem.
- barrkel 9y agoAlways copying is what ZFS does as far as I know - so that the final write on raidz can be a pointer swap at the top of the tree.
- ghusbands 9y agoIt's an implementation detail that can and does go either way in many real world systems, depending on the technologies involved.
- wmf 9y agoNo, the copy is generally done every time. In-place writes cause all kinds of problems with atomicity, ordering, torn writes, etc. Also, for ZFS/btrfs/bcachefs style filesystems the pointer contains a checksum of the block and thus it needs to be updated on every write.
- aidenn0 9y agoCoW stands for "Copy on Write" a system that updates blocks in-place is not is not CoW.
- viraptor 9y agoThere's lots of things which are called copy on write, where CoW means really copy-on-deduplication-otherwise-update-in-place. Like qcow2 filesystem. Or Cow type in Rust.
- amelius 9y agoBy the way, I'm using XFS only because it allows for more than 64k hardlinks per file. Strangely, this isn't possible with ext4 out of the box.
- xelxebar 9y agoJust curious, but what use case do you have for such a voluminous quantity of hardlinks?
- amelius 9y agoBasically copying files without actually copying them. For example, when making versioned backups, or when copying a large number of files from production to various development environments. Something I wish there was a better solution for. Perhaps there is a filesystem out there that supports CoW in a way that fits this usecase (XFS perhaps even), but I haven't looked into it. The disadvantage of using hardlinks is that you can't hardlink between users (user is a property of the file, not of the link to the file), and there's always the danger that a write takes place through one of the links. Imho, that should really be solved at the filesystem level using a CoW scheme.
- koenigdavidmj 9y agoDoes btrfs, with cp --reflink, not meet your use case?
- 9y ago
- CyberShadow 9y agoI think the biggest missing feature for home/casual use in XFS right now is shrinking. Currently, it's impossible to reduce an XFS filesystem (partition) in size, so you have to commit to your disk layout once you've set it up. Whether it's to install another OS side-by-side, grow a swap partition, experiment with or gradually migrate to another filesystem, none are currently possible without adding additional physical storage or using loop devices. The same applies to ZFS (it's not possible to shrink a ZFS pool), which is why I'm currently using btrfs (with all its pain points) on my machines.
- saghm 9y agoDoesn't LVM solve this? I know this might not work as well in some cases with multi-disk arrays, but I'm not sure I'd call that "casual use".
- CyberShadow 9y agoYou mean, by overcommitting the filesystem size, and using discard to free up underlying space? Possibly... how well does that work in practice? What happens when LVM runs out of space due to some runaway disk-filling process, is the filesystem ready to handle out-of-space errors coming from LVM in all situations?
- jbronn 9y agoFrom operations experience, the risks of data loss with btrfs far outweigh volume shrinking capabilities on either XFS or ZFS.
- CyberShadow 9y agoFortunately, btrfs is really good at backups. Well, so is ZFS, but in some situations, like choosing what filesystem to use on a rented dedicated server, any choice that could lead to a situation requiring physical hardware configuration changes is a non-starter. ZFS licensing is an issue too, as it means that you can't just boot into any ol' Linux live CD (or remotely boot into a rescue environment) to fix the system or salvage the data on it.