3 ms·
You're right that general-purpose filesystems have solved quite a few of the indexing problems already, unfortunately there are a few things stopping general fi
by cyphar 8y ago
You're right that general-purpose filesystems have solved quite a few of the indexing problems already, unfortunately there are a few things stopping general filesystems on a loopback device from being practical (or safe, or the best idea):
* The container (file) for the filesystem must necessarily be larger than the metadata+data for the filesystem because filesystems really don't like almost-full disks. And unless I'm mistaken sparse files are not usable for loopback devices (so you can't hack your way out of it).
* Most filesystems don't have a snapshot-style history so you would have to pick a specific filesystem from that list (otherwise you'd be forced to make CoW duplicates of the filesystem to create snapshots -- which is interestingly how Docker does layered storage with devicemapper) which has slightly similar problems to layered tar archives.
* The kernel's filesystem parsers are not really considered to be safe against an adversary, from what I've been told by filesystem engineers. So mounting random loopback files with filesystems on them might end badly.
* There is no way of looking at the archive using a userspace tool (without mounting), unless you re-implement the kernel parser for the filesystem. To be fair, this is true for any format, but filesystems are far more complicated and harder-to-parse than most other formats.
* Having a single blob as your entire image history and so on will mean that you can no longer have content-addressable storage for your images without adding something like content-defined chunking on top (which is then another layer of storage on top of your underlying storage).
* Using a Linux filesystem would mean you couldn't use the filesystem on different operating systems very easily. Even if it was compatible on whatever other filesystem you are using, userspace has no way of being sure there isn't a bug in either side's parser -- and what happens if one side changes the on-disk format. If the protocol is in userspace then it can be handled there.
* Most filesystems don't let you remap users, so if you wanted to run a container in a user namespace you would need to either rewrite the filesystem structure or mount the filesystem and copy it to another filesystem. To be fair, tar archives require you to do the mapping on extraction which is a similar problem, but far less complicated.
* Everyone would be opinionated about what filesystem to use, which means that you'd have to deal with every filesystem people throw at you, making it harder to be interoperable and adding choices where they aren't necessary. It should be up to the user what filesystem they use for storage, not the image distributor.
Now, this hasn't stopped people from trying to use this. Singularity's internal format is a loopback file with a filesystem inside, and they have privileged suid binaries that mount it. And it does have genuine performance benefits, and if you don't want things like content-addressability then it can work for some usecases.