2 ms·
I’ve always been curious what the most optimal “network-portable mountable filesystem image” format for Linux would be (for e.g. being a zero-install auto-fetch
by derefr 5y ago
I’ve always been curious what the most optimal “network-portable mountable filesystem image” format for Linux would be (for e.g. being a zero-install auto-fetched-from-the-network “mount and run” format for applications ala Flatpak/Snap; or being a NeXT-alike distribution container for “bundles” — where, as in macOS, it’s just as viable to open the app/document bundle [read-only] from the image, as it is to copy it out of the image.)
Such a format would have to balance the considerations of download time, storage size at rest, mount time (incl. any integrity checking), and access-time overhead when mounted. It would especially have to worry about per-file-open overhead, given that one of the primary use-cases would be rsync-ing the file tree out of the image onto your disk.
It sounds like you’re saying that a SquashFS filesystem image would actually be bad for this use-case, vs. just downloading a regular (sparse) loopback image of an ext4 filesystem over a compressed transport. Which is kind of surprising to me! I would think that the use-cases stated above would have been one of the design goals.
I’d also be curious about the relative merits of the “pure userspace” option — i.e. rather than mounting a disk-image and then accessing the files inside it through regular VFS inodes, instead having your zero-install daemon or whatever just use regular tarballs, by linking in a library like libarchive and using it to get internal virtual handles on the files. (Lots of games seem to choose this option, so it must have pretty good perf.)
- selfhoster11 5y agoI had a similar problem. Large archive files, quick random access was paramount. I considered SquashFS, but rejected it due to poor tooling on Windows, and I wanted my thing to be cross-platform. The format also seemed like something I'd need to spend some time understanding. Surprisingly, ZIP came to my rescue. It's surprisingly flexible, and includes support for Unix attributes and modern compression. In comparison to tar.xz, it was strictly superior in every way. ISO is also a good format with broad support, with many platforms now supporting mounting the contents using the built-in OS features. The only downside I could find is lack of compression, but there are supposedly format extensions that can help with that.