5 ms·

Useful enough that it should be an OS-level standard feature, imho. Unix-like OSes allow mounting disk images to explore their contents. But there's many more

Useful enough that it should be an OS-level standard feature, imho. Unix-like OSes allow mounting disk images to explore their contents. But there's many more file formats where exploring files-inside-files is useful. Compressed archives, for one. Some file managers support those, but (imho) application-level is not the optimal layer to put this functionality. Could be implemented with a kind of driver-per-filetype.

frizlab 2y ago
Honest question: How is this useful? I don’t see any use-case where this would come in handy.
- TimeBearingDown 2y ago
  You could seed compressed archives of massive text files or similar via BitTorrent while making the contents available to your apps in read-only mode.
- RetroTechie 2y ago
  It allows you to use any tool available for regular files, on the files-in-files as well. As opposed to extract contents and then work on that (requiring extra steps + disk space). Or be limited to what specialized utilities support.
- russfink 2y ago
  Exporting data to some format would be easy.
- crabbone 2y ago
  We used this in Gitlab CI. Unfortunately, the only way they deal with artifacts is by putting them in Zip files. Cache between builds would thus be stored as a Zip file. However, fully extracting it before each build would sometimes take as much, if not more time than to just build fresh. Mounting a Zip file as a filesystem allows extracting entries on-demand, at the time a file access would've been made. This was a notable speedup in our compilation process.
  - pizzafeelsright 2y ago
    tar is what you're looking for, no?
    - crabbone 2y ago
      It was a while ago, and I haven't used Gitlab in a few years. Maybe they've added TAR as an option since, but Zip was the only option at that time.
- w10-1 2y ago
  It reduces the code required to convert from N-producers to M-consumer from N x M to N+M because they're reading from and to a well-understood common form.
jraph 2y ago
It exists :-) For zip archives, there are fuse-zip and mount-zip which are FUSE filesystem. As an intermediate between OS level and application-level, there are desktop environment level: gvfs for GNOME and KIO for KDE, but they are compatible only in their own ecosystems.
- lambdaxyzw 2y ago
  Would be nice to have something that integrates with 7z - it supports a lot of weird archive types, including "weird" ones I care about (for example PE files, better known as ".exe files").
  - russfink 2y ago
    Or zstd. I have some dd blobs of partitions, the blobs are zstandard-compressed, would like to mount them.
    - mxmlnkn 2y ago
      Ratarmount also works for that. However, it, and any other tool I know of, works only well if it was compressed with pzstd because of a limitation of the zstd format. It needs separate zstd frames for fast seeking.
- ramses0 2y ago
  ratarmount for tar files.
duped 2y ago
Really what you'd like to see is a way to write the mount command for each file type (do one thing well) and another command to detect the file type and dispatch accordingly (probably similar to the `file` command), all in user space. The only thing standing in the way of this today is that MacOS doesn't expose a user space file system API. You can do this on Linux, Windows, and BSDs today. (No, file provider extensions don't cut it, Apple devs who read this, please give us a FUSE equivalent, we know it exists).
- Groxx 2y ago
  Does https://osxfuse.github.io/ https://osxfuse.github.io/ cover this? Or is there some fundamental issue? (beyond "it's not built in")
  - skissane 2y ago
    Recent macOS versions do have a general purpose built-in API for user mode filesystems. That API is incompatible with FUSE. The big problem is it is undocumented and you need an entitlement from Apple to use it, and Apple won’t give you that. Apple do have a publicly available API for cloud file systems (Dropbox-style products), but it makes a lot of assumptions which makes it effectively unusable for other use cases. Then there are third party solutions like osxfuse. These have the problem that they rely on kernel extensions and Apple keeps on making those harder and harder, and is aiming to get rid of them; plus, they are all now proprietary licensed, albeit often with a free license for open source use. One approach that does work without any kernel extensions or private APIs is to make your user filesystem an NFS server and then mount that. One competitor to osxfuse does that, but it also is proprietary
    - ranger_danger 2y ago
      According to the MacFUSE author, their specific approach is not actually undocumented: > Apple has put it in an umbrella called "unsupported" (in the kernel interfaces section) ... either Apple will not take this interface away, and if they do, it will be to provide a better interface http://preserve.mactech.com/articles/mactech/Vol.23/23.03/MacFUSE-SinghInterview/index.html http://preserve.mactech.com/articles/mactech/Vol.23/23.03/Ma...
kybernetikos 2y ago
It's not exactly the same, but nushell provides ways of exploring inside files.
lmm 2y ago
This was a core design feature of reiserfsv4, but Linux ultimately refused to merge it, probably not helped by the whole murdering-his-wife thing.
- skissane 2y ago
  > This was a core design feature of reiserfsv4, but Linux ultimately refused to merge it IIRC, because it contained these strange beasts which functioned as both files and directories - i.e. cat would return data, but then you could cd into them and run ls. Linus (among others) didn’t want to permit those violations of the file-directory dichotomy into the Linux kernel.
  - stuaxo 2y ago
    Oh that's funny- I remember a much much earlier Linus mentioning how this would he possible in Linux, I didn't know anyone actually did it. I think you really should be able to "cd" into any kind of structured data.
Sophira 2y ago
This already exists - avfs[0] does this as a FUSE filesystem. It's not the most intuitive to use, but it works, and is extensible. [0] https://avf.sourceforge.net/ https://avf.sourceforge.net/
xk3 2y ago
> Compressed archives, for one You can look inside of archives pretty easily with `lsar` (part of the unar package). It works with disk images like ISO 9660 files too But yes, especially for nested archives, having deeper OS support would be nice.
crabbone 2y ago
I thought archivemount already did that. Am I missing something? Anyway, even if that's not what you are looking for, FUSE is a more general mechanism that will allow you to do what you want (well, it seems like, at least) and much more.

frizlab 2y ago

Honest question: How is this useful? I don’t see any use-case where this would come in handy.

TimeBearingDown 2y ago
You could seed compressed archives of massive text files or similar via BitTorrent while making the contents available to your apps in read-only mode.
RetroTechie 2y ago
It allows you to use any tool available for regular files, on the files-in-files as well. As opposed to extract contents and then work on that (requiring extra steps + disk space). Or be limited to what specialized utilities support.
russfink 2y ago
Exporting data to some format would be easy.
crabbone 2y ago
We used this in Gitlab CI. Unfortunately, the only way they deal with artifacts is by putting them in Zip files. Cache between builds would thus be stored as a Zip file. However, fully extracting it before each build would sometimes take as much, if not more time than to just build fresh. Mounting a Zip file as a filesystem allows extracting entries on-demand, at the time a file access would've been made. This was a notable speedup in our compilation process.
- pizzafeelsright 2y ago
  tar is what you're looking for, no?
  - crabbone 2y ago
    It was a while ago, and I haven't used Gitlab in a few years. Maybe they've added TAR as an option since, but Zip was the only option at that time.
w10-1 2y ago
It reduces the code required to convert from N-producers to M-consumer from N x M to N+M because they're reading from and to a well-understood common form.

jraph 2y ago

It exists :-) For zip archives, there are fuse-zip and mount-zip which are FUSE filesystem. As an intermediate between OS level and application-level, there are desktop environment level: gvfs for GNOME and KIO for KDE, but they are compatible only in their own ecosystems.

lambdaxyzw 2y ago
Would be nice to have something that integrates with 7z - it supports a lot of weird archive types, including "weird" ones I care about (for example PE files, better known as ".exe files").
- russfink 2y ago
  Or zstd. I have some dd blobs of partitions, the blobs are zstandard-compressed, would like to mount them.
  - mxmlnkn 2y ago
    Ratarmount also works for that. However, it, and any other tool I know of, works only well if it was compressed with pzstd because of a limitation of the zstd format. It needs separate zstd frames for fast seeking.
ramses0 2y ago
ratarmount for tar files.

duped 2y ago

Really what you'd like to see is a way to write the mount command for each file type (do one thing well) and another command to detect the file type and dispatch accordingly (probably similar to the `file` command), all in user space. The only thing standing in the way of this today is that MacOS doesn't expose a user space file system API. You can do this on Linux, Windows, and BSDs today. (No, file provider extensions don't cut it, Apple devs who read this, please give us a FUSE equivalent, we know it exists).

Groxx 2y ago
Does https://osxfuse.github.io/ https://osxfuse.github.io/ cover this? Or is there some fundamental issue? (beyond "it's not built in")
- skissane 2y ago
  Recent macOS versions do have a general purpose built-in API for user mode filesystems. That API is incompatible with FUSE. The big problem is it is undocumented and you need an entitlement from Apple to use it, and Apple won’t give you that. Apple do have a publicly available API for cloud file systems (Dropbox-style products), but it makes a lot of assumptions which makes it effectively unusable for other use cases. Then there are third party solutions like osxfuse. These have the problem that they rely on kernel extensions and Apple keeps on making those harder and harder, and is aiming to get rid of them; plus, they are all now proprietary licensed, albeit often with a free license for open source use. One approach that does work without any kernel extensions or private APIs is to make your user filesystem an NFS server and then mount that. One competitor to osxfuse does that, but it also is proprietary
  - ranger_danger 2y ago
    According to the MacFUSE author, their specific approach is not actually undocumented: > Apple has put it in an umbrella called "unsupported" (in the kernel interfaces section) ... either Apple will not take this interface away, and if they do, it will be to provide a better interface http://preserve.mactech.com/articles/mactech/Vol.23/23.03/MacFUSE-SinghInterview/index.html http://preserve.mactech.com/articles/mactech/Vol.23/23.03/Ma...

kybernetikos 2y ago

It's not exactly the same, but nushell provides ways of exploring inside files.

lmm 2y ago

This was a core design feature of reiserfsv4, but Linux ultimately refused to merge it, probably not helped by the whole murdering-his-wife thing.

skissane 2y ago
> This was a core design feature of reiserfsv4, but Linux ultimately refused to merge it IIRC, because it contained these strange beasts which functioned as both files and directories - i.e. cat would return data, but then you could cd into them and run ls. Linus (among others) didn’t want to permit those violations of the file-directory dichotomy into the Linux kernel.
- stuaxo 2y ago
  Oh that's funny- I remember a much much earlier Linus mentioning how this would he possible in Linux, I didn't know anyone actually did it. I think you really should be able to "cd" into any kind of structured data.

Sophira 2y ago

This already exists - avfs[0] does this as a FUSE filesystem. It's not the most intuitive to use, but it works, and is extensible. [0] https://avf.sourceforge.net/ https://avf.sourceforge.net/

xk3 2y ago

> Compressed archives, for one You can look inside of archives pretty easily with `lsar` (part of the unar package). It works with disk images like ISO 9660 files too But yes, especially for nested archives, having deeper OS support would be nice.

crabbone 2y ago

I thought archivemount already did that. Am I missing something? Anyway, even if that's not what you are looking for, FUSE is a more general mechanism that will allow you to do what you want (well, it seems like, at least) and much more.