3 ms·
I'm not an FS expert and others have answered your question much better, but I don't want you think I am ignoring you, so I'll talk about this part instead: >
by byuu 12y ago
I'm not an FS expert and others have answered your question much better, but I don't want you think I am ignoring you, so I'll talk about this part instead:
> the filesystem could offer an abstraction over a patch and pointer to the original file
I've implemented a delta-based patching system before: the idea is that, given two binary buffers (ostensibly files), encode the differences between them. I have no idea how Xdelta (VCDIFF-based) and bsdiff manage to do this at reasonable speeds (their code is too much for me to understand), but the best I could manage for this was O(n^2). It's a very complicated and difficult thing to do efficiently. But indeed it can result in massive file space savings. You could cheat a bit by intercepting fwrite commands, but that won't catch insertions or deletions. Nor programs that overwrite files instead of updating them in-place (eg most of them.) As you can guess, to really make something like this efficient would require program authors to rethink how they write to files, which is unlikely to ever gain traction.
What I'd really like to see, but know that we'll never get thanks to xkcd.com/927, is a metadata system for files that is portable. Instead of relying on file extensions or unreliable magic byte header detection, you'd have the MIME type included in this metadata. Along with the displayed file name that can have any characters in them (sans maybe the path separator), the file attributes (read/write/exec/user/group/owner/hidden), creation+modification+access times, etc. And then when you'd send a file through your e-mail client or FTP it up to some web server, it'd copy over the metadata along with it. So when you moved your file from extfs over to NTFS, and then copied that over to ZFS, you wouldn't lose all of that metadata.
But good luck getting all file system authors and browser vendors to agree to a common format to transparently wrap files with.
- psture 12y agoThe basis for transferring file metadata is already there in zip in the form of -V for VMS file system metadata. This works well and can be used to transfer files via intermediate platforms which don't support that metadata. It's not as seamless as one might wish; you do have to use zip. do the transfer and then unzip on the target, and of course remember to use -V in the first place. FTP implementations also exist in the VMS world for transferring the file metadata, but I haven't used these myself so can't comment on their usability in practice.
- byuu 12y agoIndeed, ZIP gets us part of the way there, and also handles the issue of transferring more than one file in one download request. The downside to ZIP is that the information is cast away once you decompress it. If your file system doesn't maintain that metadata (eg Windows and file permissions), then it's just gone. You also can't really just keep your files in ZIP because operating systems don't really natively integrate transparent ZIP support into files, and you probably don't want the file compressed in most cases (your app may want to read it often, and not want to pay the cost of decompressing the whole thing.) But that is very close to what I am seeking, yes.