4 ms·

Congrats on a tremendous technical feat! Any advantage of using a full-scale FS over something like git-annex (http://git-annex.branchable.com/ http://git-anne

by akg 14y ago

Congrats on a tremendous technical feat! Any advantage of using a full-scale FS over something like git-annex (http://git-annex.branchable.com/ http://git-annex.branchable.com/)?

artagnon 14y ago
The seamless workflow that only a full-blown filesystem can offer- new revisions are created automatically when files are edited and saved.
- fuzzix 14y ago
  I like it. It reminds me of VMS's versioning. I implemented something like VMS style versioning with Fuse and Perl for a talk/demo a couple of years ago - it simply created a new version on open() if any writeable flags were passed - not robust, but functional enough to demo. edit Possessed by VMS, not multiple VMSes.
- p9idf 14y ago
  Does your file system create revisions of the entire tree, or only revisions of individual files within the tree as VMS and RCS do?† Because there are some problems with the latter approach. One problem described by Geoff Collyer (the C News guy) is that "version numbers of related files are not kept in synch, so you can't just cp *';6' /tmp/compile".†† † I tried answering that question myself, but your program kept crashing. †† http://9fans.net/archive/2001/11/759 http://9fans.net/archive/2001/11/759
  - artagnon 14y ago
    Revisions of individual files only. Yeah, you can't do that kind of sorcery with phoenixfs. † Fixed. Sorry about that.
- spartango 14y ago
  You should be able to achieve this with Filesystem events (inotify/kqueue/filesystemwatcher). Just another way of doing things. I'm not the biggest fan of FUSE filesystems(until highly-tuned, ala NTFS-3G), but they certainly do work.
  - halayli 14y ago
    No, you cannot achieve atomicity using these calls. By the time you receive a notification the file might have been edited twice.
    - gwern 14y ago
      If the file is changing that fast, do we really care? I'd be happy to have all my files tracked at only 5s temporal resolution...
      - rhizome 14y ago
        Sure, but which 5s? :)
    - spartango 14y ago
      This is true, but not without drawbacks: * Using FUSE you gain the atomicity through synchronicity; when a write occurs, you stand between the disk and request. In so doing, you introduce the potential for latency (nevermind the latency inherent to FUSE). * Using fsevents, you get notifications asynchronously, so there's no I/O performance hit. You may not get the events right away, but with fsevent APIs there's a bounded queue so you generally don't miss events, unless there are too many and the buffer is overflowed[1,2]. The events don't contain the actual file data, so you will lose intermediate edit data. Thinking about this, it would be cool to exploit a Copy-on-Write filesystem (like ZFS) to scalably add the edit data to fsevent apis. I suppose you could preserve write buffers in memory temporarily instead of CoW, but that would have a nasty memory footprint. [1]http://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher.aspx http://msdn.microsoft.com/en-us/library/system.io.filesystem... - Internal buffer size [2]http://docs.oracle.com/javase/7/docs/api/java/nio/file/WatchService.html http://docs.oracle.com/javase/7/docs/api/java/nio/file/Watch... (see the watchkey queue and overflow)

by akg 14y ago

Congrats on a tremendous technical feat! Any advantage of using a full-scale FS over something like git-annex (http://git-annex.branchable.com/ http://git-annex.branchable.com/)?

artagnon 14y ago

The seamless workflow that only a full-blown filesystem can offer- new revisions are created automatically when files are edited and saved.

fuzzix 14y ago
I like it. It reminds me of VMS's versioning. I implemented something like VMS style versioning with Fuse and Perl for a talk/demo a couple of years ago - it simply created a new version on open() if any writeable flags were passed - not robust, but functional enough to demo. edit Possessed by VMS, not multiple VMSes.
p9idf 14y ago
Does your file system create revisions of the entire tree, or only revisions of individual files within the tree as VMS and RCS do?† Because there are some problems with the latter approach. One problem described by Geoff Collyer (the C News guy) is that "version numbers of related files are not kept in synch, so you can't just cp *';6' /tmp/compile".†† † I tried answering that question myself, but your program kept crashing. †† http://9fans.net/archive/2001/11/759 http://9fans.net/archive/2001/11/759
- artagnon 14y ago
  Revisions of individual files only. Yeah, you can't do that kind of sorcery with phoenixfs. † Fixed. Sorry about that.
spartango 14y ago
You should be able to achieve this with Filesystem events (inotify/kqueue/filesystemwatcher). Just another way of doing things. I'm not the biggest fan of FUSE filesystems(until highly-tuned, ala NTFS-3G), but they certainly do work.
- halayli 14y ago
  No, you cannot achieve atomicity using these calls. By the time you receive a notification the file might have been edited twice.
  - gwern 14y ago
    If the file is changing that fast, do we really care? I'd be happy to have all my files tracked at only 5s temporal resolution...
    - rhizome 14y ago
      Sure, but which 5s? :)
  - spartango 14y ago
    This is true, but not without drawbacks: * Using FUSE you gain the atomicity through synchronicity; when a write occurs, you stand between the disk and request. In so doing, you introduce the potential for latency (nevermind the latency inherent to FUSE). * Using fsevents, you get notifications asynchronously, so there's no I/O performance hit. You may not get the events right away, but with fsevent APIs there's a bounded queue so you generally don't miss events, unless there are too many and the buffer is overflowed[1,2]. The events don't contain the actual file data, so you will lose intermediate edit data. Thinking about this, it would be cool to exploit a Copy-on-Write filesystem (like ZFS) to scalably add the edit data to fsevent apis. I suppose you could preserve write buffers in memory temporarily instead of CoW, but that would have a nasty memory footprint. [1]http://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher.aspx http://msdn.microsoft.com/en-us/library/system.io.filesystem... - Internal buffer size [2]http://docs.oracle.com/javase/7/docs/api/java/nio/file/WatchService.html http://docs.oracle.com/javase/7/docs/api/java/nio/file/Watch... (see the watchkey queue and overflow)