3 ms·

You should be able to achieve this with Filesystem events (inotify/kqueue/filesystemwatcher). Just another way of doing things. I'm not the biggest fan of FUSE

You should be able to achieve this with Filesystem events (inotify/kqueue/filesystemwatcher). Just another way of doing things. I'm not the biggest fan of FUSE filesystems(until highly-tuned, ala NTFS-3G), but they certainly do work.

halayli 14y ago
No, you cannot achieve atomicity using these calls. By the time you receive a notification the file might have been edited twice.
- gwern 14y ago
  If the file is changing that fast, do we really care? I'd be happy to have all my files tracked at only 5s temporal resolution...
  - rhizome 14y ago
    Sure, but which 5s? :)
- spartango 14y ago
  This is true, but not without drawbacks: * Using FUSE you gain the atomicity through synchronicity; when a write occurs, you stand between the disk and request. In so doing, you introduce the potential for latency (nevermind the latency inherent to FUSE). * Using fsevents, you get notifications asynchronously, so there's no I/O performance hit. You may not get the events right away, but with fsevent APIs there's a bounded queue so you generally don't miss events, unless there are too many and the buffer is overflowed[1,2]. The events don't contain the actual file data, so you will lose intermediate edit data. Thinking about this, it would be cool to exploit a Copy-on-Write filesystem (like ZFS) to scalably add the edit data to fsevent apis. I suppose you could preserve write buffers in memory temporarily instead of CoW, but that would have a nasty memory footprint. [1]http://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher.aspx http://msdn.microsoft.com/en-us/library/system.io.filesystem... - Internal buffer size [2]http://docs.oracle.com/javase/7/docs/api/java/nio/file/WatchService.html http://docs.oracle.com/javase/7/docs/api/java/nio/file/Watch... (see the watchkey queue and overflow)

3 ms·

You should be able to achieve this with Filesystem events (inotify/kqueue/filesystemwatcher). Just another way of doing things. I'm not the biggest fan of FUSE

by spartango 14y ago

halayli 14y ago
No, you cannot achieve atomicity using these calls. By the time you receive a notification the file might have been edited twice.
- gwern 14y ago
  If the file is changing that fast, do we really care? I'd be happy to have all my files tracked at only 5s temporal resolution...
  - rhizome 14y ago
    Sure, but which 5s? :)
- spartango 14y ago
  This is true, but not without drawbacks: * Using FUSE you gain the atomicity through synchronicity; when a write occurs, you stand between the disk and request. In so doing, you introduce the potential for latency (nevermind the latency inherent to FUSE). * Using fsevents, you get notifications asynchronously, so there's no I/O performance hit. You may not get the events right away, but with fsevent APIs there's a bounded queue so you generally don't miss events, unless there are too many and the buffer is overflowed[1,2]. The events don't contain the actual file data, so you will lose intermediate edit data. Thinking about this, it would be cool to exploit a Copy-on-Write filesystem (like ZFS) to scalably add the edit data to fsevent apis. I suppose you could preserve write buffers in memory temporarily instead of CoW, but that would have a nasty memory footprint. [1]http://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher.aspx http://msdn.microsoft.com/en-us/library/system.io.filesystem... - Internal buffer size [2]http://docs.oracle.com/javase/7/docs/api/java/nio/file/WatchService.html http://docs.oracle.com/javase/7/docs/api/java/nio/file/Watch... (see the watchkey queue and overflow)