3 ms·

I've been involved in a few consumer-oriented file synchronization projects, and I can assure you that the client technology is not at all simple or easy. Consi

by wrs 16y ago

I've been involved in a few consumer-oriented file synchronization projects, and I can assure you that the client technology is not at all simple or easy. Consider that all existing applications have no idea that you're changing the semantics of the filesystem, but you have to make them act in a way that makes sense to the end user (not a filesystem expert). Not to mention all the little issues of conflicts, deletion, renaming, crazy stuff that apps do for "atomic save", and on and on. It's a combination of having to understand app-filesystem interactions as well as user expectations, both of which are largely undocumented and unspoken -- but if you mess with either in the slightest way, somebody will freak out.

huhtenberg 16y ago
> crazy stuff that apps do for "atomic save" Care for an example?
- wrs 16y ago
  It's been a while since I worked on this, but if I recall correctly...here's an example: Some Windows apps save by (1) writing a new file, (2) deleting the old file, (3) renaming new to old. The idea is that if it dies in the middle, you don't end up with no complete files. This sequence of events is actually recognized in the Windows filesystem code (if it happens quickly enough) and turned into a file-replace operation, which has very different semantics, particularly the effect of copying the metadata (think extended attributes) from old to new. Actually the whole question of what to do with "advanced" filesystem semantics (like extended attributes) is pretty hard, especially if you're trying to sync across platforms where those semantics differ. You could just ignore them, but then if an app actually uses them (and why else would they exist?) then the file won't round-trip through the sync platform.
  - huhtenberg 16y ago
    > This sequence of events is actually recognized in the Windows filesystem code.. and turned into a file-replace operation Woah. I have quite a bit of experience digging through the crap in Windows kernel, but this seems to top all of that. Do you happen to have any references? MSDN, some forum threads, anything? Frankly I just find it hard to believe.
    - wrs 16y ago
      This is documented now, at least for creation times, short names, and file IDs. Apparently the filesystem can choose what to "tunnel" across the delete+rename. http://support.microsoft.com/kb/172190 http://support.microsoft.com/kb/172190 http://www.osronline.com/article.cfm?id=22 http://www.osronline.com/article.cfm?id=22 History: http://blogs.msdn.com/b/oldnewthing/archive/2005/07/15/439261.aspx http://blogs.msdn.com/b/oldnewthing/archive/2005/07/15/43926...
      - huhtenberg 16y ago
        Reading... thanks.

by wrs 16y ago

huhtenberg 16y ago

> crazy stuff that apps do for "atomic save" Care for an example?

wrs 16y ago
It's been a while since I worked on this, but if I recall correctly...here's an example: Some Windows apps save by (1) writing a new file, (2) deleting the old file, (3) renaming new to old. The idea is that if it dies in the middle, you don't end up with no complete files. This sequence of events is actually recognized in the Windows filesystem code (if it happens quickly enough) and turned into a file-replace operation, which has very different semantics, particularly the effect of copying the metadata (think extended attributes) from old to new. Actually the whole question of what to do with "advanced" filesystem semantics (like extended attributes) is pretty hard, especially if you're trying to sync across platforms where those semantics differ. You could just ignore them, but then if an app actually uses them (and why else would they exist?) then the file won't round-trip through the sync platform.
- huhtenberg 16y ago
  > This sequence of events is actually recognized in the Windows filesystem code.. and turned into a file-replace operation Woah. I have quite a bit of experience digging through the crap in Windows kernel, but this seems to top all of that. Do you happen to have any references? MSDN, some forum threads, anything? Frankly I just find it hard to believe.
  - wrs 16y ago
    This is documented now, at least for creation times, short names, and file IDs. Apparently the filesystem can choose what to "tunnel" across the delete+rename. http://support.microsoft.com/kb/172190 http://support.microsoft.com/kb/172190 http://www.osronline.com/article.cfm?id=22 http://www.osronline.com/article.cfm?id=22 History: http://blogs.msdn.com/b/oldnewthing/archive/2005/07/15/439261.aspx http://blogs.msdn.com/b/oldnewthing/archive/2005/07/15/43926...
    - huhtenberg 16y ago
      Reading... thanks.