46 ms·

I did work on backup solutions and supported Dropbox, and I think that the author used Dropbox as a reference to support his own concerns about filesystems that

I did work on backup solutions and supported Dropbox, and I think that the author used Dropbox as a reference to support his own concerns about filesystems that are not related in any case to the Dropbox case. From the filesystem point of view, I don't think that Dropbox is so much concerned about you losing your data because of corruption. As, anyway they are supposed to be safe and versioned in the cloud. And I think that their app is very simple, ie like no specific 'driver' hack. Their real costly problem is related to metadata! All different file systems support different naming, encoding or character sets for file names. Also, they all have their specificities and limitations regarding the support of extended metadata or user right info. Also some will support file change notification or not, some will have a valid file last modification value that can have different level of decimal precision, etc... So, as Dropbox ambition is that their app can fully backup and restore your system data, supporting backuping from one and restoring to another or the combinatory of all possible for each case is a nightmare. Just a simple example: let suppose that you have a filename with the char "ü". With unicode normalization, this can be stored as a single character or as 2 characters "u" + ".." (letter + particule). Everyone uses the first version, but Mac HFS uses the second form. The crazy thing is that if you try to save a file with the first form in HFS, it will accept but will silently convert the filename. So, let's suppose that it was Dropbox asking to restore such a filename with the letter as a single character, later when listing your files to see if they are in sync, it will see another filename (based on filename bytes) but not the one it expects. So it might want to download it again and again and again if it was not smart.

ChrisSD 7y ago
Yeah, the article used comments on Dropbox as a kicking off point to talk more generally about filesystem issues. The author is aware that in this specific case it was more about metadata: > I believe their hand was forced by the way they want to store/use data, which they can only do with ext, but even if that wasn't the case... The Dropbox examples seem to have clouded the issues actually being discussed.
j4_hnews 7y ago
Also to note, Dropbox was likely inspired by the open source Unison file-synchronization tool. So they have to handle all of the shortcomings and caveats that Unison does not handle out of the box: Caveats: https://www.cis.upenn.edu/~bcpierce/unison/download/releases/stable/unison-manual.html#caveats https://www.cis.upenn.edu/~bcpierce/unison/download/releases... Symbolic links: https://www.cis.upenn.edu/~bcpierce/unison/download/releases/stable/unison-manual.html#symlinks https://www.cis.upenn.edu/~bcpierce/unison/download/releases... Permissions: https://www.cis.upenn.edu/~bcpierce/unison/download/releases/stable/unison-manual.html#perms https://www.cis.upenn.edu/~bcpierce/unison/download/releases... Cross platform issues (unsupported characters across operating systems and file systems): https://www.cis.upenn.edu/~bcpierce/unison/download/releases/stable/unison-manual.html#crossplatform https://www.cis.upenn.edu/~bcpierce/unison/download/releases... Edit: Inspired by the the Unison Spec (published on 2004) which may be why we have conflict files etc. when using Dropbox http://www.cis.upenn.edu/~bcpierce/papers/unisonspec.pdf http://www.cis.upenn.edu/~bcpierce/papers/unisonspec.pdf
- hemancuso 7y ago
  Fairly sure this is not true, unless you have a source. Drew did all the client side work in python/librsync to get going.
  - j4_hnews 7y ago
    Ok, may have been inspired by. Here's some relevant backstory from 2011: https://www.wired.com/2011/12/backdrop-dropbox/ https://www.wired.com/2011/12/backdrop-dropbox/
coldtea 7y ago
>From the filesystem point of view, I don't think that Dropbox is so much concerned about you losing your data because of corruption. As, anyway they are supposed to be safe and versioned in the cloud. They could get to the Cloud corrupted, and then continue replicating in your devices, so no...
- moolcool 7y ago
  That's where the "and versioned" comes in handy
  - coldtea 7y ago
    Versioned wont help if it's the first copy that got into dropbox.
    - greatgib 7y ago
      If it's the first copy going to Dropbox, then it is not Dropbox that wrote it and so not Dropbox that corrupted it by writing in a wrong way...
    - greatgib 7y ago
      If it's the first copy, it will just be read by Dropbox. There will not be 'write' from Dropbox that can corrupt it...
aidos 7y ago
But Dropbox doesn’t even attempt to support the more common cases (pun intended). For example, you put a file in with specific characters in upper case and it might all come out lower case on another (Linux) system. They claim it’s because not all file systems are case sensitive, but they could at least keep the original case and use it when possible. Then again, onedrive syncing break on a whole new level. If you upload a docx file, they’ll open it and edit the meta data so the file you added will be different from the one you get back (actually they do it to images and pdfs too).

ChrisSD 7y ago

Yeah, the article used comments on Dropbox as a kicking off point to talk more generally about filesystem issues. The author is aware that in this specific case it was more about metadata: > I believe their hand was forced by the way they want to store/use data, which they can only do with ext, but even if that wasn't the case... The Dropbox examples seem to have clouded the issues actually being discussed.

j4_hnews 7y ago

Also to note, Dropbox was likely inspired by the open source Unison file-synchronization tool. So they have to handle all of the shortcomings and caveats that Unison does not handle out of the box: Caveats: https://www.cis.upenn.edu/~bcpierce/unison/download/releases/stable/unison-manual.html#caveats https://www.cis.upenn.edu/~bcpierce/unison/download/releases... Symbolic links: https://www.cis.upenn.edu/~bcpierce/unison/download/releases/stable/unison-manual.html#symlinks https://www.cis.upenn.edu/~bcpierce/unison/download/releases... Permissions: https://www.cis.upenn.edu/~bcpierce/unison/download/releases/stable/unison-manual.html#perms https://www.cis.upenn.edu/~bcpierce/unison/download/releases... Cross platform issues (unsupported characters across operating systems and file systems): https://www.cis.upenn.edu/~bcpierce/unison/download/releases/stable/unison-manual.html#crossplatform https://www.cis.upenn.edu/~bcpierce/unison/download/releases... Edit: Inspired by the the Unison Spec (published on 2004) which may be why we have conflict files etc. when using Dropbox http://www.cis.upenn.edu/~bcpierce/papers/unisonspec.pdf http://www.cis.upenn.edu/~bcpierce/papers/unisonspec.pdf

hemancuso 7y ago
Fairly sure this is not true, unless you have a source. Drew did all the client side work in python/librsync to get going.
- j4_hnews 7y ago
  Ok, may have been inspired by. Here's some relevant backstory from 2011: https://www.wired.com/2011/12/backdrop-dropbox/ https://www.wired.com/2011/12/backdrop-dropbox/

coldtea 7y ago

>From the filesystem point of view, I don't think that Dropbox is so much concerned about you losing your data because of corruption. As, anyway they are supposed to be safe and versioned in the cloud. They could get to the Cloud corrupted, and then continue replicating in your devices, so no...

moolcool 7y ago
That's where the "and versioned" comes in handy
- coldtea 7y ago
  Versioned wont help if it's the first copy that got into dropbox.
  - greatgib 7y ago
    If it's the first copy going to Dropbox, then it is not Dropbox that wrote it and so not Dropbox that corrupted it by writing in a wrong way...
  - greatgib 7y ago
    If it's the first copy, it will just be read by Dropbox. There will not be 'write' from Dropbox that can corrupt it...

aidos 7y ago

But Dropbox doesn’t even attempt to support the more common cases (pun intended). For example, you put a file in with specific characters in upper case and it might all come out lower case on another (Linux) system. They claim it’s because not all file systems are case sensitive, but they could at least keep the original case and use it when possible. Then again, onedrive syncing break on a whole new level. If you upload a docx file, they’ll open it and edit the meta data so the file you added will be different from the one you get back (actually they do it to images and pdfs too).