3 ms·
In the main I agree, but 2 caveats here: 1) The interface cannot be magically guaranteed as appropriate. In this case, Dropbox implicitly believes they need ca
by roenxi 8y ago
In the main I agree, but 2 caveats here:
1) The interface cannot be magically guaranteed as appropriate. In this case, Dropbox implicitly believes they need capabilities that the standard filesystem interface is not providing.
2) If the differences between filesystems were not interesting, we wouldn't have different filesystems. ZFS and ext4 have to be different, and hence incompatible, in some interesting way, or why have both?
- Dylan16807 8y agoThey have different performance metrics, they act differently when you unplug them, and they have different administrative features. Only the power loss scenario is something an application should ever care about, and even then it would either be "requires ZFS" or "doesn't care".
- userbinator 8y agoIf the differences between filesystems were not interesting, we wouldn't have different filesystems. The differences between filesystems are largely implementation differences, not interface ones. They all essentially provide a way to store hierarchically named streams of bytes. If it was something like a formatter, fscker, defragger or similar utility I could definitely see the reason behind "only X filesystems are supported" --- they need to access the low-level data structures, but (disclaimer: haven't used it before) AFAIK all that Dropbox does is sync changes in files between your local filesystem and some cloud server --- which doesn't seem like the sort of functionality that requires anything more than the basic FS API. (If it hasn't already been done, someone should write their own client, or patch out a stupid check and show that, yes, Dropbox does work on just about any filesystem.)
- pllbnk 8y agoI can talk from personal experience that usually one of the two problems appear when developing cross-platform products: 1. The documentation states that the API is 99% the same and should be sufficient to write most applications except the most advanced ones. And then you hit that one difference which is required for feature parity between platforms which requires either a small hack or a massive workaround which will have to be supported indefinitely. 2. Even if the APIs are the same, the performance characteristics of the systems are different because some of those systems might not be meant to satisfy the same use cases as others and different workarounds are required for cross-platform support. Dropbox desktop application might look simple from the outside, but it must contain some advanced features that its user base might require: directory exclusion from syncing, real-time file sync progress monitoring, displaying the UI status of the synchronization, real-time file change monitoring (as opposed to scanning the directory periodically for modification dates or content changes in the worst case) and many others I couldn't think of right now. So, basically, you can almost always find some open-source alternative for your needs but it will probably only be tested on one platform and will probably contain only a subset of features. I recently had a hard time finding a good enough implementation for Google Drive for Ubuntu.
- nottorp 8y agoHmm. So what the hell is Dropbox doing to my file system if they need those advanced features? All they should need is to be notified when a file changes and to be able to read and write said files?
- zaarn 8y agoExamples: The filesize reported by stat() may or may not have an relation to actual disk space consumed. Counting all files and adding up the stat() filesize, accounting for hardlinks, symlinks and all other normal unix weirdness including Sparse files, you could still find yourself a few bytes up to gigabytes off. In dropbox' case the two suspected culprits are f_fsid and xattr, both of which have different behaviour on different FS'. f_fsid is assumed/documented stable and it is. For varying definitions of "stable", where some filesystems mean "stable for each mount" or "stable for this device" or "stable for filesystem forever". xattrs have varying limits on how much data you can put in and how many xattrs you can have and how they are handled for hardlinks (Linux says 255 byte for the name and 64KiB for the data, XFS and Reiser have that limit, ext4 and btrfs limit the entire xattr to 4KiB).
- fao_ 8y agoxattr is arguably worse for NTFS. While there is no limit to NTFS extended attributes, they are represented transparently as a separate file on the filesystem. Unlike where on linux all tools will suffice to copy xattrs because they are part of the file, on Windows, a copy tool needs to be aware of xattrs or they will be lost. Relying on f_fsid as an actual filesystem id is batshit stupid. It is only "guaranteed" (And, bear in mind, POSIX doesn't specify anything in relation to f_fsid, so I have no idea what authority is guaranteeing this claim) that f_fsid+inode creates a uniquely identifying pair. Aside from that, a filesystem could fill it with literally anything to assure that. As the linux documentation says: "Nobody knows what f_fsid is supposed to contain" See, the Linux documentation on statvfs: The general idea is that f_fsid contains some random stuff such that the pair (f_fsid,ino) uniquely determines a file. Some operating systems use (a variation on) the device number, or the device number combined with the filesystem type. Several operating systems restrict giving out the f_fsid field to the superuser only (and zero it for unprivileged users), because this field is used in the filehandle of the filesystem when NFS-exported, and giving it out is a security concern.
- magicalhippo 8y ago> AFAIK all that Dropbox does is sync changes in files between your local filesystem and some cloud server --- which doesn't seem like the sort of functionality that requires anything more than the basic FS API. The reason syncing requires more than basic FS API is because it's simply won't scale well beyond a handful of files. You need a highly efficient way to monitor file and directory changes, and that's typically not present in a basic FS API. Case point, I was investigating using restic for backups after Crashplan pulled out of the consumer market, however it uses simple directory and file traversal (basic FS API stuff), which makes it take something like 30-40 minutes to just scan my directory tree and find out nothing changed. Crashplan monitored the filesystem and could perform a backup in seconds.