4 ms·

For the reasons that you've specified and others, I rolled my own very simple sync system based on Git. The biggest problem I personally have with Dropbbox et.

For the reasons that you've specified and others, I rolled my own very simple sync system based on Git. The biggest problem I personally have with Dropbbox et. al, is that none of the commercial solutions sync symbolic links opaquely. ("Opaque" syncing of symlinks means to sync the links themselves and not what the symlinks link to.) I use symlinks heavily and so I absolutely require this feature. Most of the commercial solutions just ignore symlinks, but Dropbox does the worst thing possible and treats symlinks transparently. This is utterly wrong and, in fact, downright dangerous! Another feature that I need for telecommuting software development is to be able to exclude artifacts from what is synced. Eclipse, for instance, constantly churns out artifacts, and I certainly don't want or need those artifacts to be synced. Two problems with my roll-your-own little syncing system remain: (1) The large file issue that you mentioned. (Sparkleshare is much worse in this regard than my system, last I tried.) (2) I know of no way to have the Git instance that I use for syncing not pay attention to the .gitignore files that are used by the Git instance that I use for version control. Annoying! In any case, I'm very glad to hear that BitTorrent seems to be coming up with a solution that will address all my issues, and I won't have to roll my own anymore.

aninhumer 13y ago
What does Dropbox do in the case of a symlink circular reference?
- jos3000 13y ago
  I can answer this question since I accidentally created a circular reference the other day. It starts syncing the recursive structure until it runs out of quota.
  - nessus42 13y ago
    > It starts syncing the recursive structure until it runs out of quota. I was curious about this myself, but I never tried it, as I feared that this was precisely the badness that would occur. Dropbox is so good in so many respects, I just can't understand how they could have made such an utterly terrible and wrong decision on this particular issue.
veidr 13y ago
You are right: Dropbox's behavior with regard to symlinks is fundamentally wrong. It is kinda lame that other services, e.g. Google Drive, just skip symlinks, but that isn't as dangerous. The hierarchical structure of a directory tree is often effectively part of an application's data, and Dropbox silently corrupts that data. This can cause crashes, or multiple conflicting copies of not-easily-merged data to be strewn about in different places. It's also hard to initially notice how broken it is, since symlinks work on the first machine where Dropbox encounters them, but then on all subsequent client devices that Dropbox sees are replaced with some version of whatever data they point to.
- nessus42 13y ago
  > You are right: Dropbox's behavior with regard to symlinks is fundamentally wrong. What's even worse, is that no matter how hard I try, I am not able to convince most people of how wrong this is--even those who claim to be and should be computer savy. E.g., on support forums for other sync services, users are typically clamoring for Dropbox-style syncing of symlinks, and they will not be convinced otherwise, in spite of all reason. And despite 30 years of hard-earned experience with symlinks that irrefutably demonstrates that transparent syncing of symlinks is nothing but badness. But thank goodness that BitTorrent got it right!
  - davidjade 13y ago
    Yeah but the real problem, the ones users are trying to solve is the limits on having to move everything into one sync folder. Transparent symlinks are the cheap fix for them so that they can keep their files where they want and still have them sync. The better solution would be to just allow any folder(s) to be synced but none of the major cloud sync providers seem to want to enable that.
    - aroemers 13y ago
      Have a look at https://www.cubby.com/ https://www.cubby.com/. It syncs existing folders you want to, and per synced folder you can decide to use their cloud or sync directly.
neolefty 13y ago
Sounds like it's time for a "design pattern" or "best practice" -- that is, covering up for a tool's faults. What if you treat DropBox as a raw, strict hierarchical store and never use it directly, but only link into it from elsewhere? Move all the links outside of it. ln ~/Dropbox/photos ~/Photos ln ~/Dropbox/family/baby/cute-movies ~/Photos/baby-cute-movies etc.
- nessus42 13y ago
  (1) You just put a symlink into your Dropbox folder with the above sequence of commands, which just serves to illustrate the danger of this approach. (2) Your suggestion is extremely difficult or impossible to do in the general case. (3) I have better things to do with my life than rearrange my entire filesystem to suit the ill-conceived whims of a product that does the wrong thing. (4) I would like to sync parts of my filesystem as they currently are. (5) I have my own syncing solution that works better for my needs, and it only took me a couple of hours to throw together. (6) I'm going to use BitTorrent Sync in the future for most of my syncing needs (as long as it turns out to be reliable) since it does syncing properly in the face of symlinks. (7) I'll still use Dropbox for simple sharing of files with friends and getting files onto my iPad. Apparently, Dropbox just doesn't care about the needs of more sophisticated users, as that isn't a very large market.
hrktb 13y ago
> The biggest problem I personally have with Dropbbox et. al, is that none of the commercial solutions sync symbolic links opaquely The tradeoff is to have all your stuff in one place, with is not a good thing most of the time IMO. A frequent problem I stumble on: there is a work folder where everything is already on svn/git, but I'd want to temporarily sync a few folders to dropbox only when needed, and get them out of dropbox when finished. As dropbox works now, to do this without using symlinks would be a PITA (I'd have to put all my files in the dropbox folder by default, and then play with the selective sync ? I am not even sure there is a way to do it without moving files around). If dropbox had support for arbitrary multiple folder selection (but the the complexity would go way up?) I'd agree with your stance on symlink, but now this is more of a necessary evil.
- nessus42 13y ago
  > If dropbox had support for arbitrary multiple folder selection (but the the complexity would go way up?) I'd agree with your stance on symlink, but now this is more of a necessary evil. I think you already answered your own objection on the right way to handle this need. Other syncing services, including BitTorrent Sync handle your need in precisely this manner. I.e., the right way. Dangerous, frustrating, and incorrect abuse of symlinks is NOT the right way to do this.

aninhumer 13y ago

What does Dropbox do in the case of a symlink circular reference?

jos3000 13y ago
I can answer this question since I accidentally created a circular reference the other day. It starts syncing the recursive structure until it runs out of quota.
- nessus42 13y ago
  > It starts syncing the recursive structure until it runs out of quota. I was curious about this myself, but I never tried it, as I feared that this was precisely the badness that would occur. Dropbox is so good in so many respects, I just can't understand how they could have made such an utterly terrible and wrong decision on this particular issue.

veidr 13y ago

You are right: Dropbox's behavior with regard to symlinks is fundamentally wrong. It is kinda lame that other services, e.g. Google Drive, just skip symlinks, but that isn't as dangerous. The hierarchical structure of a directory tree is often effectively part of an application's data, and Dropbox silently corrupts that data. This can cause crashes, or multiple conflicting copies of not-easily-merged data to be strewn about in different places. It's also hard to initially notice how broken it is, since symlinks work on the first machine where Dropbox encounters them, but then on all subsequent client devices that Dropbox sees are replaced with some version of whatever data they point to.

nessus42 13y ago
> You are right: Dropbox's behavior with regard to symlinks is fundamentally wrong. What's even worse, is that no matter how hard I try, I am not able to convince most people of how wrong this is--even those who claim to be and should be computer savy. E.g., on support forums for other sync services, users are typically clamoring for Dropbox-style syncing of symlinks, and they will not be convinced otherwise, in spite of all reason. And despite 30 years of hard-earned experience with symlinks that irrefutably demonstrates that transparent syncing of symlinks is nothing but badness. But thank goodness that BitTorrent got it right!
- davidjade 13y ago
  Yeah but the real problem, the ones users are trying to solve is the limits on having to move everything into one sync folder. Transparent symlinks are the cheap fix for them so that they can keep their files where they want and still have them sync. The better solution would be to just allow any folder(s) to be synced but none of the major cloud sync providers seem to want to enable that.
  - aroemers 13y ago
    Have a look at https://www.cubby.com/ https://www.cubby.com/. It syncs existing folders you want to, and per synced folder you can decide to use their cloud or sync directly.

neolefty 13y ago

Sounds like it's time for a "design pattern" or "best practice" -- that is, covering up for a tool's faults. What if you treat DropBox as a raw, strict hierarchical store and never use it directly, but only link into it from elsewhere? Move all the links outside of it. ln ~/Dropbox/photos ~/Photos ln ~/Dropbox/family/baby/cute-movies ~/Photos/baby-cute-movies etc.

nessus42 13y ago
(1) You just put a symlink into your Dropbox folder with the above sequence of commands, which just serves to illustrate the danger of this approach. (2) Your suggestion is extremely difficult or impossible to do in the general case. (3) I have better things to do with my life than rearrange my entire filesystem to suit the ill-conceived whims of a product that does the wrong thing. (4) I would like to sync parts of my filesystem as they currently are. (5) I have my own syncing solution that works better for my needs, and it only took me a couple of hours to throw together. (6) I'm going to use BitTorrent Sync in the future for most of my syncing needs (as long as it turns out to be reliable) since it does syncing properly in the face of symlinks. (7) I'll still use Dropbox for simple sharing of files with friends and getting files onto my iPad. Apparently, Dropbox just doesn't care about the needs of more sophisticated users, as that isn't a very large market.

hrktb 13y ago

> The biggest problem I personally have with Dropbbox et. al, is that none of the commercial solutions sync symbolic links opaquely The tradeoff is to have all your stuff in one place, with is not a good thing most of the time IMO. A frequent problem I stumble on: there is a work folder where everything is already on svn/git, but I'd want to temporarily sync a few folders to dropbox only when needed, and get them out of dropbox when finished. As dropbox works now, to do this without using symlinks would be a PITA (I'd have to put all my files in the dropbox folder by default, and then play with the selective sync ? I am not even sure there is a way to do it without moving files around). If dropbox had support for arbitrary multiple folder selection (but the the complexity would go way up?) I'd agree with your stance on symlink, but now this is more of a necessary evil.

nessus42 13y ago
> If dropbox had support for arbitrary multiple folder selection (but the the complexity would go way up?) I'd agree with your stance on symlink, but now this is more of a necessary evil. I think you already answered your own objection on the right way to handle this need. Other syncing services, including BitTorrent Sync handle your need in precisely this manner. I.e., the right way. Dangerous, frustrating, and incorrect abuse of symlinks is NOT the right way to do this.