5 ms·

It's great that many HNers like Syncthing, but personally, I seriously tried it maybe two years ago and didn't have a great experience. I wanted to sync my per

It's great that many HNers like Syncthing, but personally, I seriously tried it maybe two years ago and didn't have a great experience. I wanted to sync my personal files between three computers, two desktops and a notebook, which don't always see each other. At first, this worked great. Obviously, there were conflicts when I accidentally edited something in two places without syncing up. What I didn't figure out was how to permanently resolve those. I ran into endless cycles were I had a conflict between two systems, resolved that, but the third one hadn't seen the conflict resolution, which produced another conflict, which I resolved, which the first system hadn't seen yet, ad infinitum. At least, that's what it looked like. Among other errors, files long since deleted on all systems would still show up as conflicts. To be fair, I didn't actually lose any data, and it's very possible I did something wrong that would be obvious to other people. But to me, it wasn't at all clear what and why, which scared me enough to move away from this setup. I don't want to take a lot of chances with my data.

austincheney 5y ago
If you would like to help with an alternative I plan to add syncing as a new feature to my networked file system application soon. The application is already cross OS and feature a familiar GUI and fully recursive directory hashing with ignore lists (ignore lists are only available using terminal commands at this time). It seems like synchronization is the next logic step and I could really use feedback and testing from interested users. The application is available at https://github.com/prettydiff/share-file-systems https://github.com/prettydiff/share-file-systems I would love for syncing to occur upon changes in real time as opposed to timed intervals like a cron job.
Siira 5y ago
What do you use now?
deleted 5y ago
[deleted]
js8 5y ago
Not really sure if this is required for Syncthing but I would (cautiously) set it up so that there are no cycles between the shares. So if you have 3 machines that you need to sync, designate one as a hub which syncs with the other two, and the other two do not see each other. If you absolutely need to transfer data between two others as well, set up another completely independent share between them (a different directory on both machines). I am not sure but fully automated replication setup with cycles might be impossible due to https://en.wikipedia.org/wiki/CAP_theorem https://en.wikipedia.org/wiki/CAP_theorem. (Addendum: I think actually what is needed is even stronger - you should make sure before editing files that the designated "central hub" node is available and has been synced.)
jwalton 5y ago
I’ve lost data from SyncThing. I had a holder called “backgrounds” full of jpgs on my PC. I set it up to mirror the parent of that folder to my Linux machine and my Mac, and everything synced and was happy. I’m still not entirely sure what went wrong by at some point one of the devices decided the folder name should be “Backgrounds” with a capital B (certainly not a change I made) and then all nodes deleted all the files in this folder and then started streaming errors to the logs that they couldn’t replicate all the files in either backgrounds or Backgrounds because none of the online nodes had a copy. SyncThing is very handy, but not something I’d use as a backup solution.
- hda2 5y ago
  I'm having a hard time understanding how you lost data. Are you saying that nodes other than the one with the renamed folder are refusing to resync the renamed folder? And wouldn't the node with the renamed folder still have all your files? Can you explain further?
  - jwalton 5y ago
    I found this a long time after it actually happened, and long after whatever logs from the actual event were gone. When I found it, all nodes claimed they were out of sync, and all of them were complaining that they wanted to sync the folder "Backgrounds" but that it didn't exist on any online node. None of the machines had a folder named "Backgrounds" or "backgrounds" anymore. I imagine what happened is that this was somehow related to the fact that the PC and Mac have case-insensitive file systems, and the Linux machine does not. Somehow one of these machines renamed the folder (I have no idea how this could have happened). I would hazard to guess that SyncThing on one of the nodes (probably Linux) decided it needed to delete the "backgrounds" folder because it no longer existed, and then sync the new "Backgrounds" folder. If deleting the "backgrounds" folder was synchronized back to the PC or Mac though, they would have also deleted the "Backgrounds" folder, since in case-insensitive-land, they're the same folder. I went digging through their github at the time to see if there was an issue about case sensitivity and didn't find anything that was relevant and open. I didn't have a lot to go on, and my attempts to recreate it failed at the time, so I didn't bother opening an issue. It's entirely possible my specific issue is fixed by now.

austincheney 5y ago

If you would like to help with an alternative I plan to add syncing as a new feature to my networked file system application soon. The application is already cross OS and feature a familiar GUI and fully recursive directory hashing with ignore lists (ignore lists are only available using terminal commands at this time). It seems like synchronization is the next logic step and I could really use feedback and testing from interested users. The application is available at https://github.com/prettydiff/share-file-systems https://github.com/prettydiff/share-file-systems I would love for syncing to occur upon changes in real time as opposed to timed intervals like a cron job.

Siira 5y ago

What do you use now?

deleted 5y ago

[deleted]

js8 5y ago

Not really sure if this is required for Syncthing but I would (cautiously) set it up so that there are no cycles between the shares. So if you have 3 machines that you need to sync, designate one as a hub which syncs with the other two, and the other two do not see each other. If you absolutely need to transfer data between two others as well, set up another completely independent share between them (a different directory on both machines). I am not sure but fully automated replication setup with cycles might be impossible due to https://en.wikipedia.org/wiki/CAP_theorem https://en.wikipedia.org/wiki/CAP_theorem. (Addendum: I think actually what is needed is even stronger - you should make sure before editing files that the designated "central hub" node is available and has been synced.)

jwalton 5y ago

I’ve lost data from SyncThing. I had a holder called “backgrounds” full of jpgs on my PC. I set it up to mirror the parent of that folder to my Linux machine and my Mac, and everything synced and was happy. I’m still not entirely sure what went wrong by at some point one of the devices decided the folder name should be “Backgrounds” with a capital B (certainly not a change I made) and then all nodes deleted all the files in this folder and then started streaming errors to the logs that they couldn’t replicate all the files in either backgrounds or Backgrounds because none of the online nodes had a copy. SyncThing is very handy, but not something I’d use as a backup solution.

hda2 5y ago
I'm having a hard time understanding how you lost data. Are you saying that nodes other than the one with the renamed folder are refusing to resync the renamed folder? And wouldn't the node with the renamed folder still have all your files? Can you explain further?
- jwalton 5y ago
  I found this a long time after it actually happened, and long after whatever logs from the actual event were gone. When I found it, all nodes claimed they were out of sync, and all of them were complaining that they wanted to sync the folder "Backgrounds" but that it didn't exist on any online node. None of the machines had a folder named "Backgrounds" or "backgrounds" anymore. I imagine what happened is that this was somehow related to the fact that the PC and Mac have case-insensitive file systems, and the Linux machine does not. Somehow one of these machines renamed the folder (I have no idea how this could have happened). I would hazard to guess that SyncThing on one of the nodes (probably Linux) decided it needed to delete the "backgrounds" folder because it no longer existed, and then sync the new "Backgrounds" folder. If deleting the "backgrounds" folder was synchronized back to the PC or Mac though, they would have also deleted the "Backgrounds" folder, since in case-insensitive-land, they're the same folder. I went digging through their github at the time to see if there was an issue about case sensitivity and didn't find anything that was relevant and open. I didn't have a lot to go on, and my attempts to recreate it failed at the time, so I didn't bother opening an issue. It's entirely possible my specific issue is fixed by now.