10 ms·

Syncthing is one of the "fire and forget" kind of programs. It takes some time to setup when doing for first time, but then you pretty much forget its there, an

Syncthing is one of the "fire and forget" kind of programs. It takes some time to setup when doing for first time, but then you pretty much forget its there, and when you need it, you realise it has been chugging along doing its job. Two things I like about it in particular, are 1. It can and favors syncing over local network. This has a huge cost savings in developing world. Even when you do have massive bandwidth, local network sync still has more throughput and lower latency. My music is shared between devices and if I add a track, it takes less than a second to appear on my other devices. 2. You can set conditions, such that delets can be ignored. Eg. I have a WhatsApp message+media backup going back almost 7 years now, ~65GiB. But I don't need all that on my phone. So I just disable syncing delets on my storage. Now my phone can get away with ~3 GiB of whatsapp data (only because I don't delete very often) and I still have complete backup in case I need it. 3. Absolute rock solid stable. At $DAYJOB we use syncthing to sync multi dozen GiBs of new data every day, total file count in 10s of thousands in about ~300 directories. And in past 5 years we have had about 3 instances where we had something that needed attention, out of which 2 were not syncthing's fault. I've been using Syncthing for years and have absolutely zero complaints against it. Everyone with 2 or more devices should give it a try, just to see there are better options out there.

NortySpock 5y ago
> 2. You can set conditions, such that delet[e]s can be ignored. I was evaluating Syncthing for 1-way, append-only phone photo backup, and my Google searches warned me away from this, talking about "unsupported" and "database corruption." I assume you have not experienced this, but I guess I should have just read the docs and trusted them, instead of forum posts. https://docs.syncthing.net/advanced/folder-ignoredelete.html https://docs.syncthing.net/advanced/folder-ignoredelete.html
- e3bc54b2 5y ago
  I have been using that feature for, I don't even remember how long, but definitely since it was marked beta or something. I am not a heavyweight like some other in this thread, but my current local state is 70GB and ~30k files, global state is ~50GB (I need to prune WhatsApp on phone again) and ~10k files. I am no biggie, but its not a slouch to keep all this working in 7 devices, with overlapping directories shared between them, across 2 timezones, over local and public network. Zero complaints is exactly my experience. Syncthing Just Works™
  - egeozcan 5y ago
    Great, because even 1 complaint there would be too many, wouldn't it?
    - e3bc54b2 5y ago
      I agree. And that's my gripe with Dropbox, Drive, OneDrive (the big 3 that I was able to try, no apple devices). Every one of them has shit the bed on me, on more than one occasion. That's after burning my bandwidth and taking away my privacy. Syncthing just wins out every way (as long as problem scope is file sync, not cloud storage or sharing).
- egeozcan 5y ago
  That scared me too. I just move what I want to delete from the original sync folder over to another folder, causing the phone data to be deleted, I even wrote a script that "partitions" that old data, also syncing to some random server after being fully encrypted (that server ofc doesn't have decryption keys).
  - XorNot 5y ago
    In practice you could just turn on Trash File Versioning with infinite retention too.
- stewbrew 5y ago
  i had some of these problems with earlier versions. removing the database (after making sure everything is on sync) usually solved the issue. but that's years ago.
- sbam 5y ago
  >I was evaluating Syncthing for 1-way, append-only phone photo backup, and my Google searches warned me away from this, talking about "unsupported" and "database corruption." Same here. What do you use instead? I haven't been able to find a good open-source multiplatform project for backups/1-way sync.
  - NortySpock 5y ago
    It's not open-source, but I ended up using PhotoSync for myself and my wife to a locked-down samba share on my LAN . They had a pretty reasonable (~$6) in-app purchase to unlock samba access, and the fact that it's a German GmbH means I'm slightly more confident they're not trying to copy all our photos to a dark data-vault. Happy user so far, though it took some cross-checking against the samba share feel confident it was backing up reliably (I'm just particular about trust-but-verify :) https://www.photosync-app.com/home.html https://www.photosync-app.com/home.html
  - blooalien 5y ago
    I have several folders in SyncThing set for one-way only sync (from one device or machine to another only, never syncing back). It's easy to set various nifty folder options in the SyncThing web UI and on the Android app. (Dunno about Apple devices, because I own none.)
srinathkrishna 5y ago
Excellent idea to sync Whatsapp. Do you use any means to read your Whatsapp chat history on other devices - say your laptop?
- e3bc54b2 5y ago
  There are a few applications that can read WhatsApp database. I used one couple years ago to run some analytics on my chat, but don't remember now. I'm fairly certain it is still around and works, though.
api 5y ago
Tangent but: I sometimes feel like programs that just work like this don't get traction because they don't provoke as much discussion, search volume, etc. A complex mess that needs to be constantly babysat creates a cottage industry around it and gets discussed a lot, but something that works is like the quiet kid who sits in the corner and gets A's and that everyone ignores Kubernetes is today's poster child for this. The Hashicorp stack can do just about all the same things, but it just works so there's no cottage industry of consultants and no market for as-a-service implementations. Why promote something like that?
- e3bc54b2 5y ago
  > programs that just work like this don't get traction because they don't provoke as much discussion. Agree very much. That's why I never let go of a chance to enavgelise Syncthing and its OSS ilk in other fields.
orblivion 5y ago
I can come up with a couple issues related to Syncthing (whether or not it's about Syncthing per se). 1) I like to minimize the number of APT repositories I use. Debian's and Ubuntu's are well behind on Syncthing versions. Every so often (much more rarely than before), they have breaking changes. As such, my phone can no longer sync with my computers. Perhaps the answer is that Debian should just not be packaging it, because they have such a conservative policy. I'm considering downgrading my phone's version, or building the latest version from source. Worst case I may add their APT repo but I'm not eager. 2) At least with the version I've been using, opting for local-only sync isn't so straightforward. It's per-instance, not per-folder. This means that I have to have my home server running it as local-only, even for small things that I wouldn't mind getting synced over the wire. It means that if I want to have a local-only directory shared between my phone and laptop, it has to go through my home server first. Unless there's a better way I don't know about. Otherwise Syncthing has been pretty great. I like your no-delete thing, I did not consider that. I could definitely use it for photos.
- yunohn 5y ago
  > Worst case I may add their APT repo but I'm not eager. What’s wrong with adding a first party apt repo for syncthing (or other software)?
  - joshu 5y ago
    supply chain attacks
- jedahan 5y ago
  Just checked the debian issue tracker. Not sure if there is supporting work that could move it along, but you are not alone in wanting the latest release https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=983500 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=983500
  - orblivion 5y ago
    I think it would go against Debian's philosophy to move fast enough to keep up with Syncthing.
45ure 5y ago
>2. You can set conditions, such that delets can be ignored. Eg. I have a WhatsApp message+media backup going back almost 7 years now, ~65GiB. But I don't need all that on my phone. So I just disable syncing delets on my storage. Now my phone can get away with ~3 GiB of whatsapp data (only because I don't delete very often) and I still have complete backup in case I need it. I would like to know, how are you using Syncthing to create local backups for WhatsApp database/msgstore (crypt14) and media files. You also mention 65GiB as your complete backup ─ have you tested it via a restore? If so, how?
- swiley 5y ago
  Android used to let you administrate your devices and it was easy to do stuff like this.
  - seized 5y ago
    Is still is easy to do that. I have Syncthing backing up data from my phone easily and reliably.
- cesarb 5y ago
  > I would like to know, how are you using Syncthing to create local backups for WhatsApp database/msgstore (crypt14) and media files. Not the parent poster, but I also do this; I just configured Syncthing to share the /storage/emulated/0/WhatsApp (aka /sdcard/WhatsApp) directory. WhatsApp stores its daily local backup on the Backups subdirectory of that directory, and the media files are all in the Media subdirectory of that directory. > have you tested it via a restore? If so, how? I have actually used it to move all the WhatsApp and Signal data from an old phone to a new phone. Just have Syncthing synchronize these directories (with the same path) on the new phone before installing WhatsApp and Signal, then install and launch WhatsApp and Signal. When first launched, if that directory already contains a backup, both WhatsApp and Signal ask if you want to restore from that backup. Signal then asks you to type a long backup encryption key you should have written down somewhere, while WhatsApp asks its servers for the backup encryption key.
- e3bc54b2 5y ago
Royi 5y ago
Regarding WhatsApp, How do you merge different database files into one coherent big database?
- e3bc54b2 5y ago
  I.. don't. My days of tinkering directly with WA db are over. These days I just want an easy and reliable backup. After restoring, WhatsApp recognizes the files and takes care of the rest.
divbzero 5y ago
What are the “3 instances where we had something that needed attention, out of which 2 were not syncthing's fault”? I’m curious what rare exceptions there are to watch for.
- e3bc54b2 5y ago
  Unfortunately I don't clearly remember because last one was a while ago, but it was Sync conflicts due to external factors. The one where it was Syncthing's fault was years ago and is very likely fixed now.

NortySpock 5y ago

> 2. You can set conditions, such that delet[e]s can be ignored. I was evaluating Syncthing for 1-way, append-only phone photo backup, and my Google searches warned me away from this, talking about "unsupported" and "database corruption." I assume you have not experienced this, but I guess I should have just read the docs and trusted them, instead of forum posts. https://docs.syncthing.net/advanced/folder-ignoredelete.html https://docs.syncthing.net/advanced/folder-ignoredelete.html

e3bc54b2 5y ago
I have been using that feature for, I don't even remember how long, but definitely since it was marked beta or something. I am not a heavyweight like some other in this thread, but my current local state is 70GB and ~30k files, global state is ~50GB (I need to prune WhatsApp on phone again) and ~10k files. I am no biggie, but its not a slouch to keep all this working in 7 devices, with overlapping directories shared between them, across 2 timezones, over local and public network. Zero complaints is exactly my experience. Syncthing Just Works™
- egeozcan 5y ago
  Great, because even 1 complaint there would be too many, wouldn't it?
  - e3bc54b2 5y ago
    I agree. And that's my gripe with Dropbox, Drive, OneDrive (the big 3 that I was able to try, no apple devices). Every one of them has shit the bed on me, on more than one occasion. That's after burning my bandwidth and taking away my privacy. Syncthing just wins out every way (as long as problem scope is file sync, not cloud storage or sharing).
egeozcan 5y ago
That scared me too. I just move what I want to delete from the original sync folder over to another folder, causing the phone data to be deleted, I even wrote a script that "partitions" that old data, also syncing to some random server after being fully encrypted (that server ofc doesn't have decryption keys).
- XorNot 5y ago
  In practice you could just turn on Trash File Versioning with infinite retention too.
stewbrew 5y ago
i had some of these problems with earlier versions. removing the database (after making sure everything is on sync) usually solved the issue. but that's years ago.
sbam 5y ago
>I was evaluating Syncthing for 1-way, append-only phone photo backup, and my Google searches warned me away from this, talking about "unsupported" and "database corruption." Same here. What do you use instead? I haven't been able to find a good open-source multiplatform project for backups/1-way sync.
- NortySpock 5y ago
  It's not open-source, but I ended up using PhotoSync for myself and my wife to a locked-down samba share on my LAN . They had a pretty reasonable (~$6) in-app purchase to unlock samba access, and the fact that it's a German GmbH means I'm slightly more confident they're not trying to copy all our photos to a dark data-vault. Happy user so far, though it took some cross-checking against the samba share feel confident it was backing up reliably (I'm just particular about trust-but-verify :) https://www.photosync-app.com/home.html https://www.photosync-app.com/home.html
- blooalien 5y ago
  I have several folders in SyncThing set for one-way only sync (from one device or machine to another only, never syncing back). It's easy to set various nifty folder options in the SyncThing web UI and on the Android app. (Dunno about Apple devices, because I own none.)

srinathkrishna 5y ago

Excellent idea to sync Whatsapp. Do you use any means to read your Whatsapp chat history on other devices - say your laptop?

e3bc54b2 5y ago
There are a few applications that can read WhatsApp database. I used one couple years ago to run some analytics on my chat, but don't remember now. I'm fairly certain it is still around and works, though.

api 5y ago

Tangent but: I sometimes feel like programs that just work like this don't get traction because they don't provoke as much discussion, search volume, etc. A complex mess that needs to be constantly babysat creates a cottage industry around it and gets discussed a lot, but something that works is like the quiet kid who sits in the corner and gets A's and that everyone ignores Kubernetes is today's poster child for this. The Hashicorp stack can do just about all the same things, but it just works so there's no cottage industry of consultants and no market for as-a-service implementations. Why promote something like that?

e3bc54b2 5y ago
> programs that just work like this don't get traction because they don't provoke as much discussion. Agree very much. That's why I never let go of a chance to enavgelise Syncthing and its OSS ilk in other fields.

orblivion 5y ago

I can come up with a couple issues related to Syncthing (whether or not it's about Syncthing per se). 1) I like to minimize the number of APT repositories I use. Debian's and Ubuntu's are well behind on Syncthing versions. Every so often (much more rarely than before), they have breaking changes. As such, my phone can no longer sync with my computers. Perhaps the answer is that Debian should just not be packaging it, because they have such a conservative policy. I'm considering downgrading my phone's version, or building the latest version from source. Worst case I may add their APT repo but I'm not eager. 2) At least with the version I've been using, opting for local-only sync isn't so straightforward. It's per-instance, not per-folder. This means that I have to have my home server running it as local-only, even for small things that I wouldn't mind getting synced over the wire. It means that if I want to have a local-only directory shared between my phone and laptop, it has to go through my home server first. Unless there's a better way I don't know about. Otherwise Syncthing has been pretty great. I like your no-delete thing, I did not consider that. I could definitely use it for photos.

yunohn 5y ago
> Worst case I may add their APT repo but I'm not eager. What’s wrong with adding a first party apt repo for syncthing (or other software)?
- joshu 5y ago
  supply chain attacks
jedahan 5y ago
Just checked the debian issue tracker. Not sure if there is supporting work that could move it along, but you are not alone in wanting the latest release https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=983500 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=983500
- orblivion 5y ago
  I think it would go against Debian's philosophy to move fast enough to keep up with Syncthing.

45ure 5y ago

>2. You can set conditions, such that delets can be ignored. Eg. I have a WhatsApp message+media backup going back almost 7 years now, ~65GiB. But I don't need all that on my phone. So I just disable syncing delets on my storage. Now my phone can get away with ~3 GiB of whatsapp data (only because I don't delete very often) and I still have complete backup in case I need it. I would like to know, how are you using Syncthing to create local backups for WhatsApp database/msgstore (crypt14) and media files. You also mention 65GiB as your complete backup ─ have you tested it via a restore? If so, how?

swiley 5y ago
Android used to let you administrate your devices and it was easy to do stuff like this.
- seized 5y ago
  Is still is easy to do that. I have Syncthing backing up data from my phone easily and reliably.
cesarb 5y ago
> I would like to know, how are you using Syncthing to create local backups for WhatsApp database/msgstore (crypt14) and media files. Not the parent poster, but I also do this; I just configured Syncthing to share the /storage/emulated/0/WhatsApp (aka /sdcard/WhatsApp) directory. WhatsApp stores its daily local backup on the Backups subdirectory of that directory, and the media files are all in the Media subdirectory of that directory. > have you tested it via a restore? If so, how? I have actually used it to move all the WhatsApp and Signal data from an old phone to a new phone. Just have Syncthing synchronize these directories (with the same path) on the new phone before installing WhatsApp and Signal, then install and launch WhatsApp and Signal. When first launched, if that directory already contains a backup, both WhatsApp and Signal ask if you want to restore from that backup. Signal then asks you to type a long backup encryption key you should have written down somewhere, while WhatsApp asks its servers for the backup encryption key.
e3bc54b2 5y ago

Royi 5y ago

Regarding WhatsApp, How do you merge different database files into one coherent big database?

e3bc54b2 5y ago
I.. don't. My days of tinkering directly with WA db are over. These days I just want an easy and reliable backup. After restoring, WhatsApp recognizes the files and takes care of the rest.

divbzero 5y ago

What are the “3 instances where we had something that needed attention, out of which 2 were not syncthing's fault”? I’m curious what rare exceptions there are to watch for.

e3bc54b2 5y ago
Unfortunately I don't clearly remember because last one was a while ago, but it was Sync conflicts due to external factors. The one where it was Syncthing's fault was years ago and is very likely fixed now.