12 ms·

Syncthing: A continuous file synchronization program

313 points by hello_computer 3y ago

jbverschoor 3y ago
While SyncThjng might be encrypted, it’s yet another port or server that’s open. We need ubiquitous private networks, such as wire guard, zerotier, or the buggy back-to-my-mac.
- bowel 3y ago
  You can always turn off NAT, LAN announcement, Relays and STUN -- and run SyncThing behind WireGuard. I use Tailscale and it works like a charm.
  - thefz 3y ago
    You can also run your own relay and only allow that if you want!
- mholt 3y ago
  I keep wanting Tailscale to offer a "tailnet API" whereby you can bundle a tailnet into your app, and users can connect to other instances of the app (whether their same account or other people sharing parts of their app/data) seamlessly: E2E encrypted, NAT busting, it just works. Think about it: mobile and desktop apps no longer have to upload things to some third-party service to share with yourself or a group of people. You could set up your own server of course but that requires technical expertise. The mass market could share data, sync, and communicate, straight from their devices, without any service hosting their data. It's just all on the private network. There's probably a billion-dollar market there (or more) if someone can figure that out for a lay audience: virtually unlimited, encrypted private networks on-demand that require no setup.
  - gnur 3y ago
    Pretty sure they offer that already, https://tailscale.com/blog/golink/ https://tailscale.com/blog/golink/ has more information on embedding the tailnet within a program.
    - mholt 3y ago
      It can be embedded in a program, but it doesn't automatically manage devices on a tailnet; i.e. there's no way to add all users of an app to the same tailnet (they have to explicitly create a Tailscale account and sign in and share and join) - AFAIK.
  - dmytrish 3y ago
    Isn't this litailscale?
  - barbs 3y ago
    Not sure if this is exactly what you're describing but I noticed the open-source Diablo engine DevilitionX has Zerotier built in to allow online multiplayer sessions. https://github.com/diasurgical/devilutionX/wiki/Multiplayer#zerotier https://github.com/diasurgical/devilutionX/wiki/Multiplayer#... I thought it was interesting way for an open-source game to provide a way to find other players without relying on paid dedicated servers.
    - mholt 3y ago
      Ah yeah, that's pretty cool. Something like that!!
- rakoo 3y ago
  Private networks are one part of the solution. Syncthing synchronizes, and for anyone who has ever tried it, it's not just two rsync's back and forth, whether ports are open or not
- qbasic_forever 3y ago
  If you use wireguard or something similar that gives every client their own private and encrypted IP then you don't really need syncthing. You can just use rclone or even rsync commands that point directly at the appropriate client IP in the mesh network. Syncthing is still useful though for its continuous syncing support and overall UI.
- 0xbadcafebee 3y ago
  Private networks are old and busted; zero-trust & E2EE is the new hotness. (well, if 14 years is considered new) The idea of a network as a delineation of security or trust is inherently flawed and should be considered an anti-pattern. Reliance on "private" networks merely moves attacks to the network level, making a very large target and exposing more at once. Zero trust is where the whole world should be moving: explicit authorization added at a fine-grained level to each thing each user wants to access. But people haven't been exposed to this idea yet in their day to day lives, so they can't let go of the old paradigms of firewalls and networks.
- reaperducer 3y ago
  or the buggy back-to-my-mac. Back to My Mac was awesome. When it worked. I had about an 80% success rate with it, dragging my MacBook Air to about 30 countries around the world. It was tremendously useful at a time when storage was far more expensive and internal drives much smaller than they are today. Often when it didn't work, I could just leave the computer running in the hotel while I went out and did touristy stuff, and by the time I came back things had sorted themselves out.
- Avamander 3y ago
  You can disable discovery and hardcode clients to connect to each other using hostnames that resolve to whatever private IPs you have.
- paradox460 3y ago
  You can route SyncThing over a tailnet
Joeri 3y ago
Syncthing is excellent. I’ve used it for years to keep my development folders synced between mac, linux and windows, and except for some initial figuring out of ignore patterns and the occasional hiccup with filenames that windows can’t handle it has been great at doing what it needs to and getting out of my way. (I of course also use git, but syncthing lets me choose when I push while still having all my work on all my machines.)
- thefz 3y ago
  > occasional hiccup with filenames that windows can’t handle My only complaint so far and I've been using it for years. At first it started as a way to have my Keepass file in sync between computers, and evolved from that.
- greenie_beans 3y ago
  i always heard it fucks up with git repos?
  - generalizations 3y ago
    I have an entire folder dedicated to various git repos...only thing it doesn't like (so far...) is some compilation artifacts. IDK why.
    - prophesi 3y ago
      I would make sure that there's a `.stignore` file that's identical to your `.gitignore` (except for secret keys/configs you'd like to sync on your devices) in each of those repositories.
      - Joeri 3y ago
        You can also make a global one for an entire shared folder. For example, I wildcard all node_modules folders as excluded.
  - sveinugu 3y ago
tech234a 3y ago
I use it and it works well, unfortunately there isn’t a free client for iOS though.
alexbezhan 3y ago
Can I use it to backup SQLite? Like an alternative to Litestream? Are there any negative implications for doing this?
- alexbezhan 3y ago
  Found relevant discussion here https://forum.syncthing.net/t/bahaviour-of-st-on-sqlite-files-in-various-scenarios/5921 https://forum.syncthing.net/t/bahaviour-of-st-on-sqlite-file...
- system33- 3y ago
  Syncthing "just" syncs files. Near instantly, which could mean a delay of a few seconds to maybe even a minute. And it won't de-conflict changes beyond saving the "bad" version in a file like `sqlite.db-syncthing-conflict-20230101T043044`. (Not an issue if you only have one host changing the database file) I like it and IMO it does what it does well. But litestream can "Continuously stream SQLite changes" (pulled from https://litestream.io/ https://litestream.io/, I've never heard of it before). It seems like that's litestream's killer feature, and I think syncthing would leave you wanting.
- qbasic_forever 3y ago
  Not really, it can't resolve a conflict inside the file and can't enforce a distributed lock on changing the file. Ultimately there will be ways for the file to corrupt or get out of sync, especially if multiple writers are present. There might be a way to make things work better with SQLite's more recent WAL mode but I'm not familiar with it.
- benbjohnson 3y ago
  Litestream author here. I haven't personally tried Syncthing on SQLite databases but I suspect it wouldn't work well. Syncthing doesn't have any concept of SQLite's locks so even in the best case scenario where you only have one writer and the replica is read-only, the reader on the replica would have pages changed out from under it by Syncthing and that would appear corrupt to the reader. One case where you could make it work is if you sync'd across multiple computers but only used one at a time. I mean, it's still dangerous but it seems possible.
dariosalvi78 3y ago
I use it for uploading pictures from my phone to my "private cloud" (an old RaspberryPi) and it works like a charm. I then move the uploaded pictures into organized folders, which also makes it delete the pictures from my phone, thus saving space.
aborsy 3y ago
Best sync software in my view. The only downside is that there is no good iOS app. I think the developers made the right decision focusing the software on peer to peer sync (i.e., not expanding to nearby “cloud features”: backup, photos, videos, office, etc).
- ajd1988 3y ago
  I couldn't agree more! Syncthing is like the reliable, quiet roommate who always does their dishes, whereas dealing with different file systems feels like trying to figure out who left their half-eaten pizza on the couch. Just needs the iOS app!
  - celsoazevedo 3y ago
    There's Möbius Sync, but iOS restrictions means that it can't work in the background or access folders like it does on Android/Windows/macOS/Linux.
    - est31 3y ago
      iOS literally doesn't have "just there for storage" directories that are not icloud. Either you use the photos pool, or you use icloud, or you use per-app directories that are local to each app. But then those directories cannot be accessed by other apps. Even if access to individual files is implemented, it's often very buggy. Say you have a pictures directory shared via Möbius Sync. Then you want to edit one of the pictures with a pictures editing app that is not the builtin photos app. The app crashes, or cannot save an edited copy of the file into the same directory. I encountered this in real life.
      - btobolaski 3y ago
        Working copy has some of doing this. I had it syncing my logseq notes and it worked fine (other than merge conflict type things). The setup was fairly convoluted though. This is how it is done https://github.com/CharlesChiuGit/Logseq-Git-Sync-101/wiki/For-iOS-iPadOS-users-(Working-Copy) https://github.com/CharlesChiuGit/Logseq-Git-Sync-101/wiki/F...
mteam88 3y ago
Been using for a while with my keepass database. Almost never noticed it.
- barbs 3y ago
  I remember reading somewhere that the best software does what it needs to do and then gets out of your way...and it was referring specifically to Syncthing!
ceeam 3y ago
Using resilio (which is NOT open source, of course and is actually kinda abandoned) until they implement selective sync (also "cloud" files browser). This is THE feature that is the deal breaker for me to switch. No, I don't sync hundreds of gigabytes from my personal cloud to my fokken phone, or even laptop, or even work PC. Still want access to all of them on demand.
- ceeam 3y ago
  And - yes - a program like Resilio/Syncthing is the paradigm shift to how you use your computers, if you're a serious user and have more than one machine (incl. mobile devices).
- vorpalhex 3y ago
  My workaround with syncthing is to have several smaller folders that I only add to devices as I want them, and a common folder across all devices. It works well enough.
- tener 3y ago
  You do have ignore lists, but the UX is extra bad. OTOH resilio ate my files in certain situations, so I had to ditch it.
- xcdzvyn 3y ago
  You can create multiple folders (~/Sync/everywhere, ~/Sync/phone) and share them to different devices?
- deluxeroyale 3y ago
  Same here, being hold hostage by resilio just waiting for it to either die or kill my files. I really need that selective sync.
- npteljes 3y ago
  Nextcloud also works like this on the phone. Easy to access everything, but the files only get downloaded on-demand. I love it because I also auto-upload my phone pictures, downloads and video, and sync my contacts and todo with it too. And it can share any of the files via url too. Very useful. On PC, you also selectively sync folders. I dunno about access to all the files though. Definitely works in the browser, but on PC, you either sync the folder or don't.
zapdrive 3y ago
While syncthing is excellent and I use it backup photos from my phone to my private cloud, it can sometimes glitch and delete files. I haven't got to the root cause yet, so I can't tell you what to do, but there's a way to recover your data in case of accidental deletion by syncthing. Make sure you enable "file versioning" on both your clients and server. I use the "trashcan" option. So in case a file is deleted, syncthing saves it in a folder for a pre-selected number of days instead of deleting it.
- kyrofa 3y ago
  Yikes. Nextcloud used to do that to me. I'd love to find an alternative, but I don't want to go back to those dark days.
- karlicoss 3y ago
  FWIW, never happened to me with Synching, but either way it's best to separate sync software and backup software. Sync might work flawlessly, but the user might make a mistake which would quickly wipe the files across all devices. I use borg for backups, highly recommend
zqna 3y ago
Used syncthing to: - transfer executables from my home to a ship (over sattelite link) in Indian Ocean. Much better alternative than sftp. - sync filesystems while doing on-going (several months of testing) migration from on-prem to mixed cloud (google/azure). Requires a bit of understanding why conflicts can happen (when using more than 2 hosts), once understood works like a charm. Was never disappointed, solid work.
thewataccount 3y ago
If you had 10's thousands or millions of files, and several terabytes, is syncthing a good option for this? I haven't seen a ton of great options unfortunately :/
- em-bee 3y ago
  that is a hard problem. it depends on how frequently the data changes, because syncthing or any other backup solution may be to slow to keep up. one of my clients eventually switched from rsync to gluster. instead of making backups, gluster writes multiple storage servers in parallel when files are changed, thus ensuring that there are multiple copies right away. it's not the perfect solution though because when one server is offline for a while and needs to catch up that too is very slow. the only solution there i find is to have enough redundant servers that you can afford to have a server go offline and not worry about getting it back online quickly.
- vorpalhex 3y ago
  I've been successful with syncthing up to 2tb (never tried higher) with the caveat that I only have it function between two hosts and only one host is the writer.
  - nine_k 3y ago
    I have a 2 TB setup with multiple writers; works fine.
- karlicoss 3y ago
  I sync about two hundred thousands of files without any problems, especially if your aren't changing them all the time. The only issue I can imagine is initial sync with so many files might take a while, even if total size isn't huge. As for several terabytes, for me it's a bit less than a terabyte in total for all synced folders, but also can't see why it wouldn't scale up. I guess similar, initial hashing might take some time, but otherwise it should handle it well.
- icelancer 3y ago
  I actually have this exact use case. About 25MM files, ~20 TB total. Lot of directories. Many servers (~10), only a few that have the exact same synching file/folder setup (in other words, some servers have only video, some have only code, some have it all, etc, you get it). It works... pretty well. I won't lie and say it works perfectly with all of our other file utilities and scripts we run, but 95-97% of the time I never think about it. I use it with SyncTrazor on Windows Server deployments as well as Debian servers and yeah, I could be happier, but I highly doubt there's anything out there that can beat SyncThing overall. Just know that with that many files the initial scans and DB building will be brutal, and the periodic syncs and rescans will be taxing. Highly recommend getting cheap 1-2U servers with dual sockets and lots of cores/threads - multiprocessing goes a long way in doing the file hashing. EDIT: I will add that I only use it on servers and not on any client/desktop machines, so I can't vouch for its use there.
toastercat 3y ago
Been using Syncthing for a year or so to sync plaintext notes across my phone and PCs. Worked great, so then I began using it to sync Retroarch save files between a spare Pixel 2 I use for gaming and my PC. Worked even better! I could play games on my PC, and then continue on the go. Recently, I switched completely off cloud storage to just using Syncthing and I love it. Anyway, this is great software.
charleshan 3y ago
I've been using it for a couple of years. One problem I have is that the idle cpu usage goes up to 10-20% after a while on my Mac. Drops back to ~1% after restarting the app
gsich 3y ago
Have they fixed the SD card issue?
paradox460 3y ago
I've used Syncthing in the past to keep different volumes in "sync" in a distributed system. The API was simple enough to plug into an orchestration framework, and for quick/dirty projects it's phenomenal. Right now I'm actually using it for a very small minecraft server on Fly.io. The game server runs on a "bigger" instance, and there's a web server running on a much smaller (and therefore "free") instance, for displaying the game map and similar things. Syncthing copies the generated squaremap tiles to the webserver, so when the game server is down, the map is still available
bityard 3y ago
I use syncthing to get photos from a motion sensitive security camera on a mobile data connection to my vps. Beats writing a shell script all to hell.
muppetman 3y ago
Syncthing is amazing. Dropbox was the trail blazer, but then of course became enshittified. Syncthing is everything I could wish for, it's nerdy enough there's knobs to tweak, but it "just works" out of the box and I can install it on my Dad's laptop to back his stuff up etc. Love, Love, Love Syncthing.
darkteflon 3y ago
Despite some early mis-steps with Syncthing - mostly me just failing to understand how best to use it - I now count it as one of my favourite tools. Rock-solid, gracefully handles long periods of one or more machines going offline (eg personal use computers in a home network), partly configurable with text files. The way it can be set for one-way syncs also opens up new use cases. I use it for syncing documents, video capture, certain config files - even my ROM collection to my Steam Deck. The only place you really shouldn’t use it is on Git repos - something that is probably obvious to anyone who has used a git branch, but worth re-stating. For a simple repo with only a main branch you want on just two machines - eg dev desktop and laptop - you might get away with it for a little while but it’s extremely brittle and will eventually trip you up - through no fault of Syncthing or Git, I should add. Just fundamentally incompatible paradigms.
Arn_Thor 3y ago
Does anyone have experience with Syncthing in high-latency connection scenarios, for example across continents? I need to bring and keep up to date tens of TBs between a couple of servers very far apart (~200ms ping). Almost everything I've tried transfers at a fraction of the speeds possible. The exception is Rclone where I'm able to properly saturate the connection.
- aesh2Xa1 3y ago
  https://garagehq.deuxfleurs.fr https://garagehq.deuxfleurs.fr Garage claims performance across long distances (up to 200ms per their front page). It's not a POSIX file system. Instead it's more like S3. You could put something in front of it, perhaps, to make it POSIX.
  - Arn_Thor 3y ago
    Thanks for the suggestion! I'll look into it. I just double-checked and it seems the ping I'm dealing with is just a little bit above their recommended limit
- 5e92cb50239222b 3y ago
  200ms is not "very far apart". I routinely use it on 350-450ms connections with no issues — it easily saturates my 500 Mbps link. Although in your case I would probably use ZFS snapshots through syncoid, they'd be much more efficient than anything else.
- otbutz 3y ago
  You could try to force QUIC which should handle latency a bit better. The priority of TCP/QUIC will be configurable with the upcoming v1.23.5 release: https://github.com/syncthing/syncthing/pull/8868 https://github.com/syncthing/syncthing/pull/8868
n8henrie 3y ago
For iOS, I placed a syncthing folder inside my iCloud directory and it works fairly well. My MacBook, an always-on Pi, and a few other boxes run syncthing for a directory full of Markdown files that I use with Notational Velocity[0] on Mac and 1Writer[1] (highly recommended!) on iOS. Using it this way for a couple years and it works well, occasionally go through and diff the sync conflict files that slowly accumulate, small hassle that is worth the squeeze IMO. [0]: https://notational.net/ https://notational.net/ [1]: https://apps.apple.com/app/id680469088 https://apps.apple.com/app/id680469088
npteljes 3y ago
I use this to exchange files with my family. We have a shared folder, like "d:\family share", and everyone can drop there whatever they want to share. Works really well with large files, I love it how gracefully it handles network events, like when the laptop drops wifi, or when you make the computer sleep during transfer.

While SyncThjng might be encrypted, it’s yet another port or server that’s open. We need ubiquitous private networks, such as wire guard, zerotier, or the buggy back-to-my-mac.

bowel 3y ago
You can always turn off NAT, LAN announcement, Relays and STUN -- and run SyncThing behind WireGuard. I use Tailscale and it works like a charm.
- thefz 3y ago
  You can also run your own relay and only allow that if you want!
mholt 3y ago
I keep wanting Tailscale to offer a "tailnet API" whereby you can bundle a tailnet into your app, and users can connect to other instances of the app (whether their same account or other people sharing parts of their app/data) seamlessly: E2E encrypted, NAT busting, it just works. Think about it: mobile and desktop apps no longer have to upload things to some third-party service to share with yourself or a group of people. You could set up your own server of course but that requires technical expertise. The mass market could share data, sync, and communicate, straight from their devices, without any service hosting their data. It's just all on the private network. There's probably a billion-dollar market there (or more) if someone can figure that out for a lay audience: virtually unlimited, encrypted private networks on-demand that require no setup.
- gnur 3y ago
  Pretty sure they offer that already, https://tailscale.com/blog/golink/ https://tailscale.com/blog/golink/ has more information on embedding the tailnet within a program.
  - mholt 3y ago
    It can be embedded in a program, but it doesn't automatically manage devices on a tailnet; i.e. there's no way to add all users of an app to the same tailnet (they have to explicitly create a Tailscale account and sign in and share and join) - AFAIK.
- dmytrish 3y ago
  Isn't this litailscale?
- barbs 3y ago
  Not sure if this is exactly what you're describing but I noticed the open-source Diablo engine DevilitionX has Zerotier built in to allow online multiplayer sessions. https://github.com/diasurgical/devilutionX/wiki/Multiplayer#zerotier https://github.com/diasurgical/devilutionX/wiki/Multiplayer#... I thought it was interesting way for an open-source game to provide a way to find other players without relying on paid dedicated servers.
  - mholt 3y ago
    Ah yeah, that's pretty cool. Something like that!!
rakoo 3y ago
Private networks are one part of the solution. Syncthing synchronizes, and for anyone who has ever tried it, it's not just two rsync's back and forth, whether ports are open or not
qbasic_forever 3y ago
If you use wireguard or something similar that gives every client their own private and encrypted IP then you don't really need syncthing. You can just use rclone or even rsync commands that point directly at the appropriate client IP in the mesh network. Syncthing is still useful though for its continuous syncing support and overall UI.
0xbadcafebee 3y ago
Private networks are old and busted; zero-trust & E2EE is the new hotness. (well, if 14 years is considered new) The idea of a network as a delineation of security or trust is inherently flawed and should be considered an anti-pattern. Reliance on "private" networks merely moves attacks to the network level, making a very large target and exposing more at once. Zero trust is where the whole world should be moving: explicit authorization added at a fine-grained level to each thing each user wants to access. But people haven't been exposed to this idea yet in their day to day lives, so they can't let go of the old paradigms of firewalls and networks.
reaperducer 3y ago
or the buggy back-to-my-mac. Back to My Mac was awesome. When it worked. I had about an 80% success rate with it, dragging my MacBook Air to about 30 countries around the world. It was tremendously useful at a time when storage was far more expensive and internal drives much smaller than they are today. Often when it didn't work, I could just leave the computer running in the hotel while I went out and did touristy stuff, and by the time I came back things had sorted themselves out.
Avamander 3y ago
You can disable discovery and hardcode clients to connect to each other using hostnames that resolve to whatever private IPs you have.
paradox460 3y ago
You can route SyncThing over a tailnet

Joeri 3y ago

Syncthing is excellent. I’ve used it for years to keep my development folders synced between mac, linux and windows, and except for some initial figuring out of ignore patterns and the occasional hiccup with filenames that windows can’t handle it has been great at doing what it needs to and getting out of my way. (I of course also use git, but syncthing lets me choose when I push while still having all my work on all my machines.)

thefz 3y ago
> occasional hiccup with filenames that windows can’t handle My only complaint so far and I've been using it for years. At first it started as a way to have my Keepass file in sync between computers, and evolved from that.
greenie_beans 3y ago
i always heard it fucks up with git repos?
- generalizations 3y ago
  I have an entire folder dedicated to various git repos...only thing it doesn't like (so far...) is some compilation artifacts. IDK why.
  - prophesi 3y ago
    I would make sure that there's a `.stignore` file that's identical to your `.gitignore` (except for secret keys/configs you'd like to sync on your devices) in each of those repositories.
    - Joeri 3y ago
      You can also make a global one for an entire shared folder. For example, I wildcard all node_modules folders as excluded.
- sveinugu 3y ago

tech234a 3y ago

I use it and it works well, unfortunately there isn’t a free client for iOS though.

alexbezhan 3y ago

Can I use it to backup SQLite? Like an alternative to Litestream? Are there any negative implications for doing this?

alexbezhan 3y ago
Found relevant discussion here https://forum.syncthing.net/t/bahaviour-of-st-on-sqlite-files-in-various-scenarios/5921 https://forum.syncthing.net/t/bahaviour-of-st-on-sqlite-file...
system33- 3y ago
Syncthing "just" syncs files. Near instantly, which could mean a delay of a few seconds to maybe even a minute. And it won't de-conflict changes beyond saving the "bad" version in a file like `sqlite.db-syncthing-conflict-20230101T043044`. (Not an issue if you only have one host changing the database file) I like it and IMO it does what it does well. But litestream can "Continuously stream SQLite changes" (pulled from https://litestream.io/ https://litestream.io/, I've never heard of it before). It seems like that's litestream's killer feature, and I think syncthing would leave you wanting.
qbasic_forever 3y ago
Not really, it can't resolve a conflict inside the file and can't enforce a distributed lock on changing the file. Ultimately there will be ways for the file to corrupt or get out of sync, especially if multiple writers are present. There might be a way to make things work better with SQLite's more recent WAL mode but I'm not familiar with it.
benbjohnson 3y ago
Litestream author here. I haven't personally tried Syncthing on SQLite databases but I suspect it wouldn't work well. Syncthing doesn't have any concept of SQLite's locks so even in the best case scenario where you only have one writer and the replica is read-only, the reader on the replica would have pages changed out from under it by Syncthing and that would appear corrupt to the reader. One case where you could make it work is if you sync'd across multiple computers but only used one at a time. I mean, it's still dangerous but it seems possible.

dariosalvi78 3y ago

I use it for uploading pictures from my phone to my "private cloud" (an old RaspberryPi) and it works like a charm. I then move the uploaded pictures into organized folders, which also makes it delete the pictures from my phone, thus saving space.

aborsy 3y ago

Best sync software in my view. The only downside is that there is no good iOS app. I think the developers made the right decision focusing the software on peer to peer sync (i.e., not expanding to nearby “cloud features”: backup, photos, videos, office, etc).

ajd1988 3y ago
I couldn't agree more! Syncthing is like the reliable, quiet roommate who always does their dishes, whereas dealing with different file systems feels like trying to figure out who left their half-eaten pizza on the couch. Just needs the iOS app!
- celsoazevedo 3y ago
  There's Möbius Sync, but iOS restrictions means that it can't work in the background or access folders like it does on Android/Windows/macOS/Linux.
  - est31 3y ago
    iOS literally doesn't have "just there for storage" directories that are not icloud. Either you use the photos pool, or you use icloud, or you use per-app directories that are local to each app. But then those directories cannot be accessed by other apps. Even if access to individual files is implemented, it's often very buggy. Say you have a pictures directory shared via Möbius Sync. Then you want to edit one of the pictures with a pictures editing app that is not the builtin photos app. The app crashes, or cannot save an edited copy of the file into the same directory. I encountered this in real life.
    - btobolaski 3y ago
      Working copy has some of doing this. I had it syncing my logseq notes and it worked fine (other than merge conflict type things). The setup was fairly convoluted though. This is how it is done https://github.com/CharlesChiuGit/Logseq-Git-Sync-101/wiki/For-iOS-iPadOS-users-(Working-Copy) https://github.com/CharlesChiuGit/Logseq-Git-Sync-101/wiki/F...

mteam88 3y ago

Been using for a while with my keepass database. Almost never noticed it.

barbs 3y ago
I remember reading somewhere that the best software does what it needs to do and then gets out of your way...and it was referring specifically to Syncthing!

ceeam 3y ago

Using resilio (which is NOT open source, of course and is actually kinda abandoned) until they implement selective sync (also "cloud" files browser). This is THE feature that is the deal breaker for me to switch. No, I don't sync hundreds of gigabytes from my personal cloud to my fokken phone, or even laptop, or even work PC. Still want access to all of them on demand.

ceeam 3y ago
And - yes - a program like Resilio/Syncthing is the paradigm shift to how you use your computers, if you're a serious user and have more than one machine (incl. mobile devices).
vorpalhex 3y ago
My workaround with syncthing is to have several smaller folders that I only add to devices as I want them, and a common folder across all devices. It works well enough.
tener 3y ago
You do have ignore lists, but the UX is extra bad. OTOH resilio ate my files in certain situations, so I had to ditch it.
xcdzvyn 3y ago
You can create multiple folders (~/Sync/everywhere, ~/Sync/phone) and share them to different devices?
deluxeroyale 3y ago
Same here, being hold hostage by resilio just waiting for it to either die or kill my files. I really need that selective sync.
npteljes 3y ago
Nextcloud also works like this on the phone. Easy to access everything, but the files only get downloaded on-demand. I love it because I also auto-upload my phone pictures, downloads and video, and sync my contacts and todo with it too. And it can share any of the files via url too. Very useful. On PC, you also selectively sync folders. I dunno about access to all the files though. Definitely works in the browser, but on PC, you either sync the folder or don't.

zapdrive 3y ago

While syncthing is excellent and I use it backup photos from my phone to my private cloud, it can sometimes glitch and delete files. I haven't got to the root cause yet, so I can't tell you what to do, but there's a way to recover your data in case of accidental deletion by syncthing. Make sure you enable "file versioning" on both your clients and server. I use the "trashcan" option. So in case a file is deleted, syncthing saves it in a folder for a pre-selected number of days instead of deleting it.

kyrofa 3y ago
Yikes. Nextcloud used to do that to me. I'd love to find an alternative, but I don't want to go back to those dark days.
karlicoss 3y ago
FWIW, never happened to me with Synching, but either way it's best to separate sync software and backup software. Sync might work flawlessly, but the user might make a mistake which would quickly wipe the files across all devices. I use borg for backups, highly recommend

zqna 3y ago

Used syncthing to: - transfer executables from my home to a ship (over sattelite link) in Indian Ocean. Much better alternative than sftp. - sync filesystems while doing on-going (several months of testing) migration from on-prem to mixed cloud (google/azure). Requires a bit of understanding why conflicts can happen (when using more than 2 hosts), once understood works like a charm. Was never disappointed, solid work.

thewataccount 3y ago

If you had 10's thousands or millions of files, and several terabytes, is syncthing a good option for this? I haven't seen a ton of great options unfortunately :/

em-bee 3y ago
that is a hard problem. it depends on how frequently the data changes, because syncthing or any other backup solution may be to slow to keep up. one of my clients eventually switched from rsync to gluster. instead of making backups, gluster writes multiple storage servers in parallel when files are changed, thus ensuring that there are multiple copies right away. it's not the perfect solution though because when one server is offline for a while and needs to catch up that too is very slow. the only solution there i find is to have enough redundant servers that you can afford to have a server go offline and not worry about getting it back online quickly.
vorpalhex 3y ago
I've been successful with syncthing up to 2tb (never tried higher) with the caveat that I only have it function between two hosts and only one host is the writer.
- nine_k 3y ago
  I have a 2 TB setup with multiple writers; works fine.
karlicoss 3y ago
I sync about two hundred thousands of files without any problems, especially if your aren't changing them all the time. The only issue I can imagine is initial sync with so many files might take a while, even if total size isn't huge. As for several terabytes, for me it's a bit less than a terabyte in total for all synced folders, but also can't see why it wouldn't scale up. I guess similar, initial hashing might take some time, but otherwise it should handle it well.
icelancer 3y ago
I actually have this exact use case. About 25MM files, ~20 TB total. Lot of directories. Many servers (~10), only a few that have the exact same synching file/folder setup (in other words, some servers have only video, some have only code, some have it all, etc, you get it). It works... pretty well. I won't lie and say it works perfectly with all of our other file utilities and scripts we run, but 95-97% of the time I never think about it. I use it with SyncTrazor on Windows Server deployments as well as Debian servers and yeah, I could be happier, but I highly doubt there's anything out there that can beat SyncThing overall. Just know that with that many files the initial scans and DB building will be brutal, and the periodic syncs and rescans will be taxing. Highly recommend getting cheap 1-2U servers with dual sockets and lots of cores/threads - multiprocessing goes a long way in doing the file hashing. EDIT: I will add that I only use it on servers and not on any client/desktop machines, so I can't vouch for its use there.

toastercat 3y ago

Been using Syncthing for a year or so to sync plaintext notes across my phone and PCs. Worked great, so then I began using it to sync Retroarch save files between a spare Pixel 2 I use for gaming and my PC. Worked even better! I could play games on my PC, and then continue on the go. Recently, I switched completely off cloud storage to just using Syncthing and I love it. Anyway, this is great software.

charleshan 3y ago

I've been using it for a couple of years. One problem I have is that the idle cpu usage goes up to 10-20% after a while on my Mac. Drops back to ~1% after restarting the app

gsich 3y ago

Have they fixed the SD card issue?

paradox460 3y ago

I've used Syncthing in the past to keep different volumes in "sync" in a distributed system. The API was simple enough to plug into an orchestration framework, and for quick/dirty projects it's phenomenal. Right now I'm actually using it for a very small minecraft server on Fly.io. The game server runs on a "bigger" instance, and there's a web server running on a much smaller (and therefore "free") instance, for displaying the game map and similar things. Syncthing copies the generated squaremap tiles to the webserver, so when the game server is down, the map is still available

bityard 3y ago

I use syncthing to get photos from a motion sensitive security camera on a mobile data connection to my vps. Beats writing a shell script all to hell.

muppetman 3y ago

Syncthing is amazing. Dropbox was the trail blazer, but then of course became enshittified. Syncthing is everything I could wish for, it's nerdy enough there's knobs to tweak, but it "just works" out of the box and I can install it on my Dad's laptop to back his stuff up etc. Love, Love, Love Syncthing.

darkteflon 3y ago

Despite some early mis-steps with Syncthing - mostly me just failing to understand how best to use it - I now count it as one of my favourite tools. Rock-solid, gracefully handles long periods of one or more machines going offline (eg personal use computers in a home network), partly configurable with text files. The way it can be set for one-way syncs also opens up new use cases. I use it for syncing documents, video capture, certain config files - even my ROM collection to my Steam Deck. The only place you really shouldn’t use it is on Git repos - something that is probably obvious to anyone who has used a git branch, but worth re-stating. For a simple repo with only a main branch you want on just two machines - eg dev desktop and laptop - you might get away with it for a little while but it’s extremely brittle and will eventually trip you up - through no fault of Syncthing or Git, I should add. Just fundamentally incompatible paradigms.

Arn_Thor 3y ago

Does anyone have experience with Syncthing in high-latency connection scenarios, for example across continents? I need to bring and keep up to date tens of TBs between a couple of servers very far apart (~200ms ping). Almost everything I've tried transfers at a fraction of the speeds possible. The exception is Rclone where I'm able to properly saturate the connection.

aesh2Xa1 3y ago
https://garagehq.deuxfleurs.fr https://garagehq.deuxfleurs.fr Garage claims performance across long distances (up to 200ms per their front page). It's not a POSIX file system. Instead it's more like S3. You could put something in front of it, perhaps, to make it POSIX.
- Arn_Thor 3y ago
  Thanks for the suggestion! I'll look into it. I just double-checked and it seems the ping I'm dealing with is just a little bit above their recommended limit
5e92cb50239222b 3y ago
200ms is not "very far apart". I routinely use it on 350-450ms connections with no issues — it easily saturates my 500 Mbps link. Although in your case I would probably use ZFS snapshots through syncoid, they'd be much more efficient than anything else.
otbutz 3y ago
You could try to force QUIC which should handle latency a bit better. The priority of TCP/QUIC will be configurable with the upcoming v1.23.5 release: https://github.com/syncthing/syncthing/pull/8868 https://github.com/syncthing/syncthing/pull/8868

n8henrie 3y ago

For iOS, I placed a syncthing folder inside my iCloud directory and it works fairly well. My MacBook, an always-on Pi, and a few other boxes run syncthing for a directory full of Markdown files that I use with Notational Velocity[0] on Mac and 1Writer[1] (highly recommended!) on iOS. Using it this way for a couple years and it works well, occasionally go through and diff the sync conflict files that slowly accumulate, small hassle that is worth the squeeze IMO. [0]: https://notational.net/ https://notational.net/ [1]: https://apps.apple.com/app/id680469088 https://apps.apple.com/app/id680469088

npteljes 3y ago

I use this to exchange files with my family. We have a shared folder, like "d:\family share", and everyone can drop there whatever they want to share. Works really well with large files, I love it how gracefully it handles network events, like when the laptop drops wifi, or when you make the computer sleep during transfer.