17 ms·

Syncthing – a continuous file synchronization program

545 points by tambourine_man 5y ago

ianbtaylor 5y ago
I recently started using this to sync a few folders full of some code and documents between three of my computers. So far, it works very nicely, to the point I don't think about which machine I sit down at, because I know it'll already have the stuff I'm working on, all ready to go, without any fussing around or needing to running `git pull` in a bunch of different folders to get caught up to where I left off. I like that there's no cloud dependency and no single point of failure.
nett18 5y ago
Something that interests me even more is that the company behind, Kastelo Inc., possibly bears its name from the esperantoan word for 'Castle'. Can anyone confirm?
- commoner 5y ago
  You're probably right. Kastelo previously launched a service called Arigi that was later discontinued. The announcement notes that "arigi" means "to amass · to gather · to put together" in Esperanto. https://webcache.googleusercontent.com/search?q=cache:IAwDkf0wpYwJ:https://kastelo.net/blog/2017-09/introducing-arigi/ https://webcache.googleusercontent.com/search?q=cache:IAwDkf...
  - 0xADADA 5y ago
    or its a tech startup that just named it 'Castle' spelled with a K-elo to make it unique
bedros 5y ago
only if I can filter what to sync locally
- commoner 5y ago
  Here's Syncthing's documentation on specifying files to ignore or include using the .stignore file: https://docs.syncthing.net/users/ignoring.html https://docs.syncthing.net/users/ignoring.html This also works in the web and Android clients, which expose .stignore through the "Ignore Patterns" setting.
- KennyBlanken 5y ago
  https://www.google.com/search?q=syncthing+filter https://www.google.com/search?q=syncthing+filter
  - prirai 5y ago
    Google, really.
- rg111 5y ago
  Yes you can.
asicsp 5y ago
GitHub repo: https://github.com/syncthing/syncthing https://github.com/syncthing/syncthing Some previous discussions: * "Syncthing: Open Source Dropbox and BitTorrent Sync Replacement" https://news.ycombinator.com/item?id=7734114 https://news.ycombinator.com/item?id=7734114 (7 years ago|184 comments) * "Syncthing is everything I used to love about computers" https://news.ycombinator.com/item?id=23537243 https://news.ycombinator.com/item?id=23537243 (1 year ago|159 comments) * "Syncthing: Syncing All the Things" https://news.ycombinator.com/item?id=27929194 https://news.ycombinator.com/item?id=27929194 (3 months ago|172 comments)
KennyBlanken 5y ago
Syncthing works brilliantly! The web UI is excellent, warns you when you're about to do something ill-advised, and stuff like QR codes makes adding clients and folders fairly easy. The separation between folders and devices is handled well. You can easily have half a dozen shared folders between several computers with any mix-and-match combination including which one is 'authoritative' and so on. I used to use it on Android to keep my 'camera roll' synced to my desktop. I preferred setting the ignore file to exclude android's thumbnails directory, but otherwise it worked great. I'd take a photo with my phone and almost before I could open the shortcut on the desktop, boom, the photo was there, since the android client detects changes in the filesystem. I also used it to sync my password database, and a cross-platform notebook app's database. Worked flawlessly for all of them. Ditto for a folder shared between my MacOS laptop and Windows desktop. Lots of controls, especially in the forked android client. Don't want the sync client to run on any network other than your home network? Done. Don't want your syncthing clients to try and do NAT transversal / discovery outside whatever network it's on? No problem. The biggest bummer is that there's no iOS client. Had to switch to Nextcloud, which so far has been working OK, but the number of people that have problems with the iOS client is quite high (problems such as syncs being so slow it takes days to sync a camera roll), and it's generally sluggish and doesn't work in the background (yes, I know, Apple's fault.) However, Nextcloud does allow me to sync contacts and calendars (I think. I haven't tried to set it up yet.) One warning: Sycnthing really does not like it when you delete the dotfile folder inside the shared folder(s). Don't do that :)
- _emacsomancer_ 5y ago
  There's Mobius, a third party client for iOS: https://www.mobiussync.com/ https://www.mobiussync.com/ . I've never used it, so I don't know how good it is.
- Reventlov 5y ago
  I would not qualify the web UI of "excellent" : understanding the relations between paired devices, remote devices, folders you want to sync, and so on is harder than what it should be. Likewise, the android application is so bad the best way is to open the web UI on your phone to do stuff (and sometimes, it's the only way to achieve some goals). There is definitely a lot of room for improvement for easier adoption, yet, this is one of the few sync app that work for me, even behind NAT, with closed ports, and so on.
icheyne 5y ago
I could never get this to work on my home network. ¯\_(ツ)_/¯
- prirai 5y ago
  You have to force it to use wlan or wireless. It solves the problem. Also, wait for a few moments for it to be able to detect your other device.
- mkl 5y ago
  If you're using Windows, make sure Windows doesn't think your network is public. My dad's computers thought they were on a public network and so wouldn't see each other directly (they were syncing though, very slowly via an external relay).
gnulinux 5y ago
Is there an obvious way to use this with rclone?
- stef25 5y ago
  What are you trying to do?
- jorislacance 5y ago
  I guess it is more one or the other depending on the use case. I've been synchronizing local data to remote data using NFS and Syncthing for a while now instead of doing batches of rclone, if you were referring to this use case.
- ncphil 5y ago
  ... I actually use rclone mount _instead_ of sync for cloud storage. Tried syncthing a long while ago after finding nextcloud unreliable. Then stumbled acrosss a comment somewhere that talked about the mount feature in rclone (https://rclone.org/commands/rclone_mount/ https://rclone.org/commands/rclone_mount/). This is probably only practical for me now because we've got a 1G/1G pipe to the Internet. On the internal network I'm back to using nfs mounts (or samba for the Windows machines). Syncthing is good, but over time I've found all sync solutions to fall short. For backups I use a combination of rsync to the file server on both Linux and Windows, and then rclone out to the cloud.
iszomer 5y ago
It's sufficient enough for my needs. I use Joplin with it as well as no-frills file syncing across my home network. Just a note on revision history: it's better to configure this feature on the notetaking app alone while disabling it on Syncthing. My only gripe is that this utility may soon become deprecated as Android continues to update their convoluted storage schemes.
- deaddabe 5y ago
  Can you share a reference wrt evolving Android storage schemes?
  - commoner 5y ago
    Android 11+ enforces scoped storage, which restricts access to the filesystem in many cases.[1] Syncthing requests the "All files access" permission[2] to get around this, but the /sdcard/Android/data directory (private app data) is still off limits.[3] [1] https://source.android.com/devices/storage/scoped https://source.android.com/devices/storage/scoped [2] https://developer.android.com/training/data-storage/manage-all-files https://developer.android.com/training/data-storage/manage-a... [3] https://github.com/syncthing/syncthing-android/issues/1638 https://github.com/syncthing/syncthing-android/issues/1638
jwr 5y ago
Syncthing has been working for me flawlessly over the last year. Very impressed by it, especially compared to Dropbox, which is a CPU hog and has become a nagging salesperson on my computer. Highly recommended.
amanzi 5y ago
Nothing but good things to say about SyncThings. I'm only using it on a couple of computers to keep a small about of things in sync, but it always works flawlessly.
deaddabe 5y ago
I have switched from Nextcloud to Syncthing. The former was a resource hog that was eating up too much everything on my modest VPS. I share the files on my local network at home between phone, laptop and desktop. My only issues so far: - I need to manually start Syncthing on the devices, whereas Nextcloud was always running on boot. Not sure if I should start it every time, especially on the Android phone which cannot sync while away from home. I do not want to empty the battery while outside of my LAN, + the persistent notification "Syncthing is running" is annoying. - I need to power up at least the laptop or desktop to backup my phone data. One solution that I will try is to install a headless Syncthing instance on my NAS. This way the phone can always sync to the NAS every time, and then laptop/desktop can sync from NAS when needed. The joys of peer2peer! At least, I am sure that my moderately sensitive files are not leaving my local network to be stored on my VPS, which can be hacked someday. I also need to figure out how to install Wireguard on my router to allow to access my NAS from my phone while on a trip, but that's another story.
- Maakuth 5y ago
  I've dealt with the Android issue by configuring Syncthing to only run when the phone is charging. I'm in the habit of leaving my phone to charger every night anyway, so I'm regularly synced. It would be great if there would be a 'Sync now' function though, that would run the sync and then go back to ordinary mode afterwards, for the rare case that I need my folder synced in the instant.
- Daegalus 5y ago
  try out Synctrayzor for the laptop and desktop. it lets you set it to auto start on boot.
  - deaddabe 5y ago
    Thanks for the suggestion. This is an app for Windows [0] but everything I use is Linux, so the setup to start SyncThing it at boot should be straightforward. I am more worried about the Android part. The best solution would be to start SyncThing when on my LAN via matching SSID, and disable it when I am out of range. [0] https://github.com/canton7/SyncTrayzor https://github.com/canton7/SyncTrayzor
aborsy 5y ago
A couple of questions. * Is there an iOS app on agenda? * Can I choose something like S3 or even Dropbox (yes!) to be a central server (more precisely a peer)?
- commoner 5y ago
  * There doesn't seem to be much progress on an official iOS app, but it is something that users are asking for.[1] An unofficial closed source app called Möbius Sync does exist for iOS.[2] * S3 and Dropbox are not officially supported,[3] but you can always sync one of your Syncthing nodes to cloud storage on your own.[4] [1] https://github.com/syncthing/syncthing/issues/102 https://github.com/syncthing/syncthing/issues/102 [2] https://www.mobiussync.com https://www.mobiussync.com [3] https://forum.syncthing.net/t/how-to-sync-files-with-aws-s3-bucket/14806 https://forum.syncthing.net/t/how-to-sync-files-with-aws-s3-... [4] https://news.ycombinator.com/item?id=18135903 https://news.ycombinator.com/item?id=18135903
  - KennyBlanken 5y ago
    ...the whole point of syncthing is that you avoid "the cloud" and all its privacy nightmares. I'm baffled people do this.
    - heinrichhartman 5y ago
      off-site backup?
    - aborsy 5y ago
      That’s why I put “yes” in parenthesis. Synchting supports end to end encryption. So you can use cloud storage with encrypted folder, for connectivity (no need for relays NAT traversal etc). Clients sync through an always-on peer.
sneak 5y ago
Syncthing is great, but bear in mind in the default configuration (unless you set STNOUPGRADE=1) it will automatically download a replacement binary and exec it, granting the developer remote code execution on your machine at any time, Solarwinds-style. https://news.ycombinator.com/item?id=27933308 https://news.ycombinator.com/item?id=27933308 No-interaction autoupdate really should be opt-in. It's super dangerous software otherwise. Anyone with release credentials could own many many thousands of machines with a single action.
- andrewshadura 5y ago
  Not on Debian though.
  - sneak 5y ago
    ...if you install via apt. Binaries downloaded from the website still have this issue. PS: Thank you Debian maintainers.
  - picardythird 5y ago
    Is this something to worry about if installing via the package manager of a different distribution?
Daegalus 5y ago
I use this for syncing game files (like MMO extensions, ui files. WTF folder when I used to play WoW or the FFXIV folder in my documents) and my code workspace folder between 4 devices, one being a homelab server. For the game files it's just my gaming laptop and gaming desktop I setup a connection for the folder from every device to every other device. Syncthing has a unique ID for the share, so they all share the same ID for that folder and it does all the work syncing. none of my project files are ever out of date on any device. Lets me sit down on any device and just work. (Desktop, windows laptop, Linux laptop, or ssh into server) Another great thing is is you set it up in WSL2, it just works, and all it has to do is be on a different port. Lets me sync my workspace files into there and get full Linux filesystem file watching ( the \\wsl$\ path doesn't work well from windows side)
pacifika 5y ago
It refused a few files with unusual characters in their file name, so you’ll want to periodically check the logs, as it is stricter than the host OS. Apart from that, all good
- fretn 5y ago
  Do you happen to have a list of these unusual characters ?
  - Tor3 5y ago
    It seems to be anything which isn't either ASCII or UTF-8. My father's PC had files written by old versions of OpenOffice (when it was called StarOffice), with "native" names but before his PC was set up to use UTF-8. Early versions of Syncthing didn't have problems with those files, but later versions refused to back them up. I had to rename all those files one by one to get it working again.
    - nickcw 5y ago
      If you ever have to do to this again checkout convmv - this can convert for example latin1 encoded file names into utf8 encoded file names.
      - 12ian34 5y ago
        wish I'd known about this sooner - thanks
snvzz 5y ago
Not a fan[0]. [0]: https://web.archive.org/web/20210226183330/https://www.gkayaalp.com/blog/20190102_syncthing.html https://web.archive.org/web/20210226183330/https://www.gkaya...
- reacharavindh 5y ago
  I stopped taking them seriously when they wrote - “ I saw that it complained that the .stfolders did not exist. So I went ahead and created them, but by mistake, I created empty text files instead of empty folders. Did not realise it and left it to work in the background.” First, backups! Never trust any software to not have a bug. Secondly, if you don’t know what you are doing, may be don’t create random config files and hope the software deals with it. That’s in the “neither power user, nor dumb user” category that most software will not be designed for.
  - nuerow 5y ago
    > Secondly, if you don’t know what you are doing, may be don’t create random config files and hope the software deals with it. This sounds like a variant of the old "you're holding it wrong" blurb. The problem you're commenting on actually describe a multitude of problems with the application, specially how it causes data loss if it's config happens to get corrupted. The comment about the need to back up data also seems to not take into account that the service is already used to back up data across multiple devices.
    - phil-martin 5y ago
      It may have already been used as a backup system, but it was not made for that, and they even answered that in their FAQ. It was this question that stopped me setting it up, because it made me realise I was after a backup solution, not just a synchronisation tool " Is Syncthing my ideal backup application? No. Syncthing is not a great backup application because all changes to your files (modifications, deletions, etc.) will be propagated to all your devices. You can enable versioning, but we encourage you to use other tools to keep your data safe from your (or our) mistakes. " https://docs.syncthing.net/users/faq.html#is-syncthing-my-ideal-backup-application https://docs.syncthing.net/users/faq.html#is-syncthing-my-id...
octopoc 5y ago
Syncthing is not as good as Resilio Sync at syncing huge folders quickly. Resilio Sync can sync 10 MB/s on my wired home internet, but Syncthing usually uses only about 1-8 MB/s. Not sure why.
- MightyOwl13 5y ago
  I started using Syncthing a while ago and its blazing fast. If you do LAN syncs, try setting fixed IPs to your devices in your router and when pairing them in Syncthing use their ips in the address field instead of using Dynamic. So in my case my server would be 10.0.0.1 and I have two devices, let's say 10.0.0.10 and 10.0.0.20, I would putmy servers IP in as tcp://10.0.0.1:22000 in the settings page. You can then even disable most of the doscovery options (if not all). Curios if you will see improvements!
scambier 5y ago
I really like Syncthing, I have it installed on my computer, phone, and VPS, but it's definitely not made for non-technical users. I have a "family" folder with a few important documents that are also shared on my girlfriend's computer and phone, but I use Resilio Sync for those. While Resilio is slightly less secure (debatable) than Syncthing, their UI and UX are vastly better, and virtually no different than something like Dropbox or Google Drive. I fully understand that Syncthing's developers have different priorities than UI and UX, but I'd love to see it become more mainstream.
poisonborz 5y ago
I repaced Dropbox with it a year ago and I'm satisfied, it's a great project. But atm I couldn't recommend it to even knowledgeable friends, let alone family members: - Sometimes long-deleted folders pop up when older version clients connect - Syncing or connecting takes sometimes ages to start for no obvious reason, client not giving enough info on what is happening - Relation between clients is not obvious, the powerful configuration options are hard to decipher after some time - The project could start a (freemium?) cloud client option, maybe leveraging other providers for storage. Most people will not be able to set up a server, and without that it's just way less practical for the layman than any mainstream option.
- rkagerer 5y ago
  Sometimes long-deleted folders pop up when older version clients connect What causes that?
- j1elo 5y ago
  Add to that: - The handling of ignored files is confusing, to say the least. I happened to comment about it just some day ago, so to not repeat the comment -> https://news.ycombinator.com/item?id=28760365 https://news.ycombinator.com/item?id=28760365
francis-io 5y ago
With Android 11, I think this breaks my use case. Android 11 apps, with a few exceptions, can only access files they create. You can add in a "grant all" access, but google limits that to certain apps, like file sharing apps. My use case is I want to "de-cloud" my life, and use syncthing to push things like wireguard keys to my phone, along with backing up photos and pushing down audiobooks and music. I bought a fairphone 3+ with the idea of it being a forever phone, but I couldn't get things working on liniageOS 11 or /e/ os on android 11. I could stay on android 10, but things will eventually migrate to target the new SDK. I also have the issue with syncthing not being able to write to an external SD card, which was one of the selling points for me for the Fairphone. I would love a solution to this, because I'm really stuck in the water on this.
- therealmarv 5y ago
  Maybe I'm not reading it right but I've Android 11 and can see with my gallery app all image files saved to sdcard from all sorts of programs which contradicts your statement. There is still a lot of shared space on the file system. It's a good thing in general that other apps cannot access other's app internal files.
  - rglullis 5y ago
    I'm not an Android dev, but IIUIC your photos and images are made accessible to the gallery app through the ContentProvider API, i.e it does not have direct access.
  - francis-io 5y ago
    Only specific apps will get approval to offer full access to the disk. Gallary app, maybe. Audiobook app from some unknown developer? Maybe not.
- commoner 5y ago
  I don't think the changes in Android 11 are catastrophic. I'm using Syncthing on Android 11, and Syncthing can read/write to any directory in the filesystem under /sdcard, with the exception of the /sdcard/Android/data directory (private app data). Syncthing does request the "All files access" permission that gives it access to most of the filesystem. I'm not sure about external SD cards, though, since I haven't tested this.
xvector 5y ago
Lack of an iOS app is a dealbreaker.
- 12ian34 5y ago
  Switching to Android, then?!
8fingerlouie 5y ago
I used to be a user of both Syncthing and Resilio Sync, but i've replaced both with regular cloud storage with Cryptomator or rclone+crypt instead (Cryptomator for clients). It turns out that the power consumption of keeping redundant hardware running at home is about twice as high as just buying the same storage in the cloud, and then you need to figure in the cost of hardware as well. I.e. Microsoft Family 365 can be had for €60/year (with HUP), and offers 6x1TB OneDrive storage. With power at €0.44/kwh, you have a power budget of 15W before your home server costs more than the cloud offering.
- flipbrad 5y ago
  I use far less than 15W using an old laptop or RasPi their cost is negligible over their useful lifetime
  - 8fingerlouie 5y ago
    The 15W doesn't account for hardware. As soon as you add in hardware (and redundancy), you're looking at a lower budget. If i was to self host something similar, i would use a server (could be RPi4) with a set of 2x6TB drives in RAID1(for availability), and a remote backup somewhere (so another 6TB drive). The hardware cost of a single 6TB (Seagate Ironwolf 6TB, €202 at Amazon) harddrive could pay for 3 years of Family 365. So we're looking at 6 years of Family 365 to buy the harddrives alone (and 9 months more if you buy a RPi 4). I'm not counting the backup drive as you'll need that as well even if running in the cloud. Assuming a lifespan of 5 years, and an operating power of 5W average, you're looking at 44 KWh per harddrive per year, so 2x44x5 = 440 KWh over 5 years, or €194 (at €0.44/KWh). So the total cost of running your own setup over 5 years would be: €404 2 x 6TB harddrives €194 power consumption (not counting server power) Thats €598 for 5 years, or €119.5 per year, and that's also assuming no hardware breaks down. Compare that to the €60 that Family 365 costs per year.
- rglullis 5y ago
  I run a NAS with syncthing and a dappnode at home. The dappnode runs Storj, which I allocated about 2TB. It is paying me now about 6€/month, which is barely enough to cover the electricity costs and the cost of the drive. It certainly hasn't helped to cover any of the cost from the NAS. Decentralized solutions are always going to be less cost-effective than central services. We still should do it anyway. Optimizing for cost is a mistake. It makes us overly dependent on the big companies. It makes us fragile and susceptible to any change in their terms.
raek 5y ago
Regarding backups... Syncthing is great for propagating changes to all devices, but does not offer much in terms of backups and previous versions of files. (And I'm fine with that. I consider that to be out of scope for Syncthing.) I've been thinking about solving the backup part by letting one of the devices perform btrfs snapshots on its storage. Does anyone know about any write-up that describes or compares such solutions? Setting up a cron job / timer unit that commits the synced folder to a git repo is also an alternative, but has some disadvantages: stores contents twice (once in the sync folder, once in the git object database), and pruning older versions is difficult (git is mainly built to keep all history).
- hikarudo 5y ago
  > I've been thinking about solving the backup part by letting one of the devices perform btrfs snapshots on its storage. I do zfs snapshots on my storage server (home desktop), which is in Syncthing's "receive only" mode. I use it to sync my camera roll and Whatsapp media files from my Android phone. I use pyznap for snapshot management. > Setting up a cron job / timer unit that commits the synced folder to a git repo is also an alternative Another option would be to backup the synced folder using a backup tool like borg, restic, duplicacy etc. They have options to prune the snapshots.
  - andrewaylett 5y ago
    I have a script that takes a btrfs snapshot, uses borg to back up the consistent snapshot (to rsync.net), then removes the snapshot again. It then instructs Borg about which snapshots to retain, so my space shouldn't grow without bound. For extra extra snapshotting fun, rsync.net then takes ZFS snapshots of the borg backup. So even if my snapshot cleaning gets over-enthusiastic, I'll still have a snapshot of the snapshot.
- Joeri 5y ago
  I’ve set up a single always on desktop computer as my backup hub. Syncthing is used to sync everything I care about to that machine, and backblaze pulls everything to the cloud as versioned backups. I use a .stignore file to exclude some things from syncing (like node_modules) and similarly backblaze is configured to exclude some things from the cloud backup. There’s also a regular local backup to an external drive (time machine). The setup took some work to figure out, but it is effectively zero maintenance and robust.
MightyOwl13 5y ago
I'm not affiliated with the channel in any way, but Lawrence Systems did a really good overview. It's what convinced me to switch to it and use it for LAN sync with my server keeping everything from my phones, laptops and desktop. It's also good for those who might have trouble understanding how it's supposed to work. I think my biggest difficulty was with the fact that normally it doesn't necessarily have the concept of a "server" per say, but once I understood this and organized it in such a way that my server WOULD behave like a server, the rest was pretty easy. https://www.youtube.com/watch?v=O5O4ajGWZz8 https://www.youtube.com/watch?v=O5O4ajGWZz8 And same url on an Invidious instance - https://yewtu.be/watch?v=O5O4ajGWZz8 https://yewtu.be/watch?v=O5O4ajGWZz8
- j1elo 5y ago
  This guy works on his channel "Raid Owl" and uploads very nicely put videos, I found the channel while searching for homeserver technology videos, and find that he does a great job (but somehow hasn't been discovered by a greater public, judging by the smallish subscribers number) He's using Syncthing for backups (by enabling a feature that keeps N copies of each file), which while possible, I hadn't thought of as a possibility https://www.youtube.com/watch?v=V4kWJ8JcdtM https://www.youtube.com/watch?v=V4kWJ8JcdtM
- wingmanjd 5y ago
  Lawrence Systems videos are great! I've learned so much from them and have implemented quite a few things at $WORK or home due to their videos.
tavish1 5y ago
I really like syncthing. I mainly use it for syncing passwordstore directory and a few other small files that I like having synced. Once set up, I basically forget about it, unless I have to go check why this password is the old one, and in this case it is usually android killing syncthing due to low power more or something, which requires me to manually restart it.
qwertox 5y ago
This seems to rely on inotify which is not always triggered when files change when they are watched. In the following scenario realtime syncing would then fail silently: - You have a server A which contains files which you are editing. - You have setup realtime syncing between server A and a machine B. - On server A you have a Docker container which runs Samba and is hosting the directories of server A which contains the files you are editing. - You are editing those files on a machine where you mounted those shares. Upon saving, the files on server A would be modified, but inotify would not trigger and the file would not get synced to machine B.
- zufallsheld 5y ago
  Did this happen to you? This seems like a very contrived example of a problem.
  - qwertox 5y ago
    It didn't happen with Syncthing because I'm not using it (yet!), but while trying to edit files normally which were watched by inotify. I'm kind of assuming that it's the containerization of Samba which is causing the problem, since I have not installed Samba directly on the server, but it could also be the case when not using Docker. Apparently any CIFS mount is affected https://lists.samba.org/archive/linux-cifs-client/2009-April/004319.html https://lists.samba.org/archive/linux-cifs-client/2009-April...
- Tor3 5y ago
  Syncthing has two methods for detecting file changes: Watching (inotify), and Scanning. Both are enabled by default, with scanning once per hour (+/- a random element). Configurable per directory. https://docs.syncthing.net/users/syncing.html https://docs.syncthing.net/users/syncing.html
  - qwertox 5y ago
    Excellent, that's good to know.
pr8dan 5y ago
Love it. Have an "always-on" raspberry pi with usb drive just running syncthing in my home network. That way, I don't have my data "in the cloud" somewhere, I don't have to open my home network for external access to my NAS etc, and always have an up to date "source" when I start syncthing on a tablet or mobile (where i dont have them running all the time).
raman162 5y ago
Syncthing is really amazing, I have been using it for nearly two years to sync notes on all of my device; three laptops, one iPad, raspberry pi and a pixel 2. The most difficult thing was finding a solution to work on the iPad, ended up having to use an app to remotely access and edit files on my raspberry pi, it's kinda clunky but gets the job done.
harvie 5y ago
What about https://www.syncany.org/ https://www.syncany.org/ It seems to be abandoned, but i really like the idea about using arbitrary storage for bidirectional sync. Eg.: use any SSH (SFTP) server as a dropbox alternative.
grandchild 5y ago
I am a huge Syncthing fan. I even recommend it to non-techie friends, and (after a bit of initial hesitance from having to learn the concepts) they are quite fond of it too, especially as a smartphone backup solution. It's especially useful for our band, where we share large amounts of .wav files and recording projects and so on. It even works in the rehearsal room (which has no internet) through an old wifi router I set up there, that each of our laptops connects to.
rkagerer 5y ago
Glad to see another option out there for file sync. Browsing the FAQ I noticed: To further limit the amount of CPU used when syncing and scanning, set the environment variable GOMAXPROCS to the maximum number of CPU cores Syncthing should use Are they using an environment variable shared by other Go programs? Seems like bad practice... why aren't they using a variable or or switch that's unique to Syncthing instead?
- hparadiz 5y ago
  It's a GO runtime setting. Has nothing to do with Syncthing.
  - rkagerer 5y ago
    I'm not a Go programmer, but couldn't they achieve the same thing without name collision by reading a custom environment variable at startup and setting runtime.GOMAXPROCS?
    - vetinari 5y ago
      Why would they do it? This setting is managed by Go runtime, and Go runtime defines a specific env var for customization. If you want different setting for different Go programs, no problem, it is not a global config, but per process, so set the env var to desired value only for syncthing process.
- fps_doug 5y ago
  > Are they using an environment variable shared by other Go prpgrams? Seems like bad practice... Huh? So only set it for syncthing.
  - rkagerer 5y ago
    Easy on Linux, not so much (and messier) on Windows.
- OriginalPenguin 5y ago
miki123211 5y ago
I have to pitch Resilio Sync (formerly Bittorrent Sync) here. It's a proprietary, more polished alternative that offers a freeware license for personal use. My biggest issue with Syncthing was that there was no auto-discovery, if you had 25 or so computers across a group of friends, you needed to set up 25^2 connections, which wasn't really feasible. Not sure if this is still a thing, but Resilio solves this perfectly, you just input a secret key and computers just connect automatically. The developers also provide turn servers, which is useful in restrictive NAT scenarios. End-to-end encryption is supported, both in the free and the pro versions, although the latter gives you much more control over who can do what to the folder. I use Resilio as a "big Dropbox" with dozens of gigabytes of data per folder, shared across a small-ish group of trusted friends. It doesn't cost us anything and works really well. The largest folders I've seen had over a terabyte of data in them, but you probably want the pro version (with selective sync) for those.
- fps_doug 5y ago
  Syncthing supports introducer devices. If you mark a device as introducer, you'll automatically add all devices the introducer knows to your list. So basically adding a new device to an existing group requires everyone to agree on one introducer, and everyone will add the new device automatically. NAT traversal and relay servers exist too. So seems it's pretty much on par with Resilio, plus open source.
  - miki123211 5y ago
    For my specific use case, there's probably no device that could be an introducer. It's all PCs, none of them online 24/7. Besides, setting all that up requires way more technical knowledge than most people have. Sure, I could do it I guess, but explaining Resilio configuration to complete non-techies is hard enough.
    - NilsIRL 5y ago
      Introducer devices don't have to be online all the time. There is also no need to limit the number of introducer devices. I have a folder that's synced between my laptop, desktop and phone, and each has the other 2 devices as introducer.
aasasd 5y ago
On Android, Syncthing-fork is a bit nicer because it has a native app interface instead of the embedded web interface. However, on Android 11 the Play Store release doesn't work with folders other than /Android/media or the app's data folder or something like that—because of Google's continuous crusade against file access, effective for the newer API level that Syncthing-fork is targeting. But the F-Droid version doesn't have this problem. Note that the mainstream Syncthing on Android works so far only because they're targeting an older API version. They will hit the same restriction if they migrate to the new API, unless they implement SAF or whatever.
- nyanpasu64 5y ago
  Syncthing-Fork can't send broadcast mDNS packets on Android 11. As a result, two Android 11 devices can't even connect to each other over a LAN, and are reduced to sending all traffic to the Internet and back, which wastes bandwidth, probably hurts privacy, and throttles transfer speeds. The workaround I found was to hard-code one device's IP address in the other. Perhaps Syncthing is unaffected; I didn't test. Link: https://forum.syncthing.net/t/local-discovery-problem-on-android-11/16093/19 https://forum.syncthing.net/t/local-discovery-problem-on-and...
  - aasasd 5y ago
    Hmmm, I guess it only affects discovery between phones, because I just recently re-set up syncing with two laptops, and it worked fine without entering the IDs or ips or anything—just clicked on the discovered IDs. Of course, this issue still highlights how the ‘open-source’ and ‘customizable’ OS does only what Google allows it to do.
SanchoPanda 5y ago
I would like to give a shout out to `unison` for syncing needs. It can be used with a daemon, but I use it happily with cron and it has been absolutely bulletproof* for years. I have never noticed it's resource usage at all after running the initial sync. I am never modifying the same file on two systems simultaneously, so I can afford to sync hourly with no issues. While not everyone can do this, I suspect many are in the same boat. * Try to use literally the exact same binary where architectures allow, due to build dependencies.
- passthejoe 5y ago
  I have used both. I'm a BIG Unison fan.
bsdubernerd 5y ago
syncthing is excellent for sharing also between groups. I wish it worked better for occasional file sharing (for example to transfer files between phone/pc). I don't want to keep it running in the background - just sucks too much power, and on the phone I never need to keep other stuff synched. But the time it takes for the first connect to happen after starting it can be in the order if minutes on a known network, dozens of minutes if not. Not usable for this purpose. The android client is also not very well thought out. You cannot "share to syncthing" until it's active, so if you want to put a file for sharing until your network connectivity is up, you can't. You have to start it, share, then shut it down. If you want to do that on an unknown network you didn't whitelist, you have to whitelist the network as well. Kind of pointless. You'd think you can also start it when it prompts you to change the settings, but this also doesn't work. I ended up disabling all these advanced settings, as they clearly end-up being an hindrance. I just start it manually and quit it, kind of defeating the entire purpose of them.
- inside_out_life 5y ago
  for fast occasional sharing you have KDEConnect
  - bsdubernerd 5y ago
    I don't see a command-line or daemon backend. Not really comparable, even if I happened to use kde.
    - inside_out_life 5y ago
      Well, it's not comparable, you wanted different use case there's kdeconnect-cli kdeconnect seems daemon like enough to me since it's entirely detached from its myriad UIs
flanbiscuit 5y ago
I use Syncthing to sync my KeepassXC db file between laptops and my android, always on a local network. For the most part I like it. I get a lot of conflicts though and I can't use ediff (or similar) because the file is encrypted. Curious if anyone has figured out how to fix conflicts with Keypass db files because I don't think Syncthing's conflict resolution can handle them very well.
- Disparallel 5y ago
  KeepassXC has built in db file merge support! It's hidden somewhere in the menus. I've never personally had to do something more complicated than "I added a new entry on two different devices before syncing" but it's worked well for me so far.
  - rcMgD2BwE72F 5y ago
    >It's hidden somewhere in the menus File > Merge with database I saved my database's password as an entry just to quickly merge it with the .sync-conflict version. It takes 2 seconds, and I don't worry about losing anything anymore. Works beautifully for me. KeePass could check for such conflicts (syncthing, dropbox or else) in the directory where the open database is stored and suggest to auto-merge.
jerieljan 5y ago
Syncthing is one of the biggest reasons why I use Android today. That sounds strange, but when you have old spare phones, they're very useful for this. Even for the daily driver phone that I use, it's so good at keeping files in sync between my other PCs.
mkr-hn 5y ago
The only thing that keeps me from using it is the lack of an official iOS client. Once they get that, I'll be all over it.
keithwbacon 5y ago
How fast is this vs rsync over ssh?
llasse 5y ago
Is it possible to access SD card folders on the ohne by now? That used to be the major drawback some years ago for me..

I recently started using this to sync a few folders full of some code and documents between three of my computers. So far, it works very nicely, to the point I don't think about which machine I sit down at, because I know it'll already have the stuff I'm working on, all ready to go, without any fussing around or needing to running `git pull` in a bunch of different folders to get caught up to where I left off. I like that there's no cloud dependency and no single point of failure.

nett18 5y ago

Something that interests me even more is that the company behind, Kastelo Inc., possibly bears its name from the esperantoan word for 'Castle'. Can anyone confirm?

commoner 5y ago
You're probably right. Kastelo previously launched a service called Arigi that was later discontinued. The announcement notes that "arigi" means "to amass · to gather · to put together" in Esperanto. https://webcache.googleusercontent.com/search?q=cache:IAwDkf0wpYwJ:https://kastelo.net/blog/2017-09/introducing-arigi/ https://webcache.googleusercontent.com/search?q=cache:IAwDkf...
- 0xADADA 5y ago
  or its a tech startup that just named it 'Castle' spelled with a K-elo to make it unique

bedros 5y ago

only if I can filter what to sync locally

commoner 5y ago
Here's Syncthing's documentation on specifying files to ignore or include using the .stignore file: https://docs.syncthing.net/users/ignoring.html https://docs.syncthing.net/users/ignoring.html This also works in the web and Android clients, which expose .stignore through the "Ignore Patterns" setting.
KennyBlanken 5y ago
https://www.google.com/search?q=syncthing+filter https://www.google.com/search?q=syncthing+filter
- prirai 5y ago
  Google, really.
rg111 5y ago
Yes you can.

asicsp 5y ago

GitHub repo: https://github.com/syncthing/syncthing https://github.com/syncthing/syncthing Some previous discussions: * "Syncthing: Open Source Dropbox and BitTorrent Sync Replacement" https://news.ycombinator.com/item?id=7734114 https://news.ycombinator.com/item?id=7734114 (7 years ago|184 comments) * "Syncthing is everything I used to love about computers" https://news.ycombinator.com/item?id=23537243 https://news.ycombinator.com/item?id=23537243 (1 year ago|159 comments) * "Syncthing: Syncing All the Things" https://news.ycombinator.com/item?id=27929194 https://news.ycombinator.com/item?id=27929194 (3 months ago|172 comments)

KennyBlanken 5y ago

Syncthing works brilliantly! The web UI is excellent, warns you when you're about to do something ill-advised, and stuff like QR codes makes adding clients and folders fairly easy. The separation between folders and devices is handled well. You can easily have half a dozen shared folders between several computers with any mix-and-match combination including which one is 'authoritative' and so on. I used to use it on Android to keep my 'camera roll' synced to my desktop. I preferred setting the ignore file to exclude android's thumbnails directory, but otherwise it worked great. I'd take a photo with my phone and almost before I could open the shortcut on the desktop, boom, the photo was there, since the android client detects changes in the filesystem. I also used it to sync my password database, and a cross-platform notebook app's database. Worked flawlessly for all of them. Ditto for a folder shared between my MacOS laptop and Windows desktop. Lots of controls, especially in the forked android client. Don't want the sync client to run on any network other than your home network? Done. Don't want your syncthing clients to try and do NAT transversal / discovery outside whatever network it's on? No problem. The biggest bummer is that there's no iOS client. Had to switch to Nextcloud, which so far has been working OK, but the number of people that have problems with the iOS client is quite high (problems such as syncs being so slow it takes days to sync a camera roll), and it's generally sluggish and doesn't work in the background (yes, I know, Apple's fault.) However, Nextcloud does allow me to sync contacts and calendars (I think. I haven't tried to set it up yet.) One warning: Sycnthing really does not like it when you delete the dotfile folder inside the shared folder(s). Don't do that :)

_emacsomancer_ 5y ago
There's Mobius, a third party client for iOS: https://www.mobiussync.com/ https://www.mobiussync.com/ . I've never used it, so I don't know how good it is.
Reventlov 5y ago
I would not qualify the web UI of "excellent" : understanding the relations between paired devices, remote devices, folders you want to sync, and so on is harder than what it should be. Likewise, the android application is so bad the best way is to open the web UI on your phone to do stuff (and sometimes, it's the only way to achieve some goals). There is definitely a lot of room for improvement for easier adoption, yet, this is one of the few sync app that work for me, even behind NAT, with closed ports, and so on.

icheyne 5y ago

I could never get this to work on my home network. ¯\_(ツ)_/¯

prirai 5y ago
You have to force it to use wlan or wireless. It solves the problem. Also, wait for a few moments for it to be able to detect your other device.
mkl 5y ago
If you're using Windows, make sure Windows doesn't think your network is public. My dad's computers thought they were on a public network and so wouldn't see each other directly (they were syncing though, very slowly via an external relay).

gnulinux 5y ago

Is there an obvious way to use this with rclone?

stef25 5y ago
What are you trying to do?
jorislacance 5y ago
I guess it is more one or the other depending on the use case. I've been synchronizing local data to remote data using NFS and Syncthing for a while now instead of doing batches of rclone, if you were referring to this use case.
ncphil 5y ago
... I actually use rclone mount _instead_ of sync for cloud storage. Tried syncthing a long while ago after finding nextcloud unreliable. Then stumbled acrosss a comment somewhere that talked about the mount feature in rclone (https://rclone.org/commands/rclone_mount/ https://rclone.org/commands/rclone_mount/). This is probably only practical for me now because we've got a 1G/1G pipe to the Internet. On the internal network I'm back to using nfs mounts (or samba for the Windows machines). Syncthing is good, but over time I've found all sync solutions to fall short. For backups I use a combination of rsync to the file server on both Linux and Windows, and then rclone out to the cloud.

iszomer 5y ago

It's sufficient enough for my needs. I use Joplin with it as well as no-frills file syncing across my home network. Just a note on revision history: it's better to configure this feature on the notetaking app alone while disabling it on Syncthing. My only gripe is that this utility may soon become deprecated as Android continues to update their convoluted storage schemes.

deaddabe 5y ago
Can you share a reference wrt evolving Android storage schemes?
- commoner 5y ago
  Android 11+ enforces scoped storage, which restricts access to the filesystem in many cases.[1] Syncthing requests the "All files access" permission[2] to get around this, but the /sdcard/Android/data directory (private app data) is still off limits.[3] [1] https://source.android.com/devices/storage/scoped https://source.android.com/devices/storage/scoped [2] https://developer.android.com/training/data-storage/manage-all-files https://developer.android.com/training/data-storage/manage-a... [3] https://github.com/syncthing/syncthing-android/issues/1638 https://github.com/syncthing/syncthing-android/issues/1638

jwr 5y ago

Syncthing has been working for me flawlessly over the last year. Very impressed by it, especially compared to Dropbox, which is a CPU hog and has become a nagging salesperson on my computer. Highly recommended.

amanzi 5y ago

Nothing but good things to say about SyncThings. I'm only using it on a couple of computers to keep a small about of things in sync, but it always works flawlessly.

deaddabe 5y ago

I have switched from Nextcloud to Syncthing. The former was a resource hog that was eating up too much everything on my modest VPS. I share the files on my local network at home between phone, laptop and desktop. My only issues so far: - I need to manually start Syncthing on the devices, whereas Nextcloud was always running on boot. Not sure if I should start it every time, especially on the Android phone which cannot sync while away from home. I do not want to empty the battery while outside of my LAN, + the persistent notification "Syncthing is running" is annoying. - I need to power up at least the laptop or desktop to backup my phone data. One solution that I will try is to install a headless Syncthing instance on my NAS. This way the phone can always sync to the NAS every time, and then laptop/desktop can sync from NAS when needed. The joys of peer2peer! At least, I am sure that my moderately sensitive files are not leaving my local network to be stored on my VPS, which can be hacked someday. I also need to figure out how to install Wireguard on my router to allow to access my NAS from my phone while on a trip, but that's another story.

Maakuth 5y ago
I've dealt with the Android issue by configuring Syncthing to only run when the phone is charging. I'm in the habit of leaving my phone to charger every night anyway, so I'm regularly synced. It would be great if there would be a 'Sync now' function though, that would run the sync and then go back to ordinary mode afterwards, for the rare case that I need my folder synced in the instant.
Daegalus 5y ago
try out Synctrayzor for the laptop and desktop. it lets you set it to auto start on boot.
- deaddabe 5y ago
  Thanks for the suggestion. This is an app for Windows [0] but everything I use is Linux, so the setup to start SyncThing it at boot should be straightforward. I am more worried about the Android part. The best solution would be to start SyncThing when on my LAN via matching SSID, and disable it when I am out of range. [0] https://github.com/canton7/SyncTrayzor https://github.com/canton7/SyncTrayzor

aborsy 5y ago

A couple of questions. * Is there an iOS app on agenda? * Can I choose something like S3 or even Dropbox (yes!) to be a central server (more precisely a peer)?

commoner 5y ago
* There doesn't seem to be much progress on an official iOS app, but it is something that users are asking for.[1] An unofficial closed source app called Möbius Sync does exist for iOS.[2] * S3 and Dropbox are not officially supported,[3] but you can always sync one of your Syncthing nodes to cloud storage on your own.[4] [1] https://github.com/syncthing/syncthing/issues/102 https://github.com/syncthing/syncthing/issues/102 [2] https://www.mobiussync.com https://www.mobiussync.com [3] https://forum.syncthing.net/t/how-to-sync-files-with-aws-s3-bucket/14806 https://forum.syncthing.net/t/how-to-sync-files-with-aws-s3-... [4] https://news.ycombinator.com/item?id=18135903 https://news.ycombinator.com/item?id=18135903
- KennyBlanken 5y ago
  ...the whole point of syncthing is that you avoid "the cloud" and all its privacy nightmares. I'm baffled people do this.
  - heinrichhartman 5y ago
    off-site backup?
  - aborsy 5y ago
    That’s why I put “yes” in parenthesis. Synchting supports end to end encryption. So you can use cloud storage with encrypted folder, for connectivity (no need for relays NAT traversal etc). Clients sync through an always-on peer.

sneak 5y ago

Syncthing is great, but bear in mind in the default configuration (unless you set STNOUPGRADE=1) it will automatically download a replacement binary and exec it, granting the developer remote code execution on your machine at any time, Solarwinds-style. https://news.ycombinator.com/item?id=27933308 https://news.ycombinator.com/item?id=27933308 No-interaction autoupdate really should be opt-in. It's super dangerous software otherwise. Anyone with release credentials could own many many thousands of machines with a single action.

andrewshadura 5y ago
Not on Debian though.
- sneak 5y ago
  ...if you install via apt. Binaries downloaded from the website still have this issue. PS: Thank you Debian maintainers.
- picardythird 5y ago
  Is this something to worry about if installing via the package manager of a different distribution?

Daegalus 5y ago

I use this for syncing game files (like MMO extensions, ui files. WTF folder when I used to play WoW or the FFXIV folder in my documents) and my code workspace folder between 4 devices, one being a homelab server. For the game files it's just my gaming laptop and gaming desktop I setup a connection for the folder from every device to every other device. Syncthing has a unique ID for the share, so they all share the same ID for that folder and it does all the work syncing. none of my project files are ever out of date on any device. Lets me sit down on any device and just work. (Desktop, windows laptop, Linux laptop, or ssh into server) Another great thing is is you set it up in WSL2, it just works, and all it has to do is be on a different port. Lets me sync my workspace files into there and get full Linux filesystem file watching ( the \\wsl$\ path doesn't work well from windows side)

pacifika 5y ago

It refused a few files with unusual characters in their file name, so you’ll want to periodically check the logs, as it is stricter than the host OS. Apart from that, all good

fretn 5y ago
Do you happen to have a list of these unusual characters ?
- Tor3 5y ago
  It seems to be anything which isn't either ASCII or UTF-8. My father's PC had files written by old versions of OpenOffice (when it was called StarOffice), with "native" names but before his PC was set up to use UTF-8. Early versions of Syncthing didn't have problems with those files, but later versions refused to back them up. I had to rename all those files one by one to get it working again.
  - nickcw 5y ago
    If you ever have to do to this again checkout convmv - this can convert for example latin1 encoded file names into utf8 encoded file names.
    - 12ian34 5y ago
      wish I'd known about this sooner - thanks

snvzz 5y ago

Not a fan[0]. [0]: https://web.archive.org/web/20210226183330/https://www.gkayaalp.com/blog/20190102_syncthing.html https://web.archive.org/web/20210226183330/https://www.gkaya...

reacharavindh 5y ago
I stopped taking them seriously when they wrote - “ I saw that it complained that the .stfolders did not exist. So I went ahead and created them, but by mistake, I created empty text files instead of empty folders. Did not realise it and left it to work in the background.” First, backups! Never trust any software to not have a bug. Secondly, if you don’t know what you are doing, may be don’t create random config files and hope the software deals with it. That’s in the “neither power user, nor dumb user” category that most software will not be designed for.
- nuerow 5y ago
  > Secondly, if you don’t know what you are doing, may be don’t create random config files and hope the software deals with it. This sounds like a variant of the old "you're holding it wrong" blurb. The problem you're commenting on actually describe a multitude of problems with the application, specially how it causes data loss if it's config happens to get corrupted. The comment about the need to back up data also seems to not take into account that the service is already used to back up data across multiple devices.
  - phil-martin 5y ago
    It may have already been used as a backup system, but it was not made for that, and they even answered that in their FAQ. It was this question that stopped me setting it up, because it made me realise I was after a backup solution, not just a synchronisation tool " Is Syncthing my ideal backup application? No. Syncthing is not a great backup application because all changes to your files (modifications, deletions, etc.) will be propagated to all your devices. You can enable versioning, but we encourage you to use other tools to keep your data safe from your (or our) mistakes. " https://docs.syncthing.net/users/faq.html#is-syncthing-my-ideal-backup-application https://docs.syncthing.net/users/faq.html#is-syncthing-my-id...

octopoc 5y ago

Syncthing is not as good as Resilio Sync at syncing huge folders quickly. Resilio Sync can sync 10 MB/s on my wired home internet, but Syncthing usually uses only about 1-8 MB/s. Not sure why.

MightyOwl13 5y ago
I started using Syncthing a while ago and its blazing fast. If you do LAN syncs, try setting fixed IPs to your devices in your router and when pairing them in Syncthing use their ips in the address field instead of using Dynamic. So in my case my server would be 10.0.0.1 and I have two devices, let's say 10.0.0.10 and 10.0.0.20, I would putmy servers IP in as tcp://10.0.0.1:22000 in the settings page. You can then even disable most of the doscovery options (if not all). Curios if you will see improvements!

scambier 5y ago

I really like Syncthing, I have it installed on my computer, phone, and VPS, but it's definitely not made for non-technical users. I have a "family" folder with a few important documents that are also shared on my girlfriend's computer and phone, but I use Resilio Sync for those. While Resilio is slightly less secure (debatable) than Syncthing, their UI and UX are vastly better, and virtually no different than something like Dropbox or Google Drive. I fully understand that Syncthing's developers have different priorities than UI and UX, but I'd love to see it become more mainstream.

poisonborz 5y ago

I repaced Dropbox with it a year ago and I'm satisfied, it's a great project. But atm I couldn't recommend it to even knowledgeable friends, let alone family members: - Sometimes long-deleted folders pop up when older version clients connect - Syncing or connecting takes sometimes ages to start for no obvious reason, client not giving enough info on what is happening - Relation between clients is not obvious, the powerful configuration options are hard to decipher after some time - The project could start a (freemium?) cloud client option, maybe leveraging other providers for storage. Most people will not be able to set up a server, and without that it's just way less practical for the layman than any mainstream option.

rkagerer 5y ago
Sometimes long-deleted folders pop up when older version clients connect What causes that?
j1elo 5y ago
Add to that: - The handling of ignored files is confusing, to say the least. I happened to comment about it just some day ago, so to not repeat the comment -> https://news.ycombinator.com/item?id=28760365 https://news.ycombinator.com/item?id=28760365

francis-io 5y ago

With Android 11, I think this breaks my use case. Android 11 apps, with a few exceptions, can only access files they create. You can add in a "grant all" access, but google limits that to certain apps, like file sharing apps. My use case is I want to "de-cloud" my life, and use syncthing to push things like wireguard keys to my phone, along with backing up photos and pushing down audiobooks and music. I bought a fairphone 3+ with the idea of it being a forever phone, but I couldn't get things working on liniageOS 11 or /e/ os on android 11. I could stay on android 10, but things will eventually migrate to target the new SDK. I also have the issue with syncthing not being able to write to an external SD card, which was one of the selling points for me for the Fairphone. I would love a solution to this, because I'm really stuck in the water on this.

therealmarv 5y ago
Maybe I'm not reading it right but I've Android 11 and can see with my gallery app all image files saved to sdcard from all sorts of programs which contradicts your statement. There is still a lot of shared space on the file system. It's a good thing in general that other apps cannot access other's app internal files.
- rglullis 5y ago
  I'm not an Android dev, but IIUIC your photos and images are made accessible to the gallery app through the ContentProvider API, i.e it does not have direct access.
- francis-io 5y ago
  Only specific apps will get approval to offer full access to the disk. Gallary app, maybe. Audiobook app from some unknown developer? Maybe not.
commoner 5y ago
I don't think the changes in Android 11 are catastrophic. I'm using Syncthing on Android 11, and Syncthing can read/write to any directory in the filesystem under /sdcard, with the exception of the /sdcard/Android/data directory (private app data). Syncthing does request the "All files access" permission that gives it access to most of the filesystem. I'm not sure about external SD cards, though, since I haven't tested this.

xvector 5y ago

Lack of an iOS app is a dealbreaker.

12ian34 5y ago
Switching to Android, then?!

8fingerlouie 5y ago

I used to be a user of both Syncthing and Resilio Sync, but i've replaced both with regular cloud storage with Cryptomator or rclone+crypt instead (Cryptomator for clients). It turns out that the power consumption of keeping redundant hardware running at home is about twice as high as just buying the same storage in the cloud, and then you need to figure in the cost of hardware as well. I.e. Microsoft Family 365 can be had for €60/year (with HUP), and offers 6x1TB OneDrive storage. With power at €0.44/kwh, you have a power budget of 15W before your home server costs more than the cloud offering.

flipbrad 5y ago
I use far less than 15W using an old laptop or RasPi their cost is negligible over their useful lifetime
- 8fingerlouie 5y ago
  The 15W doesn't account for hardware. As soon as you add in hardware (and redundancy), you're looking at a lower budget. If i was to self host something similar, i would use a server (could be RPi4) with a set of 2x6TB drives in RAID1(for availability), and a remote backup somewhere (so another 6TB drive). The hardware cost of a single 6TB (Seagate Ironwolf 6TB, €202 at Amazon) harddrive could pay for 3 years of Family 365. So we're looking at 6 years of Family 365 to buy the harddrives alone (and 9 months more if you buy a RPi 4). I'm not counting the backup drive as you'll need that as well even if running in the cloud. Assuming a lifespan of 5 years, and an operating power of 5W average, you're looking at 44 KWh per harddrive per year, so 2x44x5 = 440 KWh over 5 years, or €194 (at €0.44/KWh). So the total cost of running your own setup over 5 years would be: €404 2 x 6TB harddrives €194 power consumption (not counting server power) Thats €598 for 5 years, or €119.5 per year, and that's also assuming no hardware breaks down. Compare that to the €60 that Family 365 costs per year.
rglullis 5y ago
I run a NAS with syncthing and a dappnode at home. The dappnode runs Storj, which I allocated about 2TB. It is paying me now about 6€/month, which is barely enough to cover the electricity costs and the cost of the drive. It certainly hasn't helped to cover any of the cost from the NAS. Decentralized solutions are always going to be less cost-effective than central services. We still should do it anyway. Optimizing for cost is a mistake. It makes us overly dependent on the big companies. It makes us fragile and susceptible to any change in their terms.

raek 5y ago

Regarding backups... Syncthing is great for propagating changes to all devices, but does not offer much in terms of backups and previous versions of files. (And I'm fine with that. I consider that to be out of scope for Syncthing.) I've been thinking about solving the backup part by letting one of the devices perform btrfs snapshots on its storage. Does anyone know about any write-up that describes or compares such solutions? Setting up a cron job / timer unit that commits the synced folder to a git repo is also an alternative, but has some disadvantages: stores contents twice (once in the sync folder, once in the git object database), and pruning older versions is difficult (git is mainly built to keep all history).

hikarudo 5y ago
> I've been thinking about solving the backup part by letting one of the devices perform btrfs snapshots on its storage. I do zfs snapshots on my storage server (home desktop), which is in Syncthing's "receive only" mode. I use it to sync my camera roll and Whatsapp media files from my Android phone. I use pyznap for snapshot management. > Setting up a cron job / timer unit that commits the synced folder to a git repo is also an alternative Another option would be to backup the synced folder using a backup tool like borg, restic, duplicacy etc. They have options to prune the snapshots.
- andrewaylett 5y ago
  I have a script that takes a btrfs snapshot, uses borg to back up the consistent snapshot (to rsync.net), then removes the snapshot again. It then instructs Borg about which snapshots to retain, so my space shouldn't grow without bound. For extra extra snapshotting fun, rsync.net then takes ZFS snapshots of the borg backup. So even if my snapshot cleaning gets over-enthusiastic, I'll still have a snapshot of the snapshot.
Joeri 5y ago
I’ve set up a single always on desktop computer as my backup hub. Syncthing is used to sync everything I care about to that machine, and backblaze pulls everything to the cloud as versioned backups. I use a .stignore file to exclude some things from syncing (like node_modules) and similarly backblaze is configured to exclude some things from the cloud backup. There’s also a regular local backup to an external drive (time machine). The setup took some work to figure out, but it is effectively zero maintenance and robust.

MightyOwl13 5y ago

I'm not affiliated with the channel in any way, but Lawrence Systems did a really good overview. It's what convinced me to switch to it and use it for LAN sync with my server keeping everything from my phones, laptops and desktop. It's also good for those who might have trouble understanding how it's supposed to work. I think my biggest difficulty was with the fact that normally it doesn't necessarily have the concept of a "server" per say, but once I understood this and organized it in such a way that my server WOULD behave like a server, the rest was pretty easy. https://www.youtube.com/watch?v=O5O4ajGWZz8 https://www.youtube.com/watch?v=O5O4ajGWZz8 And same url on an Invidious instance - https://yewtu.be/watch?v=O5O4ajGWZz8 https://yewtu.be/watch?v=O5O4ajGWZz8

j1elo 5y ago
This guy works on his channel "Raid Owl" and uploads very nicely put videos, I found the channel while searching for homeserver technology videos, and find that he does a great job (but somehow hasn't been discovered by a greater public, judging by the smallish subscribers number) He's using Syncthing for backups (by enabling a feature that keeps N copies of each file), which while possible, I hadn't thought of as a possibility https://www.youtube.com/watch?v=V4kWJ8JcdtM https://www.youtube.com/watch?v=V4kWJ8JcdtM
wingmanjd 5y ago
Lawrence Systems videos are great! I've learned so much from them and have implemented quite a few things at $WORK or home due to their videos.

tavish1 5y ago

I really like syncthing. I mainly use it for syncing passwordstore directory and a few other small files that I like having synced. Once set up, I basically forget about it, unless I have to go check why this password is the old one, and in this case it is usually android killing syncthing due to low power more or something, which requires me to manually restart it.

qwertox 5y ago

This seems to rely on inotify which is not always triggered when files change when they are watched. In the following scenario realtime syncing would then fail silently: - You have a server A which contains files which you are editing. - You have setup realtime syncing between server A and a machine B. - On server A you have a Docker container which runs Samba and is hosting the directories of server A which contains the files you are editing. - You are editing those files on a machine where you mounted those shares. Upon saving, the files on server A would be modified, but inotify would not trigger and the file would not get synced to machine B.

zufallsheld 5y ago
Did this happen to you? This seems like a very contrived example of a problem.
- qwertox 5y ago
  It didn't happen with Syncthing because I'm not using it (yet!), but while trying to edit files normally which were watched by inotify. I'm kind of assuming that it's the containerization of Samba which is causing the problem, since I have not installed Samba directly on the server, but it could also be the case when not using Docker. Apparently any CIFS mount is affected https://lists.samba.org/archive/linux-cifs-client/2009-April/004319.html https://lists.samba.org/archive/linux-cifs-client/2009-April...
Tor3 5y ago
Syncthing has two methods for detecting file changes: Watching (inotify), and Scanning. Both are enabled by default, with scanning once per hour (+/- a random element). Configurable per directory. https://docs.syncthing.net/users/syncing.html https://docs.syncthing.net/users/syncing.html
- qwertox 5y ago
  Excellent, that's good to know.

pr8dan 5y ago

Love it. Have an "always-on" raspberry pi with usb drive just running syncthing in my home network. That way, I don't have my data "in the cloud" somewhere, I don't have to open my home network for external access to my NAS etc, and always have an up to date "source" when I start syncthing on a tablet or mobile (where i dont have them running all the time).

raman162 5y ago

Syncthing is really amazing, I have been using it for nearly two years to sync notes on all of my device; three laptops, one iPad, raspberry pi and a pixel 2. The most difficult thing was finding a solution to work on the iPad, ended up having to use an app to remotely access and edit files on my raspberry pi, it's kinda clunky but gets the job done.

harvie 5y ago

What about https://www.syncany.org/ https://www.syncany.org/ It seems to be abandoned, but i really like the idea about using arbitrary storage for bidirectional sync. Eg.: use any SSH (SFTP) server as a dropbox alternative.

grandchild 5y ago

I am a huge Syncthing fan. I even recommend it to non-techie friends, and (after a bit of initial hesitance from having to learn the concepts) they are quite fond of it too, especially as a smartphone backup solution. It's especially useful for our band, where we share large amounts of .wav files and recording projects and so on. It even works in the rehearsal room (which has no internet) through an old wifi router I set up there, that each of our laptops connects to.

rkagerer 5y ago

Glad to see another option out there for file sync. Browsing the FAQ I noticed: To further limit the amount of CPU used when syncing and scanning, set the environment variable GOMAXPROCS to the maximum number of CPU cores Syncthing should use Are they using an environment variable shared by other Go programs? Seems like bad practice... why aren't they using a variable or or switch that's unique to Syncthing instead?

hparadiz 5y ago
It's a GO runtime setting. Has nothing to do with Syncthing.
- rkagerer 5y ago
  I'm not a Go programmer, but couldn't they achieve the same thing without name collision by reading a custom environment variable at startup and setting runtime.GOMAXPROCS?
  - vetinari 5y ago
    Why would they do it? This setting is managed by Go runtime, and Go runtime defines a specific env var for customization. If you want different setting for different Go programs, no problem, it is not a global config, but per process, so set the env var to desired value only for syncthing process.
fps_doug 5y ago
> Are they using an environment variable shared by other Go prpgrams? Seems like bad practice... Huh? So only set it for syncthing.
- rkagerer 5y ago
  Easy on Linux, not so much (and messier) on Windows.
OriginalPenguin 5y ago

miki123211 5y ago

I have to pitch Resilio Sync (formerly Bittorrent Sync) here. It's a proprietary, more polished alternative that offers a freeware license for personal use. My biggest issue with Syncthing was that there was no auto-discovery, if you had 25 or so computers across a group of friends, you needed to set up 25^2 connections, which wasn't really feasible. Not sure if this is still a thing, but Resilio solves this perfectly, you just input a secret key and computers just connect automatically. The developers also provide turn servers, which is useful in restrictive NAT scenarios. End-to-end encryption is supported, both in the free and the pro versions, although the latter gives you much more control over who can do what to the folder. I use Resilio as a "big Dropbox" with dozens of gigabytes of data per folder, shared across a small-ish group of trusted friends. It doesn't cost us anything and works really well. The largest folders I've seen had over a terabyte of data in them, but you probably want the pro version (with selective sync) for those.

fps_doug 5y ago
Syncthing supports introducer devices. If you mark a device as introducer, you'll automatically add all devices the introducer knows to your list. So basically adding a new device to an existing group requires everyone to agree on one introducer, and everyone will add the new device automatically. NAT traversal and relay servers exist too. So seems it's pretty much on par with Resilio, plus open source.
- miki123211 5y ago
  For my specific use case, there's probably no device that could be an introducer. It's all PCs, none of them online 24/7. Besides, setting all that up requires way more technical knowledge than most people have. Sure, I could do it I guess, but explaining Resilio configuration to complete non-techies is hard enough.
  - NilsIRL 5y ago
    Introducer devices don't have to be online all the time. There is also no need to limit the number of introducer devices. I have a folder that's synced between my laptop, desktop and phone, and each has the other 2 devices as introducer.

aasasd 5y ago

On Android, Syncthing-fork is a bit nicer because it has a native app interface instead of the embedded web interface. However, on Android 11 the Play Store release doesn't work with folders other than /Android/media or the app's data folder or something like that—because of Google's continuous crusade against file access, effective for the newer API level that Syncthing-fork is targeting. But the F-Droid version doesn't have this problem. Note that the mainstream Syncthing on Android works so far only because they're targeting an older API version. They will hit the same restriction if they migrate to the new API, unless they implement SAF or whatever.

nyanpasu64 5y ago
Syncthing-Fork can't send broadcast mDNS packets on Android 11. As a result, two Android 11 devices can't even connect to each other over a LAN, and are reduced to sending all traffic to the Internet and back, which wastes bandwidth, probably hurts privacy, and throttles transfer speeds. The workaround I found was to hard-code one device's IP address in the other. Perhaps Syncthing is unaffected; I didn't test. Link: https://forum.syncthing.net/t/local-discovery-problem-on-android-11/16093/19 https://forum.syncthing.net/t/local-discovery-problem-on-and...
- aasasd 5y ago
  Hmmm, I guess it only affects discovery between phones, because I just recently re-set up syncing with two laptops, and it worked fine without entering the IDs or ips or anything—just clicked on the discovered IDs. Of course, this issue still highlights how the ‘open-source’ and ‘customizable’ OS does only what Google allows it to do.

SanchoPanda 5y ago

I would like to give a shout out to `unison` for syncing needs. It can be used with a daemon, but I use it happily with cron and it has been absolutely bulletproof* for years. I have never noticed it's resource usage at all after running the initial sync. I am never modifying the same file on two systems simultaneously, so I can afford to sync hourly with no issues. While not everyone can do this, I suspect many are in the same boat. * Try to use literally the exact same binary where architectures allow, due to build dependencies.

passthejoe 5y ago
I have used both. I'm a BIG Unison fan.

bsdubernerd 5y ago

syncthing is excellent for sharing also between groups. I wish it worked better for occasional file sharing (for example to transfer files between phone/pc). I don't want to keep it running in the background - just sucks too much power, and on the phone I never need to keep other stuff synched. But the time it takes for the first connect to happen after starting it can be in the order if minutes on a known network, dozens of minutes if not. Not usable for this purpose. The android client is also not very well thought out. You cannot "share to syncthing" until it's active, so if you want to put a file for sharing until your network connectivity is up, you can't. You have to start it, share, then shut it down. If you want to do that on an unknown network you didn't whitelist, you have to whitelist the network as well. Kind of pointless. You'd think you can also start it when it prompts you to change the settings, but this also doesn't work. I ended up disabling all these advanced settings, as they clearly end-up being an hindrance. I just start it manually and quit it, kind of defeating the entire purpose of them.

inside_out_life 5y ago
for fast occasional sharing you have KDEConnect
- bsdubernerd 5y ago
  I don't see a command-line or daemon backend. Not really comparable, even if I happened to use kde.
  - inside_out_life 5y ago
    Well, it's not comparable, you wanted different use case there's kdeconnect-cli kdeconnect seems daemon like enough to me since it's entirely detached from its myriad UIs

flanbiscuit 5y ago

I use Syncthing to sync my KeepassXC db file between laptops and my android, always on a local network. For the most part I like it. I get a lot of conflicts though and I can't use ediff (or similar) because the file is encrypted. Curious if anyone has figured out how to fix conflicts with Keypass db files because I don't think Syncthing's conflict resolution can handle them very well.

Disparallel 5y ago
KeepassXC has built in db file merge support! It's hidden somewhere in the menus. I've never personally had to do something more complicated than "I added a new entry on two different devices before syncing" but it's worked well for me so far.
- rcMgD2BwE72F 5y ago
  >It's hidden somewhere in the menus File > Merge with database I saved my database's password as an entry just to quickly merge it with the .sync-conflict version. It takes 2 seconds, and I don't worry about losing anything anymore. Works beautifully for me. KeePass could check for such conflicts (syncthing, dropbox or else) in the directory where the open database is stored and suggest to auto-merge.

jerieljan 5y ago

Syncthing is one of the biggest reasons why I use Android today. That sounds strange, but when you have old spare phones, they're very useful for this. Even for the daily driver phone that I use, it's so good at keeping files in sync between my other PCs.

mkr-hn 5y ago

The only thing that keeps me from using it is the lack of an official iOS client. Once they get that, I'll be all over it.

keithwbacon 5y ago

How fast is this vs rsync over ssh?

llasse 5y ago

Is it possible to access SD card folders on the ohne by now? That used to be the major drawback some years ago for me..