11 ms·

Syncthing graduation day

282 points by kim0 8y ago

jacob019 8y ago
Syncthing is amazing, been using ut for years. So flexible, nothing beats it for filesync.
trulyrandom 8y ago
Syncthing is great. I use it to sync my password database and some other stuff across my devices and have not run into a single issue. I set it up 3 years ago and haven't had to touch it since. I had no idea it was technically in "beta" all this time.
- hossbeast 8y ago
  Syncthing + KeePass. Amazing convenience and Time saver.
Heliosmaster 8y ago
Congratulations to the team! I like a lot the announcement: it's very easy to stay perpetually in beta to escape commitment. Their reasons for graduating are sensible and I wish more developers of software on version 0.x since a long time would replicate it.
bowlich 8y ago
Been using Resilio-sync but have been really excited about switching to Syncthing, glad to hear about the 1.0.0.
- roboyoshi 8y ago
  same, the ST-WebUI is just not doing it for me. I paid for Resilio now and I'm quite happy with it. But I also really wish for Syncthing to become a lot more mature and widely adopted to have as an alternative in case resilio discontinues.
onli 8y ago
When I started using syncthing some years ago it worked, but it was still a bit brittle. I ran over time into several situations where the sync would just stop or not start, be slow, some error happened somewhere and the GUI did not show them properly, there was no built in reacting to file modification, just a regular sync interval. It was good enough to be used, but certainly not done yet. Yesterday I installed it again on my main PC (part of a migration from a different linux distro) and I was impressed: The notifications when adding new devices arrived with a very small delay, syncing just worked, file monitoring is now built in, the error message related to that (inotify limit) was helpful. Syncthing came a long way and is great software, congrats for reaching 1.0!
- ioddly 8y ago
  I switched to Syncthing from Dropbox a few months ago. While there are a handful of warts, the actual syncing seems to Just Work (TM). I am massively impressed with this, as building even a centralized file hosting service is quite the undertaking. I donated and I'd encourage others to do it as well, link on the homepage: https://syncthing.net/ https://syncthing.net/
  - avtar 8y ago
    > handful of warts What type of issues?
    - ioddly 8y ago
      What I'm thinking of are synced ignore lists of some sort. I think I read in the issue tracker that this is something they plan to change eventually. And related, sync % reflecting ignored files. For example I've ignored a bunch of transient things like node_modules so my backup server sync is permanently at 70% even though it's actually up to date with all the files I want synced.
      - uvatbc 8y ago
        I do this in two steps: 1. An "stignore.txt" at the root of a shared directory. This gets automatically synced to every device. 2. "#include stignore.txt" in '.stignore' that I need to manually write on every new device.
blattimwind 8y ago
Does it still consume vast amounts of CPU and memory? A few years back I used it on a few hundred GB and it almost constantly used 1-2 CPU cores. On the server it racked up thousands of CPU hours, i.e. it cost real money to run. Apart from that I don't recall any major problems - which is very positive for anything related to file synchronization.
- lscotte 8y ago
  It never has for me, and I've got some large sync'd dirs (~70GB being the largest). Been using it for several years and it's great.
- dantillberg 8y ago
  I've used syncthing for a few years to synchronize 100k+ files and 10k+ folders. I used to regularly get CPU usage spikes from syncthing whenever it scanned folders for changes. The default operating mode of syncthing used to be "do a full scan every N (configurable) seconds for changes," and you needed to run a separate program (syncthing-inotify for me) to use filesystem watches instead. Those full scans would be cheap if you're synchronizing just a few folders, but it's CPU-expensive to regularly check tens of thousands of folders for changes (it might have been stat-ing every file for modtimes, too, I dunno). Over the past year or so, file watching (via inotify/etc) has been built into syncthing itself. I enabled watching on all my computers and syncthing's CPU usage has gone down considerably as a result, while user happiness has increased; I spend less time banging on syncthing's pipes and more time doing stuff with the files it synchronizes.
  - GordonS 8y ago
    I tried it a few years ago on a mix on Linux and Windows systems and quickly abandoned it for Seafile because of the CPU issues. Any idea if it now supports (reliable) real-time file watching on Windows?
    - uvatbc 8y ago
      Windows and Mac support is much better these days. However, the case insensitive file system in Windows makes it annoying to use with *nix systems.
pixelmonkey 8y ago
Is this a good thing to use to, say, keep two git repo checkouts in sync between a laptop and dev server, to avoid the need to do "deploys" for hacking/testing? Or is that overkill?
- otachack 8y ago
  You can certainly do that but I ask why not just use git and set up the same remote? Unless you don't have one.
- stephen 8y ago
  Fwiw mirror might be a good fit for that sort of "super real time while editting source files" use case: https://github.com/stephenh/mirror https://github.com/stephenh/mirror Disclaimer it's my basically-well-functioning hobby project. And obviously wouldn't recommend it for doing actual deployments with it.
Jare 8y ago
I have nothing but applause and kudos to add here, they are really deserved. Syncthing is fantastic.
lousken 8y ago
Last time I used syncthing was couple weeks ago v. 0.14.54 and it's still hit or miss for me in terms of upload speed. Whenever I'm sharing something with friends the upload speed is completly random from 1Mbit to 20Mbit(my max upload speed). I didn't figure out how to debug this issue but I've tried even running my own relay just in case but that didn't solve it either.
dexterdog 8y ago
I've used syncthing to keep my music, books and docs all in sync between my home server and my laptop. It does a great job. I have also used it for a few years now in place of plex sync. I have plex keep certain unwatched shows optimized for mobile and I just sync that directory to my laptop and phone so I always have my unwatched shows with me. Plex sync was totally unreliable for this so I moved away from it and syncthing has never disappointed.
eximius 8y ago
How does this compare with Perkeep (previously Camlistore)? They seem to be very similar, though I haven't made the jump to try either out. What I'd like is integration with on of Perkeep or Syncthing from my freeNAS box to my local devices.
- qaute 8y ago
  IIRC, Syncthing is decentralized P2P and synchronizes standard tree(ish) filesystems (like most similar projects). Perkeep does away with the filesystem and instead uses a blob-based merkle-tree-like system with good properties for very long term archival (and as a side effect, can also use P2P and multiple encrypted cloud backups easily). So Syncthing can copy preexisting file structures and fits in usual workflows, but Perkeep has some data storage benefits. I don't know how their performance compares.
  - eximius 8y ago
    When you say P2P, it's only in a technical sense of not having centralized servers, right? I don't want to interact with nodes I don't own, for my particular use case.
    - hellcow 8y ago
      That's correct. It only syncs between nodes you configure.
    - puetzk 8y ago
      If your nodes have direct network access to each other, no. But if two nodes that have added each other's keys cannot reach each other directly (e.g. are in separate private networks/behind firewalls/etc) but both can reach an external relay server, it may act as a dumb pipe bouncing encrypted traffic between them. https://docs.syncthing.net/users/relaying.html https://docs.syncthing.net/users/relaying.html There is a public pool of such relay servers (https://relays.syncthing.net/ https://relays.syncthing.net/), and by default syncthing will reach out and connect/announce its itself there so other nodes looking for it could contact it via a relay. You can host your own relay server and configure your clients to use it exclusively: https://docs.syncthing.net/users/strelaysrv.html#client-configuration https://docs.syncthing.net/users/strelaysrv.html#client-conf... or if you only want syncing when your nodes are on the same local network you should be able to just not configure any relays at all.
interfixus 8y ago
As others here have noticed, Syncthing has improved by leaps and bounds over the last couple of years. I use it for syncing my phone (everything!) to a pc, and keepin several ps's up to date with each other. These days, it just works. Works so well, in fact, that I have begun experimenting with up-sync to a production server. So far, things are looking real good.
I_am_tiberius 8y ago
Is syncthing already able to do client side encryption (so you can host your files on a server without the server knowing the files you are storing there)? I remember some years back it only used encryption during the file transfer.
- rahiel 8y ago
  It's still an open issue: https://github.com/syncthing/syncthing/issues/109 https://github.com/syncthing/syncthing/issues/109 But you can combine it with an encryption layer yourself by using any of the projects listed here: https://nuetzlich.net/gocryptfs/comparison/ https://nuetzlich.net/gocryptfs/comparison/
- Arkanosis 8y ago
  Not yet, see https://github.com/syncthing/syncthing/issues/109 https://github.com/syncthing/syncthing/issues/109 That's pretty much the only feature I miss with Syncthing.
bayindirh 8y ago
I'm using syncthing to sync three computers. Two slaves connected to one 24/7/365 online master. Unless I'm behind a very aggressive firewall, I can connect to master to sync everything. It creates a big peace-of-mind. Debian's version was built with iNotify for a very long time, so I never got big CPU spikes. I think iNotify is merged into master last year or so.
- old-gregg 8y ago
  How reliable has it been? Do you mind sharing the number of files in the shared directories and their total size? I am asking because when I was searching for a cloudless Dropbox replacement, some tools would lose some files (or truncate them to zero). Then I found Resilio Sync and I've been using it for about 3 years now without any problems. Now I have a feeling that Bittorrent Inc will abandon the project after the acquisition. Syncthing is looking promising.
  - deleted 8y ago
    [deleted]
  - callahad 8y ago
    SyncThing has been completely reliable for the past two years across my Android phone, Linux laptop, and a Linux NAS. I use it to backup photos / videos that I capture on my phone, as well as to sync my KeePass database and Org Mode files which I edit on both my phone and laptop, often concurrently. My largest folder is 46 GB across 9,416 files. The SyncThing Usage Data dashboard at https://data.syncthing.net/ https://data.syncthing.net/ has really interesting statistics: apparently at the 50th percentile, users are syncing more than 5,300 files and over 12.3 GB of data.
    - marksomnian 8y ago
      Wow, either I'm reading the stats wrong, or someone has 84.6 TiB in Syncthing. That's impressive.
      - 8y ago
pbowyer 8y ago
One of the forum posts says: > I really love to use syncthing as a anti-bit-rot check system (i already do^^). Bit-rot is something I want to (manually) protect against, without setting up a NAS running ZFS. How would you use Syncthing?
- nolok 8y ago
  I assume if bit-rot happens then a conflict between versions happen (as if the file was changed manually), so you know a file got changed and can recover the previous one ? I doubt there is any self healing capabilities By the way if you're on linux btrfs has self healing capabilities (and any of the cheap 2/4 bay 64bit synology nas can use the btrfs self healing and coupled with raid protect your data from rot). Not a very big expense for a lot of safety.
  - pbowyer 8y ago
    Windows unfortunately - love it most of the time but when it comes to file systems... Think a NAS will be a future investment.
    - _emacsomancer_ 8y ago
      A NAS of course could be a rather cheap system running Linux/FreeBSD with three drives in a RAID-Z1 configuration, with SAMBA server. Then you're not dependent on what fs Windows supports (though I recall there being some work towards supporting zfs on Windows, so perhaps eventually).
    - nolok 8y ago
      If going Synology, make sure to go for their 64bit models (I don't think they still sell 32 bits, but unsure); it's needed to support btrfs. Other good brand is QNAP. good quality 2 bay can be found below 300€, 4 bays below 500€. As the other commenter said you can also built it yourself but frankly configuring Samba and its permissions and updating your mdadm config when changing a drive and all the other things gets boring and borderline dangerous really quick, just know that if you buy a pre-made NAS what you're paying is not really the hardware (you can get better for cheaper if homemade), it's the software. I like DSM (synology's software) because it doesn't get in your way, it lets anyone use easily yet still offers the advanced options. If you've never had a NAS or don't want to spend hours in config files every now and then, I recommend it. (as usual the question for devices like this is not so much "how easy is it to set up" but "how easy it to recover if say a drive died and you need to repair your array")
fencepost 8y ago
I can certainly see the need for something like this, though I've seen a lot more folks on another forum talking about ownCloud and Nextcloud as alternatives to Dropbox business plans. I really dislike that the Syncthing website appears to have mastered modern "we won't tell you what it is" marketing speak. Did I miss something and this is now a game with scoring for speaking around topics? "Syncthing replaces proprietary sync and cloud services with something open, trustworthy and decentralized. Your data is your data alone and you deserve to choose where it is stored, if it is shared with some third party and how it's transmitted over the Internet." Oooooooh, new shiny with buzzwords! What's it do? Is it a replacement for Chrome's cross-system syncing? Firefox Sync (which I think can already be self-hosted actually)? Password managers? Maybe hosted Exchange? Wait, it's not? Oh, wait, it's a multi-system file syncing package. Well why didn't you say so while you were telling me I deserved to have control and blah blah blah. Cripes, it even says "a large number of features have already been implemented" then provides a list that implies that running on Windows, OSX, Linux, the BSDs and Solaris is one of those features. There are several places where what it does is implied, but to actually get to a statement of "For this guide let’s assume you have two machines between which you want to synchronise files." I have to go to the bottom of the site, click Docs, then either click on Introduction and Getting Started (in the documentation structure) or click on the "getting started guide" link within the first paragraph of the Introduction. grmbl get off my lawn Edit: I should note, I'm not talking about the 1.0 announcement on their forums - that's for folks already actively involved with it. I'm talking about the very spare https://syncthing.net https://syncthing.net
- boterock 8y ago
  Lately, I find myself looking for the github icon before anything else in a website. I find a github readme much more readable than any landing website.
  - ncallaway 8y ago
    Wow, no kidding. The first two sentences of the readme are: > Syncthing is a continuous file synchronization program. It synchronizes files between two or more computers. That copy should absolutely be somewhere on the main website.
bytematic 8y ago
I love this because it's fast and has a way to ignore files which dropbox lacks
wpietri 8y ago
For those, like me, wondering what Syncthing is, Wikipedia describes it as "a free, open-source peer-to-peer file synchronization application available for Windows, Mac, Linux, Android, Solaris, Darwin, and BSD. It can sync files between devices on a local network, or between remote devices over the Internet. Data security and data safety are built into the design of the software."
- brink 8y ago
  Or in other terms - it allows you to host your own dropbox between your own devices using a protocol similar to torrent.
  - anderspitman 8y ago
    Not quite. Dropbox makes it very easy to share (publicly or privately) your data. Seafile or Nextcloud I'd say are more similar to Dropbox
    - brink 8y ago
      Ah yes, minus public sharing. :) My bad.
x1798DE 8y ago
The biggest problem I have with syncthing is that it can't be used with iOS, and there are no plans to ever support iOS. I don't personally use iOS, but my wife does, so I can't get her phone to automatically sync pictures of our family with my phone and computers. I suppose the people involved don't use the platform, and I realize iOS is more annoying to develop for than Android, but it's a pretty major downside and it's tempting me to use a cloud provider instead. I'd much rather just pay $15-$20 or something for an iOS app. Maybe now with the 1.0 release it will be stable enough that a third party can step up and make an iOS app?
- keldaris 8y ago
  If you can find other people who share your desire and present the developers with a clear case for making decent money that way, I'm sure either they or a third party would take up the challenge soon enough. Many of us who have no personal interest in the Apple ecosystem won't support it just because someone asks because of the effort and investment required - being meaningfully paid for it changes the equation.
blue1 8y ago
I (still?) use Sparkleshare. Is it worth switching?
maxyme 8y ago
I've been using syncthing for a few years because I was sick of missing files when going from my laptop to desktop and vice versa. I wanted to sync most folders in my /home directory (documents, photos, videos, code...) While ignoring node_modules folders. It has worked surprisingly fantastically. I thought I would need a server to use as a "master" but syncthing actually supports merging folders automatically. It does not need a static IP address for either computer and it will opportunistically sync my laptop and desktop when they are both on the same network. I had no idea before I tried using it that syncthing could support such a setup so well. High praise.
nicolaslem 8y ago
I didn't know Syncthing and after installing it on a couple of machines I'm very impressed! However, I'm surprised too see that by default the whole interface is available on 127.0.0.1:8384. It is possible to set a password but it's a bit hidden in the settings and the documentation does not insist on it. This makes me wonder, how bad is it for a malicious program running locally to have access to this API?
- zamadatix 8y ago
  It's not great but not particularly horrible either. It takes the interesting combination of "this machine is locked down to the point it does not allow outbound connections by default" and "this machine has a default syncthing install with no password set and an actively running piece of malware that can't create an outbound connection otherwise". Pretty rare someone manages every outbound connection but doesn't configure a password on the tool handling syncing all of their data across the network. Still, would be better if it prompted for a password the first time a user visited the page to set up the client.
- tagrun 8y ago
  So? Local programs also have access to the filesystem, which includes the synced data, config files and procfs. What exactly is your threat model?
  - dddddaviddddd 8y ago
    On a multiuser system it could be an issue. Other users with access to make HTTP requests could add a malicious endpoint. e.g. on a shared server of some sort. I could see this in academia or some business settings. Threat model would be an attacker with a foothold on one machine gaining access to the documents of another user.
    - tagrun 8y ago
      Could be, theoretically, but this would probably be inconveniencing all syncthing users for a negligible fraction of the userbase. On a shared computer where you perceive your peers as a threat, adding a password to syncthing web UI is probably a minor entry in a long list of things you have to do to secure your processes and data, and to ensure the kernel and programs you're running are trustable (in practice, you can't verify that with 100% confidence; they can be backdoored, out-of-date with known security flaws, etc etc). Ensuring security in such a malicious (where your co-workers/peers are apparently a nasty bunch) shared environment is so hard that to begin with that I personally wouldn't put any sensitive data there at all.
krick 8y ago
I'd love to start using Syncthing, but the absence of storage encryption is absolutely a deal breaker. As I understand, it was a planned feature quite a while ago, but now it isn't even in the backlog anymore. A pity, really.
- mkl 8y ago
  It runs on your own devices, which should either be trusted or have their own encryption (it's built into every major OS). Why would additional encryption be useful?
  - zorked 8y ago
    The point is exactly running on untrusted devices - get a el cheapo server at some VPS provider and have it backup all your data + serve it from a always-on decent datacenter connection. I've also been waiting for that one feature :/
    - mkl 8y ago
      I see. So it would intentionally not be a live usable copy. I've been using Duplicity going to Wasabi for that purpose - encryption, versioning, etc. One of my Syncthing nodes is an always-on PC at home, so the files sync with other devices whenever/wherever they have a network connection, which is what your encrypted node would do.
- 2ion 8y ago
  That problem is easily solved at other places in the FS stack. Use an crypto overlay like EncFS and sync only encrypted data. Alternatively, even stream encrypted updates to a directory via a gpg job and sync the encrypted blobs. Another layer, but the script would be so simple it'd essentially write itself. While it would be convenient to have configurable encryption on endpoints, syncthing is composable, and thus this issue can be worked around. If you have a platform that doesn't allow you to compose tools easily like Android, that's limited technology.
- anotherevan 8y ago
blinkingled 8y ago
How's this different from Unison? The documentation on GitHub seems sparse.
- stewbrew 8y ago
  Unison does one-to-one synchronization (local or remote). It puts more emphasis on reconciling sync conflicts. It can handle symlinks. It isn't really maintained, is it. It has problems on windows. It's written in Ocaml which had its heydays back then for a short period of time. Unison rather is a conventional tool which you call and then wait until it's done. Syncthing does many-to-many remote servers synchronization (but you can also do 1:1 or 1:M). It leaves reconciliation of conflicts to the users. If you're lucky, a copy of the file in question is created that has as "sync.conflict" suffix. It ignores symlinks (or copies them as is). It is actively maintained & has a much wider user community. It works on windows ... as well as android etc. It's written in Go. Synchting can also be run in the background to watch folders and keep everything in sync as files change. (This actually is the main use case.) You can use syncthing for almost everything unison does when you want to sync servers, workstations, phones etc. You cannot use syncthing to keep a memory stick in sync. IMHO unison is still better at syncing two directories with frequent merge conflicts. It's a pity it got trapped in the ocaml hole. I personally replaced unison with syncthing for most use cases.
  - blinkingled 8y ago
    Thanks, I misunderstood that Unison had any sort of 1:M sync - seems like that part is a hack using multiple pairs in a star topology. Also good point about OCaml and not having iOS/Android support.
mattbillenstein 8y ago
Been using syncthing for a few years - I had used BitTorrent Sync until my default search provider became Yahoo! and I could no longer trust anything BitTorrent Inc was doing... Syncthing is great - a folder that syncs, I have a VPC in the cloud, so for the data I really care about, it's mirrored offsite more or less in real time in case my house burns down.
emmanueloga_ 8y ago
I've seen Syncthing before but I don't think it is exactly what I need. If I'm not wrong, it is a secure p2p protocol and a suite of apps implementing them to synchronize folders between devices, in the way you would do with Rsync. That means if I have several terabytes that I want to sync between devices, I'd need to have enough space in every device, is that right? (and each device would get a 1:1 copy of the folders). I read that when there are conflicts, the apps create a new "filename".sync-conflict file so you can manually resolve conflicts. What I'd like is a secure way to backup all my files to some cloud provider (S3?) across all my devices, and have an MRU cache in a folder where I can configure the size (for instance, keep at most, say, 512mb of the files I most recently used). For things like photos, it would be better if the sync thing would get thumbnails from the cloud, so I would not need to download the whole thing just to find photos. GDrive / GPhotos is the thing I use now. The combo has a lot of bells and whistles and I'd be fine having a simpler product where I could have more (perceived?) control on how things display, when syncing happens, etc (in lieu of other things like the nice A.I. photo search, integration with GSuite of apps, etc.). Kinda like using a file explorer on my local desktop.
- onli 8y ago
  Do you know https://github.com/s3ql/s3ql https://github.com/s3ql/s3ql? It mounts remote cloud storage like a local filesystem and promises a bunch of optimizations against network latency, including a local cache. Apart from the thumbnail feature that should fit perfectly to what you are searching.
  - emmanueloga_ 8y ago
    Thanks! I did not know about it, I'll take a look.
    - jabberthemutt 8y ago
      Also rclone
      - 8y ago
mikob 8y ago
I switched from Dropbox->Seafile after Dropbox stopped supporting BTRFS. Seafile is minimalistic like syncthing, but also has an iOS app that will do photo syncing which is important for me.

282 points by kim0 8y ago

Syncthing is amazing, been using ut for years. So flexible, nothing beats it for filesync.

Syncthing is great. I use it to sync my password database and some other stuff across my devices and have not run into a single issue. I set it up 3 years ago and haven't had to touch it since. I had no idea it was technically in "beta" all this time.

hossbeast 8y ago
Syncthing + KeePass. Amazing convenience and Time saver.

Heliosmaster 8y ago

Congratulations to the team! I like a lot the announcement: it's very easy to stay perpetually in beta to escape commitment. Their reasons for graduating are sensible and I wish more developers of software on version 0.x since a long time would replicate it.

bowlich 8y ago

Been using Resilio-sync but have been really excited about switching to Syncthing, glad to hear about the 1.0.0.

roboyoshi 8y ago
same, the ST-WebUI is just not doing it for me. I paid for Resilio now and I'm quite happy with it. But I also really wish for Syncthing to become a lot more mature and widely adopted to have as an alternative in case resilio discontinues.

onli 8y ago

When I started using syncthing some years ago it worked, but it was still a bit brittle. I ran over time into several situations where the sync would just stop or not start, be slow, some error happened somewhere and the GUI did not show them properly, there was no built in reacting to file modification, just a regular sync interval. It was good enough to be used, but certainly not done yet. Yesterday I installed it again on my main PC (part of a migration from a different linux distro) and I was impressed: The notifications when adding new devices arrived with a very small delay, syncing just worked, file monitoring is now built in, the error message related to that (inotify limit) was helpful. Syncthing came a long way and is great software, congrats for reaching 1.0!

ioddly 8y ago
I switched to Syncthing from Dropbox a few months ago. While there are a handful of warts, the actual syncing seems to Just Work (TM). I am massively impressed with this, as building even a centralized file hosting service is quite the undertaking. I donated and I'd encourage others to do it as well, link on the homepage: https://syncthing.net/ https://syncthing.net/
- avtar 8y ago
  > handful of warts What type of issues?
  - ioddly 8y ago
    What I'm thinking of are synced ignore lists of some sort. I think I read in the issue tracker that this is something they plan to change eventually. And related, sync % reflecting ignored files. For example I've ignored a bunch of transient things like node_modules so my backup server sync is permanently at 70% even though it's actually up to date with all the files I want synced.
    - uvatbc 8y ago
      I do this in two steps: 1. An "stignore.txt" at the root of a shared directory. This gets automatically synced to every device. 2. "#include stignore.txt" in '.stignore' that I need to manually write on every new device.

blattimwind 8y ago

Does it still consume vast amounts of CPU and memory? A few years back I used it on a few hundred GB and it almost constantly used 1-2 CPU cores. On the server it racked up thousands of CPU hours, i.e. it cost real money to run. Apart from that I don't recall any major problems - which is very positive for anything related to file synchronization.

lscotte 8y ago
It never has for me, and I've got some large sync'd dirs (~70GB being the largest). Been using it for several years and it's great.
dantillberg 8y ago
I've used syncthing for a few years to synchronize 100k+ files and 10k+ folders. I used to regularly get CPU usage spikes from syncthing whenever it scanned folders for changes. The default operating mode of syncthing used to be "do a full scan every N (configurable) seconds for changes," and you needed to run a separate program (syncthing-inotify for me) to use filesystem watches instead. Those full scans would be cheap if you're synchronizing just a few folders, but it's CPU-expensive to regularly check tens of thousands of folders for changes (it might have been stat-ing every file for modtimes, too, I dunno). Over the past year or so, file watching (via inotify/etc) has been built into syncthing itself. I enabled watching on all my computers and syncthing's CPU usage has gone down considerably as a result, while user happiness has increased; I spend less time banging on syncthing's pipes and more time doing stuff with the files it synchronizes.
- GordonS 8y ago
  I tried it a few years ago on a mix on Linux and Windows systems and quickly abandoned it for Seafile because of the CPU issues. Any idea if it now supports (reliable) real-time file watching on Windows?
  - uvatbc 8y ago
    Windows and Mac support is much better these days. However, the case insensitive file system in Windows makes it annoying to use with *nix systems.

pixelmonkey 8y ago

Is this a good thing to use to, say, keep two git repo checkouts in sync between a laptop and dev server, to avoid the need to do "deploys" for hacking/testing? Or is that overkill?

otachack 8y ago
You can certainly do that but I ask why not just use git and set up the same remote? Unless you don't have one.
stephen 8y ago
Fwiw mirror might be a good fit for that sort of "super real time while editting source files" use case: https://github.com/stephenh/mirror https://github.com/stephenh/mirror Disclaimer it's my basically-well-functioning hobby project. And obviously wouldn't recommend it for doing actual deployments with it.

Jare 8y ago

I have nothing but applause and kudos to add here, they are really deserved. Syncthing is fantastic.

lousken 8y ago

Last time I used syncthing was couple weeks ago v. 0.14.54 and it's still hit or miss for me in terms of upload speed. Whenever I'm sharing something with friends the upload speed is completly random from 1Mbit to 20Mbit(my max upload speed). I didn't figure out how to debug this issue but I've tried even running my own relay just in case but that didn't solve it either.

dexterdog 8y ago

I've used syncthing to keep my music, books and docs all in sync between my home server and my laptop. It does a great job. I have also used it for a few years now in place of plex sync. I have plex keep certain unwatched shows optimized for mobile and I just sync that directory to my laptop and phone so I always have my unwatched shows with me. Plex sync was totally unreliable for this so I moved away from it and syncthing has never disappointed.

eximius 8y ago

How does this compare with Perkeep (previously Camlistore)? They seem to be very similar, though I haven't made the jump to try either out. What I'd like is integration with on of Perkeep or Syncthing from my freeNAS box to my local devices.

qaute 8y ago
IIRC, Syncthing is decentralized P2P and synchronizes standard tree(ish) filesystems (like most similar projects). Perkeep does away with the filesystem and instead uses a blob-based merkle-tree-like system with good properties for very long term archival (and as a side effect, can also use P2P and multiple encrypted cloud backups easily). So Syncthing can copy preexisting file structures and fits in usual workflows, but Perkeep has some data storage benefits. I don't know how their performance compares.
- eximius 8y ago
  When you say P2P, it's only in a technical sense of not having centralized servers, right? I don't want to interact with nodes I don't own, for my particular use case.
  - hellcow 8y ago
    That's correct. It only syncs between nodes you configure.
  - puetzk 8y ago
    If your nodes have direct network access to each other, no. But if two nodes that have added each other's keys cannot reach each other directly (e.g. are in separate private networks/behind firewalls/etc) but both can reach an external relay server, it may act as a dumb pipe bouncing encrypted traffic between them. https://docs.syncthing.net/users/relaying.html https://docs.syncthing.net/users/relaying.html There is a public pool of such relay servers (https://relays.syncthing.net/ https://relays.syncthing.net/), and by default syncthing will reach out and connect/announce its itself there so other nodes looking for it could contact it via a relay. You can host your own relay server and configure your clients to use it exclusively: https://docs.syncthing.net/users/strelaysrv.html#client-configuration https://docs.syncthing.net/users/strelaysrv.html#client-conf... or if you only want syncing when your nodes are on the same local network you should be able to just not configure any relays at all.

interfixus 8y ago

As others here have noticed, Syncthing has improved by leaps and bounds over the last couple of years. I use it for syncing my phone (everything!) to a pc, and keepin several ps's up to date with each other. These days, it just works. Works so well, in fact, that I have begun experimenting with up-sync to a production server. So far, things are looking real good.

I_am_tiberius 8y ago

Is syncthing already able to do client side encryption (so you can host your files on a server without the server knowing the files you are storing there)? I remember some years back it only used encryption during the file transfer.

rahiel 8y ago
It's still an open issue: https://github.com/syncthing/syncthing/issues/109 https://github.com/syncthing/syncthing/issues/109 But you can combine it with an encryption layer yourself by using any of the projects listed here: https://nuetzlich.net/gocryptfs/comparison/ https://nuetzlich.net/gocryptfs/comparison/
Arkanosis 8y ago
Not yet, see https://github.com/syncthing/syncthing/issues/109 https://github.com/syncthing/syncthing/issues/109 That's pretty much the only feature I miss with Syncthing.

bayindirh 8y ago

I'm using syncthing to sync three computers. Two slaves connected to one 24/7/365 online master. Unless I'm behind a very aggressive firewall, I can connect to master to sync everything. It creates a big peace-of-mind. Debian's version was built with iNotify for a very long time, so I never got big CPU spikes. I think iNotify is merged into master last year or so.

old-gregg 8y ago
How reliable has it been? Do you mind sharing the number of files in the shared directories and their total size? I am asking because when I was searching for a cloudless Dropbox replacement, some tools would lose some files (or truncate them to zero). Then I found Resilio Sync and I've been using it for about 3 years now without any problems. Now I have a feeling that Bittorrent Inc will abandon the project after the acquisition. Syncthing is looking promising.
- deleted 8y ago
  [deleted]
- callahad 8y ago
  SyncThing has been completely reliable for the past two years across my Android phone, Linux laptop, and a Linux NAS. I use it to backup photos / videos that I capture on my phone, as well as to sync my KeePass database and Org Mode files which I edit on both my phone and laptop, often concurrently. My largest folder is 46 GB across 9,416 files. The SyncThing Usage Data dashboard at https://data.syncthing.net/ https://data.syncthing.net/ has really interesting statistics: apparently at the 50th percentile, users are syncing more than 5,300 files and over 12.3 GB of data.
  - marksomnian 8y ago
    Wow, either I'm reading the stats wrong, or someone has 84.6 TiB in Syncthing. That's impressive.
    - 8y ago

pbowyer 8y ago

One of the forum posts says: > I really love to use syncthing as a anti-bit-rot check system (i already do^^). Bit-rot is something I want to (manually) protect against, without setting up a NAS running ZFS. How would you use Syncthing?

nolok 8y ago
I assume if bit-rot happens then a conflict between versions happen (as if the file was changed manually), so you know a file got changed and can recover the previous one ? I doubt there is any self healing capabilities By the way if you're on linux btrfs has self healing capabilities (and any of the cheap 2/4 bay 64bit synology nas can use the btrfs self healing and coupled with raid protect your data from rot). Not a very big expense for a lot of safety.
- pbowyer 8y ago
  Windows unfortunately - love it most of the time but when it comes to file systems... Think a NAS will be a future investment.
  - _emacsomancer_ 8y ago
    A NAS of course could be a rather cheap system running Linux/FreeBSD with three drives in a RAID-Z1 configuration, with SAMBA server. Then you're not dependent on what fs Windows supports (though I recall there being some work towards supporting zfs on Windows, so perhaps eventually).
  - nolok 8y ago
    If going Synology, make sure to go for their 64bit models (I don't think they still sell 32 bits, but unsure); it's needed to support btrfs. Other good brand is QNAP. good quality 2 bay can be found below 300€, 4 bays below 500€. As the other commenter said you can also built it yourself but frankly configuring Samba and its permissions and updating your mdadm config when changing a drive and all the other things gets boring and borderline dangerous really quick, just know that if you buy a pre-made NAS what you're paying is not really the hardware (you can get better for cheaper if homemade), it's the software. I like DSM (synology's software) because it doesn't get in your way, it lets anyone use easily yet still offers the advanced options. If you've never had a NAS or don't want to spend hours in config files every now and then, I recommend it. (as usual the question for devices like this is not so much "how easy is it to set up" but "how easy it to recover if say a drive died and you need to repair your array")

fencepost 8y ago

I can certainly see the need for something like this, though I've seen a lot more folks on another forum talking about ownCloud and Nextcloud as alternatives to Dropbox business plans. I really dislike that the Syncthing website appears to have mastered modern "we won't tell you what it is" marketing speak. Did I miss something and this is now a game with scoring for speaking around topics? "Syncthing replaces proprietary sync and cloud services with something open, trustworthy and decentralized. Your data is your data alone and you deserve to choose where it is stored, if it is shared with some third party and how it's transmitted over the Internet." Oooooooh, new shiny with buzzwords! What's it do? Is it a replacement for Chrome's cross-system syncing? Firefox Sync (which I think can already be self-hosted actually)? Password managers? Maybe hosted Exchange? Wait, it's not? Oh, wait, it's a multi-system file syncing package. Well why didn't you say so while you were telling me I deserved to have control and blah blah blah. Cripes, it even says "a large number of features have already been implemented" then provides a list that implies that running on Windows, OSX, Linux, the BSDs and Solaris is one of those features. There are several places where what it does is implied, but to actually get to a statement of "For this guide let’s assume you have two machines between which you want to synchronise files." I have to go to the bottom of the site, click Docs, then either click on Introduction and Getting Started (in the documentation structure) or click on the "getting started guide" link within the first paragraph of the Introduction. grmbl get off my lawn Edit: I should note, I'm not talking about the 1.0 announcement on their forums - that's for folks already actively involved with it. I'm talking about the very spare https://syncthing.net https://syncthing.net

boterock 8y ago
Lately, I find myself looking for the github icon before anything else in a website. I find a github readme much more readable than any landing website.
- ncallaway 8y ago
  Wow, no kidding. The first two sentences of the readme are: > Syncthing is a continuous file synchronization program. It synchronizes files between two or more computers. That copy should absolutely be somewhere on the main website.

bytematic 8y ago

I love this because it's fast and has a way to ignore files which dropbox lacks

wpietri 8y ago

For those, like me, wondering what Syncthing is, Wikipedia describes it as "a free, open-source peer-to-peer file synchronization application available for Windows, Mac, Linux, Android, Solaris, Darwin, and BSD. It can sync files between devices on a local network, or between remote devices over the Internet. Data security and data safety are built into the design of the software."

brink 8y ago
Or in other terms - it allows you to host your own dropbox between your own devices using a protocol similar to torrent.
- anderspitman 8y ago
  Not quite. Dropbox makes it very easy to share (publicly or privately) your data. Seafile or Nextcloud I'd say are more similar to Dropbox
  - brink 8y ago
    Ah yes, minus public sharing. :) My bad.

x1798DE 8y ago

The biggest problem I have with syncthing is that it can't be used with iOS, and there are no plans to ever support iOS. I don't personally use iOS, but my wife does, so I can't get her phone to automatically sync pictures of our family with my phone and computers. I suppose the people involved don't use the platform, and I realize iOS is more annoying to develop for than Android, but it's a pretty major downside and it's tempting me to use a cloud provider instead. I'd much rather just pay $15-$20 or something for an iOS app. Maybe now with the 1.0 release it will be stable enough that a third party can step up and make an iOS app?

keldaris 8y ago
If you can find other people who share your desire and present the developers with a clear case for making decent money that way, I'm sure either they or a third party would take up the challenge soon enough. Many of us who have no personal interest in the Apple ecosystem won't support it just because someone asks because of the effort and investment required - being meaningfully paid for it changes the equation.

blue1 8y ago

I (still?) use Sparkleshare. Is it worth switching?

maxyme 8y ago

I've been using syncthing for a few years because I was sick of missing files when going from my laptop to desktop and vice versa. I wanted to sync most folders in my /home directory (documents, photos, videos, code...) While ignoring node_modules folders. It has worked surprisingly fantastically. I thought I would need a server to use as a "master" but syncthing actually supports merging folders automatically. It does not need a static IP address for either computer and it will opportunistically sync my laptop and desktop when they are both on the same network. I had no idea before I tried using it that syncthing could support such a setup so well. High praise.

nicolaslem 8y ago

I didn't know Syncthing and after installing it on a couple of machines I'm very impressed! However, I'm surprised too see that by default the whole interface is available on 127.0.0.1:8384. It is possible to set a password but it's a bit hidden in the settings and the documentation does not insist on it. This makes me wonder, how bad is it for a malicious program running locally to have access to this API?

zamadatix 8y ago
It's not great but not particularly horrible either. It takes the interesting combination of "this machine is locked down to the point it does not allow outbound connections by default" and "this machine has a default syncthing install with no password set and an actively running piece of malware that can't create an outbound connection otherwise". Pretty rare someone manages every outbound connection but doesn't configure a password on the tool handling syncing all of their data across the network. Still, would be better if it prompted for a password the first time a user visited the page to set up the client.
tagrun 8y ago
So? Local programs also have access to the filesystem, which includes the synced data, config files and procfs. What exactly is your threat model?
- dddddaviddddd 8y ago
  On a multiuser system it could be an issue. Other users with access to make HTTP requests could add a malicious endpoint. e.g. on a shared server of some sort. I could see this in academia or some business settings. Threat model would be an attacker with a foothold on one machine gaining access to the documents of another user.
  - tagrun 8y ago
    Could be, theoretically, but this would probably be inconveniencing all syncthing users for a negligible fraction of the userbase. On a shared computer where you perceive your peers as a threat, adding a password to syncthing web UI is probably a minor entry in a long list of things you have to do to secure your processes and data, and to ensure the kernel and programs you're running are trustable (in practice, you can't verify that with 100% confidence; they can be backdoored, out-of-date with known security flaws, etc etc). Ensuring security in such a malicious (where your co-workers/peers are apparently a nasty bunch) shared environment is so hard that to begin with that I personally wouldn't put any sensitive data there at all.

krick 8y ago

I'd love to start using Syncthing, but the absence of storage encryption is absolutely a deal breaker. As I understand, it was a planned feature quite a while ago, but now it isn't even in the backlog anymore. A pity, really.

mkl 8y ago
It runs on your own devices, which should either be trusted or have their own encryption (it's built into every major OS). Why would additional encryption be useful?
- zorked 8y ago
  The point is exactly running on untrusted devices - get a el cheapo server at some VPS provider and have it backup all your data + serve it from a always-on decent datacenter connection. I've also been waiting for that one feature :/
  - mkl 8y ago
    I see. So it would intentionally not be a live usable copy. I've been using Duplicity going to Wasabi for that purpose - encryption, versioning, etc. One of my Syncthing nodes is an always-on PC at home, so the files sync with other devices whenever/wherever they have a network connection, which is what your encrypted node would do.
2ion 8y ago
That problem is easily solved at other places in the FS stack. Use an crypto overlay like EncFS and sync only encrypted data. Alternatively, even stream encrypted updates to a directory via a gpg job and sync the encrypted blobs. Another layer, but the script would be so simple it'd essentially write itself. While it would be convenient to have configurable encryption on endpoints, syncthing is composable, and thus this issue can be worked around. If you have a platform that doesn't allow you to compose tools easily like Android, that's limited technology.
anotherevan 8y ago

blinkingled 8y ago

How's this different from Unison? The documentation on GitHub seems sparse.

stewbrew 8y ago
Unison does one-to-one synchronization (local or remote). It puts more emphasis on reconciling sync conflicts. It can handle symlinks. It isn't really maintained, is it. It has problems on windows. It's written in Ocaml which had its heydays back then for a short period of time. Unison rather is a conventional tool which you call and then wait until it's done. Syncthing does many-to-many remote servers synchronization (but you can also do 1:1 or 1:M). It leaves reconciliation of conflicts to the users. If you're lucky, a copy of the file in question is created that has as "sync.conflict" suffix. It ignores symlinks (or copies them as is). It is actively maintained & has a much wider user community. It works on windows ... as well as android etc. It's written in Go. Synchting can also be run in the background to watch folders and keep everything in sync as files change. (This actually is the main use case.) You can use syncthing for almost everything unison does when you want to sync servers, workstations, phones etc. You cannot use syncthing to keep a memory stick in sync. IMHO unison is still better at syncing two directories with frequent merge conflicts. It's a pity it got trapped in the ocaml hole. I personally replaced unison with syncthing for most use cases.
- blinkingled 8y ago
  Thanks, I misunderstood that Unison had any sort of 1:M sync - seems like that part is a hack using multiple pairs in a star topology. Also good point about OCaml and not having iOS/Android support.

mattbillenstein 8y ago

Been using syncthing for a few years - I had used BitTorrent Sync until my default search provider became Yahoo! and I could no longer trust anything BitTorrent Inc was doing... Syncthing is great - a folder that syncs, I have a VPC in the cloud, so for the data I really care about, it's mirrored offsite more or less in real time in case my house burns down.

emmanueloga_ 8y ago

I've seen Syncthing before but I don't think it is exactly what I need. If I'm not wrong, it is a secure p2p protocol and a suite of apps implementing them to synchronize folders between devices, in the way you would do with Rsync. That means if I have several terabytes that I want to sync between devices, I'd need to have enough space in every device, is that right? (and each device would get a 1:1 copy of the folders). I read that when there are conflicts, the apps create a new "filename".sync-conflict file so you can manually resolve conflicts. What I'd like is a secure way to backup all my files to some cloud provider (S3?) across all my devices, and have an MRU cache in a folder where I can configure the size (for instance, keep at most, say, 512mb of the files I most recently used). For things like photos, it would be better if the sync thing would get thumbnails from the cloud, so I would not need to download the whole thing just to find photos. GDrive / GPhotos is the thing I use now. The combo has a lot of bells and whistles and I'd be fine having a simpler product where I could have more (perceived?) control on how things display, when syncing happens, etc (in lieu of other things like the nice A.I. photo search, integration with GSuite of apps, etc.). Kinda like using a file explorer on my local desktop.

onli 8y ago
Do you know https://github.com/s3ql/s3ql https://github.com/s3ql/s3ql? It mounts remote cloud storage like a local filesystem and promises a bunch of optimizations against network latency, including a local cache. Apart from the thumbnail feature that should fit perfectly to what you are searching.
- emmanueloga_ 8y ago
  Thanks! I did not know about it, I'll take a look.
  - jabberthemutt 8y ago
    Also rclone
    - 8y ago

mikob 8y ago

I switched from Dropbox->Seafile after Dropbox stopped supporting BTRFS. Seafile is minimalistic like syncthing, but also has an iOS app that will do photo syncing which is important for me.