4 ms·

Unison File Synchronizer

80 points by smartmic 3y ago

lizknope 3y ago
I've been using Unison for over 15 years. I have my main desktop and 3 laptops. It has a simple GUI that shows which direction the file update will happen and lets me decide any conflicts if the same file has been modified in both places.
- sspiff 3y ago
  Same. I tried a bunch of alternatives over the years. Dropbox, Syncthing,... Unison is easy and consumes 0 resources in the background. It won't drain your battery scanning your folders for changes because there's more files and folders than you can get notified for by your OS. `unison -auto` on the command line is all you need to get a convenient list of changed files and whether you want to sync them. I have all my systems set up to sync with a low power mini PC that runs some other services. Hasn't ever failed and deals with lots of small files and big files and everything in between without hickups or surprises.
stavros 3y ago
I, too, thought that Unison Cloud was somehow a backend for this.
- bradhe 3y ago
  Came here to say this!
  - geerlingguy 3y ago
    Heh, when both are in the top 5 spots on HN, it's funny that they're completely unrelated.
- Tade0 3y ago
  I thought it was something related to the Unison language: https://www.unison-lang.org/ https://www.unison-lang.org/
hughesjj 3y ago
Unison was huge while I was at Amazon, but I always preferred rsync for the control (sshfs too but you had to ensure to set some flags for when you de-docked or changed wifi ap). One of the biggest gripes I had with it is thankfully fixed though (used to need same client and server version, nowadays they permit some drift assuming no breaking contractual changes). I'd also recommend setting up some ignore rules for build artifacts, etc, which definitely increases complexity but avoids constantly syncing build artifacts you don't care about or with erroneous symlinks etc. IDK at that point I figured rsync was easier, as it's a more commonly used tool for other purposes, and I'd rather learn fewer things with huge functional overlap if given the choice.
- vamega 3y ago
  Did you rsync in a loop? Or on demand? I used Unison at Amazon, and it was the most seamless of all the options I tried. The two way sync was especially useful for being able to open up test results in a browser locally
  - frfl 3y ago
    You can write a script that uses the OS's native file watching system and triggers rsync. That's an approach I used and eventually everyone else on my team, not Amazon, also ended up using. Wasn't very difficult to setup, on Linux at least.
  - hughesjj 3y ago
    On demand, which for me was a big reason to use rsync over unison. I preferred the control. That said I was definitely in the minority, but eh
aidenn0 3y ago
I haven't used Unison in a while, so maybe this isn't true anymore, but what always ended being the hangup for me was that both sides needed to use exactly the same version of Unison (IIRC at least down to the minor version number). This ends up being a hassle unless you are using exactly the same OS on both sides, since you need to either have the right version in ports/apt/pacman/whatever or manage the Unison builds yourself.
- perryh2 3y ago
  I notice the same issue with rsync between different systems.
- Filligree 3y ago
  You might like Restic as an alternative. It's a bit higher performance, and there's no server component needed.
  - shoo 3y ago
    i've used both restic and unison, they're both good tools but they're focused on different jobs, they're not really comparable. restic is specialised for backups and offers uni-directional backup and uni-directional restore from backup. unison implements bi-directional sync. if you don't want bi-directional sync and don't want specialised backup support, rsync will get the job done. if you want bi-directional sync (unifying changes made on both sides), that's something that neither rsync nor restic can handle. tangentially, re: using restic for offsite backups, one storage provider that pairs quite nicely with restic is rsync.net -- they have a lower-rate pricing offer specifically for use with restic (by default rsync.net filesystems have ZFS snapshots available, they disable ZFS snapshots for these "restic accounts" -- the restic application is then responsible for managing backup snapshots, not the filesystem). https://www.rsync.net/products/restic.html https://www.rsync.net/products/restic.html
- jxy 3y ago
  It's been relaxed a bit. I have a few end points and currently they are on two different versions. > Unison 2.53.3 (ocaml 4.12.1): Contacting server... > Unison 2.51.5 (ocaml 4.14.0): Contacting server...
bdcravens 3y ago
Performance has improved to the point where I no longer use it, but I know Unison was a critical part of docker-sync (at one point, file performance for Docker on Mac was absolutely terrible without it)
ch0ic3 3y ago
Sounds similar to https://syncthing.net/ https://syncthing.net/ ?
- barbazoo 3y ago
  > It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host) Syncthing has no practical limit in terms of number of replicas as far as I am aware. It does not support replicas on the same host though I believe.
pgen 3y ago
Warning, from the manual: Unison does not understand hard links.
Borg3 3y ago
Unison was one of the top tools to check for me. Unfortunately, its written in weird egghead language called Ocaml. It also support replicas, while I needed more p2p and VFS like aproach. So I wrote my simple DVFS repo manager in C instead. Very happy with the outcome. Maybe one day I will be brave enough to share it with the world ;) Here is command summary: http://borg.uu3.net/~borg/?dot http://borg.uu3.net/~borg/?dot
Jeff_Brown 3y ago
I'd like something like this for a three-copy backup solution. Regularly compare the three backups, and wherever one differs from the other two, correct the one with the minority opinion. It seems like this must already exist but I don't know how to find it.
exe34 3y ago
Used this over ten years ago, for a few months. Absolutely loved it, but it was a pain to get the right version installed on different devices, and I think I once lost some data with it due to my own mistake. Nowadays I use syncthing, which works quietly in the background and is really good at what I need. New pictures on my phone get synced to the laptop and backup server, while new pdfs on the laptop get synced with the phone and pinenote.
jesuslop 3y ago
For what I tried is like rclone bisync but without cloud awareness. Useful in a LAN context and with all maturity one can ask for. Rare it hasn't got more good rap.
shoo 3y ago
unison is wonderful, lesser known tool. I used unison regularly for a few years around a decade ago to do bidirectional sync between a linux VM filesystem and a host windows filesystem, for a weird dev workflow where i would pig-headedly use linux tools to develop an application that would be built and shipped to windows. I never had any trouble with unison, it was great. That said, for that particular niche use case these days maybe I'd use WSL to get a similar job done -- or just do all development in Windows if that's the platform we were shipping to.

80 points by smartmic 3y ago

I've been using Unison for over 15 years. I have my main desktop and 3 laptops. It has a simple GUI that shows which direction the file update will happen and lets me decide any conflicts if the same file has been modified in both places.

sspiff 3y ago
Same. I tried a bunch of alternatives over the years. Dropbox, Syncthing,... Unison is easy and consumes 0 resources in the background. It won't drain your battery scanning your folders for changes because there's more files and folders than you can get notified for by your OS. `unison -auto` on the command line is all you need to get a convenient list of changed files and whether you want to sync them. I have all my systems set up to sync with a low power mini PC that runs some other services. Hasn't ever failed and deals with lots of small files and big files and everything in between without hickups or surprises.

stavros 3y ago

I, too, thought that Unison Cloud was somehow a backend for this.

bradhe 3y ago
Came here to say this!
- geerlingguy 3y ago
  Heh, when both are in the top 5 spots on HN, it's funny that they're completely unrelated.
Tade0 3y ago
I thought it was something related to the Unison language: https://www.unison-lang.org/ https://www.unison-lang.org/

hughesjj 3y ago

Unison was huge while I was at Amazon, but I always preferred rsync for the control (sshfs too but you had to ensure to set some flags for when you de-docked or changed wifi ap). One of the biggest gripes I had with it is thankfully fixed though (used to need same client and server version, nowadays they permit some drift assuming no breaking contractual changes). I'd also recommend setting up some ignore rules for build artifacts, etc, which definitely increases complexity but avoids constantly syncing build artifacts you don't care about or with erroneous symlinks etc. IDK at that point I figured rsync was easier, as it's a more commonly used tool for other purposes, and I'd rather learn fewer things with huge functional overlap if given the choice.

vamega 3y ago
Did you rsync in a loop? Or on demand? I used Unison at Amazon, and it was the most seamless of all the options I tried. The two way sync was especially useful for being able to open up test results in a browser locally
- frfl 3y ago
  You can write a script that uses the OS's native file watching system and triggers rsync. That's an approach I used and eventually everyone else on my team, not Amazon, also ended up using. Wasn't very difficult to setup, on Linux at least.
- hughesjj 3y ago
  On demand, which for me was a big reason to use rsync over unison. I preferred the control. That said I was definitely in the minority, but eh

aidenn0 3y ago

I haven't used Unison in a while, so maybe this isn't true anymore, but what always ended being the hangup for me was that both sides needed to use exactly the same version of Unison (IIRC at least down to the minor version number). This ends up being a hassle unless you are using exactly the same OS on both sides, since you need to either have the right version in ports/apt/pacman/whatever or manage the Unison builds yourself.

perryh2 3y ago
I notice the same issue with rsync between different systems.
Filligree 3y ago
You might like Restic as an alternative. It's a bit higher performance, and there's no server component needed.
- shoo 3y ago
  i've used both restic and unison, they're both good tools but they're focused on different jobs, they're not really comparable. restic is specialised for backups and offers uni-directional backup and uni-directional restore from backup. unison implements bi-directional sync. if you don't want bi-directional sync and don't want specialised backup support, rsync will get the job done. if you want bi-directional sync (unifying changes made on both sides), that's something that neither rsync nor restic can handle. tangentially, re: using restic for offsite backups, one storage provider that pairs quite nicely with restic is rsync.net -- they have a lower-rate pricing offer specifically for use with restic (by default rsync.net filesystems have ZFS snapshots available, they disable ZFS snapshots for these "restic accounts" -- the restic application is then responsible for managing backup snapshots, not the filesystem). https://www.rsync.net/products/restic.html https://www.rsync.net/products/restic.html
jxy 3y ago
It's been relaxed a bit. I have a few end points and currently they are on two different versions. > Unison 2.53.3 (ocaml 4.12.1): Contacting server... > Unison 2.51.5 (ocaml 4.14.0): Contacting server...

bdcravens 3y ago

Performance has improved to the point where I no longer use it, but I know Unison was a critical part of docker-sync (at one point, file performance for Docker on Mac was absolutely terrible without it)

ch0ic3 3y ago

Sounds similar to https://syncthing.net/ https://syncthing.net/ ?

barbazoo 3y ago
> It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host) Syncthing has no practical limit in terms of number of replicas as far as I am aware. It does not support replicas on the same host though I believe.

pgen 3y ago

Warning, from the manual: Unison does not understand hard links.

Borg3 3y ago

Unison was one of the top tools to check for me. Unfortunately, its written in weird egghead language called Ocaml. It also support replicas, while I needed more p2p and VFS like aproach. So I wrote my simple DVFS repo manager in C instead. Very happy with the outcome. Maybe one day I will be brave enough to share it with the world ;) Here is command summary: http://borg.uu3.net/~borg/?dot http://borg.uu3.net/~borg/?dot

Jeff_Brown 3y ago

I'd like something like this for a three-copy backup solution. Regularly compare the three backups, and wherever one differs from the other two, correct the one with the minority opinion. It seems like this must already exist but I don't know how to find it.

exe34 3y ago

Used this over ten years ago, for a few months. Absolutely loved it, but it was a pain to get the right version installed on different devices, and I think I once lost some data with it due to my own mistake. Nowadays I use syncthing, which works quietly in the background and is really good at what I need. New pictures on my phone get synced to the laptop and backup server, while new pdfs on the laptop get synced with the phone and pinenote.

jesuslop 3y ago

For what I tried is like rclone bisync but without cloud awareness. Useful in a LAN context and with all maturity one can ask for. Rare it hasn't got more good rap.

shoo 3y ago

unison is wonderful, lesser known tool. I used unison regularly for a few years around a decade ago to do bidirectional sync between a linux VM filesystem and a host windows filesystem, for a weird dev workflow where i would pig-headedly use linux tools to develop an application that would be built and shipped to windows. I never had any trouble with unison, it was great. That said, for that particular niche use case these days maybe I'd use WSL to get a similar job done -- or just do all development in Windows if that's the platform we were shipping to.