4 ms·

This is a nice little project, but I think rsync is better suited for any serious use. It can do ssh, it's own protocol or any connection program that would tun

by beagle3 11y ago

This is a nice little project, but I think rsync is better suited for any serious use. It can do ssh, it's own protocol or any connection program that would tunnel bidirectionally. It does very efficient transfer, can create off-line patch files, and basically do any one-way file/dir sync function you can think of. Edit: this one is continuous and rsync is one-shot. Personally, I've been using inotify+rsync+timeout (intofy+rsync has a race condition, so limiting the inotify wait and syncing anyway is essential). Does sshync avoid the race?

roeme 11y ago
If you're considering replacing your current (assumedly homegrown) solution; I'd recommend using lsyncd instead.
- ultramancool 11y ago
  You might also check out Syncany if you're interested in client-side encryption or Syncthing if you're looking for something more Bittorrent Sync like.
mynegation 11y ago
I use similar solution: my own shell script with inotify + timeout + unison over ssh. Unison is much better suited for two-way synchronization than rsync.
- foxcub 11y ago
  I love unison, but how do you use it with inotify?
  - skrause 11y ago
    Newer versions of Unison (since around 2012) can watch directories with inotify: http://blog.nguyenvq.com/blog/2011/07/18/real-time-file-synchronization-like-dropbox-via-unison/ http://blog.nguyenvq.com/blog/2011/07/18/real-time-file-sync...
- larrys 11y ago
  For http://www.cis.upenn.edu/~bcpierce/unison/ http://www.cis.upenn.edu/~bcpierce/unison/ your thoughts on why that is better than using rsync ? Edit: I am now noticing this tool isn't even maintained anymore.
  - vbernat 11y ago
    A new version has been released this year. unison works remarkably well, it's understandable there is not much to add.
  - mynegation 11y ago
    From https://alliance.seas.upenn.edu/~bcpierce/wiki/index.php?n=Main.UnisonFAQGeneral https://alliance.seas.upenn.edu/~bcpierce/wiki/index.php?n=M... "Rsync is a mirroring tool; Unison is a synchronizer. That is, rsync needs to be told "this replica contains the true versions of all the files; please make the other replica look exactly the same." Unison is capable of recognizing updates in both replicas and deciding which way they should be propagated."
  - rsync 11y ago
    Larry - would you please email us at info@rsync.net ? We have a few items to discuss with you ... (your profile mentions to respond to one of your comments if someone needs to reach you ...)
    - larrys 11y ago
      Thanks - I will but not just yet (a few reasons for that actually).
- beagle3 11y ago
  Don't know if this is still the case, but in the past, unison was extremely picky about the versions at both ends; I stopped using it when I upgraded my Ubuntu box and as a result I needed to either recompile an older version for the new box or upgrade the version on all other boxes (across windows and various distros). I now use rsync for bidirectional mirroring (any two versions from the last 15 years or so can talk to each other), git for synchronization (getting history for the price of needing to add/commit). Can't beat the network efficiency and compatibility of these two.
larrys 11y ago
I use rsync. Question for you (or any expert here). Is there any possible way that rsync can alter the source file (unintentionally I mean, by accident or bug)? And what happens to an rsync file that is modified while in the middle of a transfer? [1](I haven't been able to find satisfactory answers to these questions hence the question here). [1] Say you have a 1,000,000 line file that you are rsynch-ing and you delete line 50000 and insert at line 750000?
- beagle3 11y ago
  Rsync opens source file in read only mode - so it has the same probability to modify a file that regular copying does. That said, the --remove-source-files option will delete source files after copying; --delete may remove destination files; and used together they drop the idempotence guarantee that rsync gives. So you have to be careful. Re changing file - IIRC it will keep Resending until file size+mtime at end of transfer is same as in the beginning.
drinchev 11y ago
Yes it combines also amazingly well with inotifywait[1], perhaps there is a port for other OS-es too. Here is a tip on how it can be used ( not tested ) #!/bin/bash WATCH_PATH="/some/path" DEST_PATH="/some/path" REMOTE="root@example.com" while read LINE; do FILE=$(echo ${LINE} | cut -d' ' -f2 ) /usr/bin/rsync -r ${FILE} ${REMOTE}:${DEST_PATH} done < <(/usr/bin/inotifywait -mq -e CLOSE_WRITE --format "%w %f" ${WATCH_PATH}) 1 : http://linux.die.net/man/1/inotifywait http://linux.die.net/man/1/inotifywait
- beagle3 11y ago
  For most uses, rsync -a (preserve all possible metadata, like cp -a) is preferable to rsync -r ; Also, unless you have a very large (breadth or depth) tree, you can just let rsync synchronize the entire path - it is quite efficient, will pick up additional diretories (you need inotifywait -r for that, and AFAIK you still have an unavoidable race).

by beagle3 11y ago

roeme 11y ago

If you're considering replacing your current (assumedly homegrown) solution; I'd recommend using lsyncd instead.

ultramancool 11y ago
You might also check out Syncany if you're interested in client-side encryption or Syncthing if you're looking for something more Bittorrent Sync like.

mynegation 11y ago

I use similar solution: my own shell script with inotify + timeout + unison over ssh. Unison is much better suited for two-way synchronization than rsync.

foxcub 11y ago
I love unison, but how do you use it with inotify?
- skrause 11y ago
  Newer versions of Unison (since around 2012) can watch directories with inotify: http://blog.nguyenvq.com/blog/2011/07/18/real-time-file-synchronization-like-dropbox-via-unison/ http://blog.nguyenvq.com/blog/2011/07/18/real-time-file-sync...
larrys 11y ago
For http://www.cis.upenn.edu/~bcpierce/unison/ http://www.cis.upenn.edu/~bcpierce/unison/ your thoughts on why that is better than using rsync ? Edit: I am now noticing this tool isn't even maintained anymore.
- vbernat 11y ago
  A new version has been released this year. unison works remarkably well, it's understandable there is not much to add.
- mynegation 11y ago
  From https://alliance.seas.upenn.edu/~bcpierce/wiki/index.php?n=Main.UnisonFAQGeneral https://alliance.seas.upenn.edu/~bcpierce/wiki/index.php?n=M... "Rsync is a mirroring tool; Unison is a synchronizer. That is, rsync needs to be told "this replica contains the true versions of all the files; please make the other replica look exactly the same." Unison is capable of recognizing updates in both replicas and deciding which way they should be propagated."
- rsync 11y ago
  Larry - would you please email us at info@rsync.net ? We have a few items to discuss with you ... (your profile mentions to respond to one of your comments if someone needs to reach you ...)
  - larrys 11y ago
    Thanks - I will but not just yet (a few reasons for that actually).
beagle3 11y ago
Don't know if this is still the case, but in the past, unison was extremely picky about the versions at both ends; I stopped using it when I upgraded my Ubuntu box and as a result I needed to either recompile an older version for the new box or upgrade the version on all other boxes (across windows and various distros). I now use rsync for bidirectional mirroring (any two versions from the last 15 years or so can talk to each other), git for synchronization (getting history for the price of needing to add/commit). Can't beat the network efficiency and compatibility of these two.

larrys 11y ago

I use rsync. Question for you (or any expert here). Is there any possible way that rsync can alter the source file (unintentionally I mean, by accident or bug)? And what happens to an rsync file that is modified while in the middle of a transfer? [1](I haven't been able to find satisfactory answers to these questions hence the question here). [1] Say you have a 1,000,000 line file that you are rsynch-ing and you delete line 50000 and insert at line 750000?

beagle3 11y ago
Rsync opens source file in read only mode - so it has the same probability to modify a file that regular copying does. That said, the --remove-source-files option will delete source files after copying; --delete may remove destination files; and used together they drop the idempotence guarantee that rsync gives. So you have to be careful. Re changing file - IIRC it will keep Resending until file size+mtime at end of transfer is same as in the beginning.

drinchev 11y ago

Yes it combines also amazingly well with inotifywait[1], perhaps there is a port for other OS-es too. Here is a tip on how it can be used ( not tested ) #!/bin/bash WATCH_PATH="/some/path" DEST_PATH="/some/path" REMOTE="root@example.com" while read LINE; do FILE=$(echo ${LINE} | cut -d' ' -f2 ) /usr/bin/rsync -r ${FILE} ${REMOTE}:${DEST_PATH} done < <(/usr/bin/inotifywait -mq -e CLOSE_WRITE --format "%w %f" ${WATCH_PATH}) 1 : http://linux.die.net/man/1/inotifywait http://linux.die.net/man/1/inotifywait

beagle3 11y ago
For most uses, rsync -a (preserve all possible metadata, like cp -a) is preferable to rsync -r ; Also, unless you have a very large (breadth or depth) tree, you can just let rsync synchronize the entire path - it is quite efficient, will pick up additional diretories (you need inotifywait -r for that, and AFAIK you still have an unavoidable race).