4 ms·

I am quite confused; doesn't rsync already use content-defined chunk boundaries, with a condition on the rolling hash to define boundaries? https://en.wikipedi

I am quite confused; doesn't rsync already use content-defined chunk boundaries, with a condition on the rolling hash to define boundaries? https://en.wikipedia.org/wiki/Rolling_hash#Content-based_slicing_using_a_rolling_hash https://en.wikipedia.org/wiki/Rolling_hash#Content-based_sli... https://en.wikipedia.org/wiki/Rolling_hash#Content-based_slicing_using_moving_sum https://en.wikipedia.org/wiki/Rolling_hash#Content-based_sli... The speed improvements over rsync seem related to a more efficient rolling hash algorithm, and possibly by using native windows executables instead of cygwin (windows file systems are notoriously slow, maybe that plays a role here). Or am I missing something? In any case, the performance boost is interesting. Glad the source was opened, and I hope it finds its way into rsync.

sneak 1y ago
rsync seems frozen in time; it’s been around for ages and there are so many basic and small quality of life improvements that could have been made that haven’t been. I have always assumed it’s like vim now: only really maintained in theory, not in practice.
- Zardoz84 1y ago
  So you not used vim or neovim in the last 10 years ?
  - lftl 1y ago
    To be fair, there was a roughly 6 year period when vim saw one very minor release. That slow development period was the impetus for the fork of Neovim.
    - Zardoz84 1y ago
      I know. I use Neovim. But since that, and thanks to Neovim, Vim has speedup and got some improvements.
      - dotancohen 1y ago
        Time for neorsync. That said, VIM 8 was terrific.
- chasil 1y ago
  Please bear in mind that there are [now] two distinct rsync codebases. The original is the GPL variant [today displaying "Upgrade required"]: https://rsync.samba.org/ https://rsync.samba.org/ The second is the BSD clone: https://www.openrsync.org/ https://www.openrsync.org/ The BSD version would be used on platforms that are intolerant of later versions of the GPL (Apple, Android, etc.).
deleted 1y ago
[deleted]
re 1y ago
> doesn't rsync already use content-defined chunk boundaries, with a condition on the rolling hash to define boundaries? No, it operates on fixed size blocks over the destination file. However, by using a rolling hash, it can detect those blocks at any offset within the source file to avoid re-transferring them. https://rsync.samba.org/tech_report/node2.html https://rsync.samba.org/tech_report/node2.html
ohitsdom 1y ago
The readme very nicely contrasts the approach with rsync.

sneak 1y ago

rsync seems frozen in time; it’s been around for ages and there are so many basic and small quality of life improvements that could have been made that haven’t been. I have always assumed it’s like vim now: only really maintained in theory, not in practice.

Zardoz84 1y ago
So you not used vim or neovim in the last 10 years ?
- lftl 1y ago
  To be fair, there was a roughly 6 year period when vim saw one very minor release. That slow development period was the impetus for the fork of Neovim.
  - Zardoz84 1y ago
    I know. I use Neovim. But since that, and thanks to Neovim, Vim has speedup and got some improvements.
    - dotancohen 1y ago
      Time for neorsync. That said, VIM 8 was terrific.
chasil 1y ago
Please bear in mind that there are [now] two distinct rsync codebases. The original is the GPL variant [today displaying "Upgrade required"]: https://rsync.samba.org/ https://rsync.samba.org/ The second is the BSD clone: https://www.openrsync.org/ https://www.openrsync.org/ The BSD version would be used on platforms that are intolerant of later versions of the GPL (Apple, Android, etc.).

deleted 1y ago

[deleted]

re 1y ago

> doesn't rsync already use content-defined chunk boundaries, with a condition on the rolling hash to define boundaries? No, it operates on fixed size blocks over the destination file. However, by using a rolling hash, it can detect those blocks at any offset within the source file to avoid re-transferring them. https://rsync.samba.org/tech_report/node2.html https://rsync.samba.org/tech_report/node2.html

ohitsdom 1y ago

The readme very nicely contrasts the approach with rsync.