3 ms·

I've been using this extensively recently. I was setting up remote virtual machines that boot a live ISO containing all the software for the machine. Sometimes

I've been using this extensively recently. I was setting up remote virtual machines that boot a live ISO containing all the software for the machine. Sometimes I need to change a small config file, which would lead to generating a new 1.7GiB ISO, but 99.9% of that ISO is identical to the previous one. So I used rsync. Blew my mind when after a day of working on these images, uploading 1.7GiB ISO after 1.7GiB ISO, wireguard showed that I had only sent 600MiBs. Fun surprise, rsync uses file size and modified time first to see if the files are identical. I build these ISOs with nix. Nix sets the time to Jan 1st 1970 for reproducible builds, and I suspect the ISOs are padded out to the next sector. So rsync was not noticing the new ISO images when I made small changes to config files until I added the --checksum flag.

seb1204 10mo ago
In the past I downloaded daily diffs from iso which were only few MB. I then applied this diff to my iso from yesterday. Forgot the name of this tool though. I did this on my machine, if parent wants to update in a remote machine I'm not sure it works the same way.
alright2565 10mo ago
If it was a 100GB image on the other hand—good luck! It'd be faster to copy it from scratch every time than to use rsync.
axiolite 10mo ago
You'll find something like BorgBackup will be far more efficient than rsync.
- SoftTalker 10mo ago
  But rsync is widely available, usually installed by default on linux or unix-like systems. You can just use it.
  - axiolite 10mo ago
    Borg is available for download as a standalone binary, easily dropped onto any Linux system even with very limited privs. And in the repos of every distro easily installed and kept up-to-date. By avoiding that one step and using rsync instead, you're resigning yourself to "send 600MiBs" over the network for every tiny config change. Not a good trade-off.
yencabulator 10mo ago
> Fun surprise, rsync uses file size and modified time first to see if the files are identical. [...] time to Jan 1st 1970 for reproducible builds I think it'd be a good idea for rsync to not trust timestamp 0.

seb1204 10mo ago

In the past I downloaded daily diffs from iso which were only few MB. I then applied this diff to my iso from yesterday. Forgot the name of this tool though. I did this on my machine, if parent wants to update in a remote machine I'm not sure it works the same way.

alright2565 10mo ago

If it was a 100GB image on the other hand—good luck! It'd be faster to copy it from scratch every time than to use rsync.

axiolite 10mo ago

You'll find something like BorgBackup will be far more efficient than rsync.

SoftTalker 10mo ago
But rsync is widely available, usually installed by default on linux or unix-like systems. You can just use it.
- axiolite 10mo ago
  Borg is available for download as a standalone binary, easily dropped onto any Linux system even with very limited privs. And in the repos of every distro easily installed and kept up-to-date. By avoiding that one step and using rsync instead, you're resigning yourself to "send 600MiBs" over the network for every tiny config change. Not a good trade-off.

yencabulator 10mo ago

> Fun surprise, rsync uses file size and modified time first to see if the files are identical. [...] time to Jan 1st 1970 for reproducible builds I think it'd be a good idea for rsync to not trust timestamp 0.