All of this has already happened, and all of this will happen again. - The original rsync author wrote rproxy (https://rproxy.samba.org/ https://rproxy.samba.o

All of this has already happened, and all of this will happen again. - The original rsync author wrote rproxy (https://rproxy.samba.org/ https://rproxy.samba.org/), an ingenious system wrapping http inside the rsync protocol. On subsequent visits, the proxies see the resemblance and sends a diff. I've toyed with it myself (https://github.com/rakoo/rproxy/ https://github.com/rakoo/rproxy/) but I don't think website really care about how disastrously huge and cumbersome they are to use - another idea is to invert rsync: instead of the client calculating a signature, sending it to the server, and the server calculating the delta for every request of every client, it is possible to have the server calculate the signature once every time actually changes, send the signature to the client, let the client calculate the parts they need and download only them. In short, zsync (http://zsync.moria.org.uk/ http://zsync.moria.org.uk/) There are a million things already done in this sector, but for some reason it has never caugh up... I think there's just not enough interest