3 ms·
> When A has a new release 4, you obviously want to build a diff against release 3, but then you also most likely need or want to build a diff against 2 and may
by dspillett 16y ago
> When A has a new release 4, you obviously want to build a diff against release 3, but then you also most likely need or want to build a diff against 2 and maybe even 1 to take care of people who haven't already upgraded to 3.
You wouldn't need a diff for every combination (1->2, 1->3, 1->4, 2->3, 2->4, 3->4 in the four version case), just the three diffs 1->2, 2->3 and 3->4. Then if someone has v2 you send out the diffs for 3 and 4 and have the client apply both in order to make the updated package ready to apply. The saving in space and number of diffs stored will grow as the number of versions grows.
This will be a little less efficient in terms of bandwidth use on average when people are skipping a couple of versions, but will make little or no difference if people are upgrading in a timely manner (so are only moving in single version steps most of the time) and will save space over storing diffs between all versions.
As well as the diffs I would store checksums for each version and have the client send the checksum for the version they have just-in-case, to avoid sending a diff (sending the file package instead) if the reference file seems corrupt.
Also if you store diffs for both directions you can serve old versions of packages (in case people need to roll-back due to some unexpected incompatibility, or they are developers needing to build a test environment with older library versions) without storing every version completely. This increases the diffs per package, but not nearly as much as storing one diff between every version (for 11 versions, 1 diff per change is 10, diffs in both directions is 20, diffs between all versions totals 45 (or 90 for both directions), for 21 versions those numbers are 20, 40, 190, 380).
- Groxx 16y agoIn my experience, diff + diff + diff = super-long update process. It was done for many game updates up to several years ago, and an update from the box version through 3 patches could take up to an hour. The install, meanwhile, would take 20 minutes or so at most. edit: not that I don't think this can be improved, nor that the updating software they used was any good. Just sayin'.
- GrandMasterBirt 16y agoThat directly depends on the install process for a small update. The diff minimizes traffic, the installer can just do a single install using 100 diffs. Each piece in the domino stack must play well together.
- idonthack 16y agoi expect those game updates didn't use bindiffs, they had to repackage the entirety of each updated file. when you consider that many developers do things like compress all game content files into a few large archives, it's easy to see how that adds up. you end up re-downloading the same content repeatedly when they repeatedly make small changes to a few scattered files in one large archive.
- dspillett 16y agoOften with game updates to get around the fact that patching the compressed multi-asset files directly would not be efficient (as a change early in the file would mean the rest of the file to the end would need patching unless they had the foresight to use something like gzip's rsync-friendly option), the patch would unpack the compressed resource files, patch them, then recompress. Depending on how granularly the assets are distributed amongst the installed files and how many of them were being touched by the patch this could be a lot slower than just reading the compressed file from CD to hard-drive which is what the installer would do. Of course this means that if using the multi-diff method of update distribution you would need to be careful about your selection of compression arrangement to avoid the same inefficiency (unless the saving in bandwidth for the client and the package storage servers is far more important than a bit of extra time spent on the updates client-side)..
- lallysingh 15y agoThat makes a good case for some of the existing mirrors to provide 'diffserv'-like facilities. A small daemon could do that server-side, and the client may just need a hash of the prior package's binary file. Then it downloads the diff and the new file hash :-)