4 ms·

In theory, its simpler to guarantee everything is exactly as it should be, so the 'baseline' is identical. Differential based systems can fail in multiple ways

by count 6y ago

In theory, its simpler to guarantee everything is exactly as it should be, so the 'baseline' is identical. Differential based systems can fail in multiple ways (e.g. do you know when it's all done? what if one file is missed? etc.) that are simpler than just a bulk replace. I'm sure they COULD be differential, but that's a lot of extra complexity.

hn_throwaway_99 6y ago
This still doesn't make sense to me. I mean, I don't understand the complexity of this at all: 1. Machine keeps around a baseline version of the OS install 2. A diff is downloaded and applied to the baseline version - that entire new version is then checksummed and validated to ensure it would be identical to what an entirely new version would look like. 3. That new version is then installed. I could see a downside here of requiring more disk space, but even then doesn't seem like it would be that hard for the system to make some heuristic decisions (e.g. incremental vs. full) based on the amount of free space on disk. Incremental updates are really not that hard, we've successfully implemented them for decades. Apple engineers are obviously no dummies, so would just like to understand what additional considerations make incremental updates more difficult now.
- sneak 6y ago
  > that entire new version is then checksummed and validated to ensure it would be identical to what an entirely new version would look like. Some of the complaints with the current system are that it is slow and needs a ton of free disk space, which are precisely the issues with the system you describe here. Apple engineers aren't dumb but not all of their staff have their hands in every bit of code on every project. I do know the time pressure there to make deadlines is insanely high. The new hardware announcement/ship date is set in stone weeks or months in advance and the new OS must go out the door on those new machines that day.
- StillBored 6y ago
  AFS supports snapshots, so like windows/etc you just take a volume snapshot, apply a differential update then reboot. The COW nature would make the entire thing even fairly trivial IO wise. The article notes they are using merkle trees, so its not really necessarily to validate the entire image. Only the parts that change, which should be fairly trivial, and likely is done as each part of the image is run (because you have to do it that way anyway, otherwise changes would sneak by if they were made while the OS was running). If something bad happens, you "just" revert to the last known good snapshot. The way this all gets messed up, is if there is some per build versioning information that ends up being propagated into every single part of the system like Linux does with its module loader on clean builds. Then effectively you are forced to update the entire OS just because the version number rolls, some signing key gets changed, or whatever. The differential download in that case would compress really well, but it would take an eternity to apply, and the entire image would probably end up getting duplicated following the volume snapshot. So I would expect some kind of "bug" like that, the kernel guys/whatever have their own version validation system, and the installer guys have some clever upgrade code, but put them together and neither works right.
- count 6y ago
  As you use and configure and modify the system, it starts to drift into a functionally random state. An example: directories of plugins/etc. where the update won't have any content, but content exists in the current install. Upgrading/overwriting the app wouldn't 'catch' these directories of stuff, without having/keeping track of every place in the entire fs that functions like that. A checksum would let you know that the directory is different, but not 'how', per se. By just nuking the whole fs from orbit and unrolling a new one, you have a 'known' state without any of that tracking internally (which is probably impossible to get right organizationally).

by count 6y ago

hn_throwaway_99 6y ago

This still doesn't make sense to me. I mean, I don't understand the complexity of this at all: 1. Machine keeps around a baseline version of the OS install 2. A diff is downloaded and applied to the baseline version - that entire new version is then checksummed and validated to ensure it would be identical to what an entirely new version would look like. 3. That new version is then installed. I could see a downside here of requiring more disk space, but even then doesn't seem like it would be that hard for the system to make some heuristic decisions (e.g. incremental vs. full) based on the amount of free space on disk. Incremental updates are really not that hard, we've successfully implemented them for decades. Apple engineers are obviously no dummies, so would just like to understand what additional considerations make incremental updates more difficult now.

sneak 6y ago
> that entire new version is then checksummed and validated to ensure it would be identical to what an entirely new version would look like. Some of the complaints with the current system are that it is slow and needs a ton of free disk space, which are precisely the issues with the system you describe here. Apple engineers aren't dumb but not all of their staff have their hands in every bit of code on every project. I do know the time pressure there to make deadlines is insanely high. The new hardware announcement/ship date is set in stone weeks or months in advance and the new OS must go out the door on those new machines that day.
StillBored 6y ago
AFS supports snapshots, so like windows/etc you just take a volume snapshot, apply a differential update then reboot. The COW nature would make the entire thing even fairly trivial IO wise. The article notes they are using merkle trees, so its not really necessarily to validate the entire image. Only the parts that change, which should be fairly trivial, and likely is done as each part of the image is run (because you have to do it that way anyway, otherwise changes would sneak by if they were made while the OS was running). If something bad happens, you "just" revert to the last known good snapshot. The way this all gets messed up, is if there is some per build versioning information that ends up being propagated into every single part of the system like Linux does with its module loader on clean builds. Then effectively you are forced to update the entire OS just because the version number rolls, some signing key gets changed, or whatever. The differential download in that case would compress really well, but it would take an eternity to apply, and the entire image would probably end up getting duplicated following the volume snapshot. So I would expect some kind of "bug" like that, the kernel guys/whatever have their own version validation system, and the installer guys have some clever upgrade code, but put them together and neither works right.
count 6y ago
As you use and configure and modify the system, it starts to drift into a functionally random state. An example: directories of plugins/etc. where the update won't have any content, but content exists in the current install. Upgrading/overwriting the app wouldn't 'catch' these directories of stuff, without having/keeping track of every place in the entire fs that functions like that. A checksum would let you know that the directory is different, but not 'how', per se. By just nuking the whole fs from orbit and unrolling a new one, you have a 'known' state without any of that tracking internally (which is probably impossible to get right organizationally).