15 ms·
How Microsoft reduced Windows 11 update size by 40%
- ksec 5y agoFor people with fairly decent internet connection, more often that not the compute and files copying is taking way more time than downloading. While it is important to reduce file size for the benefits of people with slower connection. I am wondering if there are ways we could trade with larger download size but much faster update overall.
- encryptluks2 5y agoToo bad they didn't make it easy to block OEM spyware from Windows Update. I bought an HP at Costco a while back that was on sale... reinstalled Windows using a clean install disc image, installed the latest drivers... next thing I know Windows update starts rolling back my drivers and installed HP spyware. I had to set a group policy option to prevent drivers from Windows update. I remember at one point doing a fresh install was the way to prevent OEM spyware, but now it seems like Microsoft is happy to include it and make it difficult to prevent.
- brnt 5y agoDid you wipe the drive? If not, there is probably a hidden OEM partition that Windows uses to install all that OEM stuff back after fresh installs.
- xunn0026 5y agoThis reads like dealing with a hardware compromised machine. At some point, why even bother?
- encryptluks2 5y agoFrom what I gather, Microsoft gets OEM info from UEFI and lets OEM manufacturers use that to provide software/drivers via Windows Update. In this instance, it kept rolling back the integrated Radeon driver directly from AMD to one that the OEM provided that was over a year old.
- KronisLV 5y ago> At some point, why even bother? Because a lot of people: - don't care about their privacy and spyware (a non-insignificant chunk of the populace, even i'm growing weary after decades of surveillance attempts) - don't know any better (perhaps the majority of the consumerbase) - only have certain software available on Windows (e.g. MobaXTerm, or have something like Sony Vegas fit their workflow and don't have the desire to change) - don't know how to use the alternative OSes well, or have bad experiences with them (Linux driver installation, anyone?) - feel like that's a fair tradeoff to make for being able to run their favourite entertainment software or video games (Proton will probably eventually be good enough, but yet doesn't have 100% coverage, which currently isn't enough to sway everyone) Personally, i just run a dual boot setup with Windows 10 (since it seems marginally less horrible than 11 for the time being) for playing legacy games and using the software that i like, and Linux Mint (or Ubuntu, or Debian, depending on the mood/year) for development stuff. Honestly, Linux is probably the better platform for many other things, everything from privacy to development (how it works with Docker and other *nix software is especially nice, no need for weird mapping, WSL/WSL2, MinGW or any other weirdness on Windows), but for a variety of reasons the adoption remains low. That said, i actually recently wrote about some of my frustrations as a part of my blog article, "On technological illiteracy", which ended up talking a bit more about the driver issues and hardware support: https://blog.kronis.dev/articles/on-technological-illiteracy https://blog.kronis.dev/articles/on-technological-illiteracy Let's just hope that things continue improving in the coming decades!
- brnt 5y agoThats how I treat any new device: compromised, by the manufacturer.
- encryptluks2 5y agoYes I wiped the drive. I didn't use any OEM partition to reinstall. I used the latest Windows ISO, including October updates that comes directly from Microsoft. This is a known issue that is well documented. You can find the HP stuff in the Windows Update catalog, as well as other OEM providers.
- brnt 5y ago> I didn't use any OEM partition to reinstall. If you wipe, its OK, but if you dont, the partition may be used by Windows, even if you install fresh from an image you prepared yourself.
- toast0 5y agoACPI tables from the system ROM can give Windows a list of urls to fetch and install. This is useful when the OEM isn't being a jerk. They can be sure you've got the drivers for the touchpad or whatever, regardless of how you install... But they don't always use their powers for good.
- xpressvideoz 5y agoBut what you call "spyware" can be beneficial to other people. I for one welcomed when several OEM apps were automatically installed through Windows Update, because my laptop has some features accessible only by those OEM apps. That said, I recognize that it might be better to give users a choice. But then again, isn't a group policy exactly designed to tackle this problem? Good defaults for normal people, and customizability for power users.
- encryptluks2 5y agoGroup policy wouldn't tackle the problem for everyone, as GP is only available in Pro editions and what not. You have to use registry hacks for Home. And no, they should have an option during install. Even if you enable it after install, it is usually too late since it'll start installing drivers during install and after... you'd have to stay disconnected from the Internet to prevent it.
- kyriakos 5y agoRecently purchased a used HP workstation and did a clean windows install. I didn't just get all the HP stuff installed automatically but it even changed my desktop wallpaper to an HP one which was then synced to any pc that uses my Microsoft account. Wallpaper may not be important but not something you'd expect.
- lnyng 5y ago> Not regress install time. They seems to achieve size reduction by transmitting only the forward upgrade patches, and let the machine generate the downgrade patches during the upgrade. How do they manage to keep install time the same while this definitely uses more resources (cpu, io)?
- arcticbull 5y agoMaybe they generate the downgrade patches as the upgrade patches arrive - but before they're applied? During that period one would expect the network to be the bottleneck in all but the most extreme circumstances. [edit] And it should be possible to do incrementally.
- rconti 5y agoI wonder if perhaps "install time" contains "download time", and by taking some average internet connection speed, they're estimating that they save more in download time than they cost in actual install+downgrade patch time? This would mean, if you have a fast link, the experience is somewhat worse, but if you have a slow link, the experience is better. Just a total guess though.
- reed1234 5y agoMy guess is to install an update windows has to apply each little patch to random files scattered across the OS necessitating a file read and write for each little change. But generating a reverse delta should only take a transform of the update file which I imagine is much less complex.
- forgotpwd16 5y ago>How do they manage to keep install time the same Do they? Or Windows updates are going to get even slower?
- wolpoli 5y agoSummary: Instead of downloading both the forward delta and the backward delta as part of the update, the system computes the backward delta and stores it on disk.
- reed1234 5y agoSeems like a very simple change.
- deelly 5y agoOh my....
- capableweb 5y agoMost changes seem simple when you think about just the change itself, but then at the scale of Microsoft (I haven't dealt with that personally), things get very complicated, very quickly. Consider all the different setups people have, what can go wrong and so on, and suddenly something that seems simple could actually have a huge impact on a large amount of people/setups.
- xoac 5y agoAre you a product manager?
- acqq 5y agoYes, that's really all to it. Compute the delta from the current revision to the base and keep that delta on the user's computer. It seems that because they haven't done that until now (keeping the delta to the base locally) they distributed these same otherwise locally computable deltas to every Windows computer in the world with every update! If I understood it correctly they patented that "approach" and wrote this much (or more) to make it appear more clever and worth of a patent? "1] The approach described above was filed on 03/12/2021 as U.S. Provisional Patent Application No. 63/160,284 “REVERSE UPDATE DATA GENERATION”"
- personjerry 5y agoInteresting point on culture that most companies have tech blog titles in the form of "How we did X" but Microsoft's is "How Microsoft did X"
- verst 5y agoIt got posted to a community forum website also heavily frequented by Microsoft users and partner companies. That forum may have posts about customers or partner companies, so it makes sense to refer to Microsoft proper in this way. Note that this isn't the main Microsoft blog, and in fact there are lots of different blogs.
- oh_sigh 5y agoMicrosoft refers to itself in the 3rd person?
- peakaboo 5y agoThere is no we in Microsoft.
- wstone 5y ago…but there is in Wendows?
- dotancohen 5y agoThere's an "oso" in Microsoft. Is that a reference to Ballmer? Of course, Linux mailing lists used to joke that "micro" and "soft" reference Bill, back when people had thick skin and cancel culture had yet to be invented.
- CRConrad 5y ago> There's an "oso" in Microsoft. Is that a reference to Ballmer? Does he have a beard nowadays? He didn't use to, back in the day -- and aren't they almost mandatory for "bears"?
- 5y ago
- xxpor 5y agoCan anyone explain why the windows update process is so freakin slow? It's just sits there scanning for ages, and then installs are really slow as well. Downloads seem fine. Is it not just asking for updates since x?
- ToFab123 5y agoDo you have a spinning rust disk?
- GrayShade 5y agoIt's slow even on my PCIe 4 NVMe drive. I boot into Windows once a week or two and I often see it spending more than 5 minutes installing a single update package. This is even slower than apt and dpkg, props to them.
- ToFab123 5y agoYour installation of Windows is that by any chance the installation that came with computer? Or is it a Windows that you have upgraded from a previous version (and perhaps that was upgraded from yet another previous version)? I never have these slow downs that I read others have. I always do a clean install from a USB stick (including deleting recovery partitions). I do this like once a year or 1 1/2 years when something is being released that is either a new version or what is we used to call a service packs. Windows sucks at upgrading. There is always something strange going on after upgrading. Things that not happens after a clean install. If it is looong time since you have performed a clean install, I recommend you to consider doing that.
- GrayShade 5y agoI installed last year when I got the computer and drive, and it's still pretty clean since I don't really use Windows these days. Or was clean until I upgraded to 11 last week. But this has always been my experience. I remember reinstalling Windows 7 and waiting hours for Windows Update to finish. Even checking for updates could take 20 minutes.
- moontear 5y ago* by deleting the Xbox game bar Just kidding - impressive how much you can safe in terms of size.
- cakoose 5y agoThe first idea -- generating the reverse delta while applying the forward delta -- seem obvious. The second idea is clever, but Chrome did it in 2011: https://news.ycombinator.com/item?id=2576878 https://news.ycombinator.com/item?id=2576878 Even if the ideas aren't novel, getting this thing built and rolled out for Windows is not at all trivial, so good job there. But calling it "new compression technology" is misleading.
- willvarfar 5y agoI desperately wish Linux distros used this trick too. I am tired of waiting for updates to download and having to clean out some cache folder that apt uses to free space.
- arianvanp 5y agoI think a lot of this work is being doing in projects like rpm-ostree
- m4rtink 5y agoYep ostree is doing delta updates by default. Flatpaks are alo stored as an ostree, so they do as well - even across all runtimes an flatpaks installed on a system! :)
- paulfurtado 5y agoFWIW, RPM based distros can use delta RPMs
- Saris 5y agoThat's more an apt thing, dnf can use delta rpms with parallel downloads and automatically cleans unused packages. I'm not sure how well it cleans the cache though. On my fast home connection dnf is way faster than apt to update my laptop unless it's a really small update, then apt is faster because it refreshes the repos quicker. I think there are alternatives to apt that can do parallel downloads and delta as well.
- 5y ago
- sfgweilr4f 5y agoSometimes I feel like Microsoft should just put Windows into an image all by itself. One big file (VHD or similar). Then each application gets its own separate image file. After all that, have some kind of funky image mounting thing that merges them together in some filesystem view. Returning us to the good old C drive we know. User data lives separately and politely away from the OS and applications. There might even be a RW image for the OS so thats not even in the OS image itself. Advantages? Just download the image. Could even delta from old to new in multiple ways. Easy revert as well. Security would then come from the usual deal around privileges but there's the possibility of exotic new approaches. Automatic revert could be a thing as well. OS images could actually have a version. Kind of like a ROM. Disadvantages? Disk space. Filesystems have come a long way though. Plenty of tricks around that. But I'm sure there are plenty of other downsides I'm sure. Don't corrupt that file. You can already boot VHD(x?) files stored on, eg C, drive so this isn't actually impossible. EDIT: These images wouldn't be unpacked on each boot. There would be files inside the image and the image is mounted as a filesystem. This is old tech now in the 2020s and its definitely not exotic. If images really are too much overhead then change the word "image" to "partition". Or some combination thereof (eg image for each app). But in reality, I'm not really convinced the overhead is really that great. Disk encryption uses significant processing power already so accessing a filesystem from an image isn't that great a leap.
- noja 5y agohttps://fedoramagazine.org/what-is-silverblue/ https://fedoramagazine.org/what-is-silverblue/
- stiray 5y agovhd is too expensive for something like this and you dont need to virtualize the whole disk, file system is enough. They could achieve something like this but on boot level: https://sandboxie-plus.com/ https://sandboxie-plus.com/
- mike_d 5y agoThis is kinda what you get from Windows on Qubes OS.
- 5y ago
- jollybean 5y agoI was looking forward to something about how they re-thought architecture and bloat, and had the self awareness to contemplate why Windows is so much bigger than before while at the same time not doing that much more.
- hulitu 5y agoThe only thought architecture in windows is telemetry. Everything else is patched.
- bryanrasmussen 5y agoI knew it actually wasn't going to be 'removed all spyware and advertising" but still disappointed nonetheless.
- oblio 5y agoLet me translate that for you in a way that makes more sense for Microsoft: > removed major revenue sources
- bryanrasmussen 5y agoin the case of Microsoft we've proved that even if you pay for the product you're still not the customer.
- jazzyjackson 5y agoit still puzzles me, who’s buying? how does microsoft turn my user statistics into cash?
- MikusR 5y agoSome years ago Microsoft was spreading FUD about Linux. Now Linux users do the same about Microsoft.
- oblio 5y agoThat comment came from a Windows 10 machine, but ok :-))
- oblio 5y agoI don't think they monetize telemetry directly, it's more valuable for them if they control access to it tightly. But the original post was complaining about ads, too :-)
- underscore_ku 5y agoi reduced the updates size by 100% using Linux, hehe
- blippage 5y agoI upvoted you, because you know what: you're right. I shake my head at all the tomfoolery that Windows users are expected to go through. Seriously, Microsoft blather on about compelling experience this, rich interface that, but the whole thing seems a nightmare to me. What's more, it's been "normalised". People go through all this garbage and they rave about how good Windows is.
- tester34 5y agoAny concrete complaints?
- npteljes 5y agoI, for one, hate how disregarded the system makes me feel. The experience is much closer to a remotely managed computer, like the workstation I have at my day job, than to using a personal computer. This applies to how the updates are handled, the telemetry stuff in general, the way the Control Panel and other settings are arranged, and so on. It's hard to pinpoint one specific thing, because the whole system is steered in this direction.
- INTPenis 5y agoWonder how Delta RPMs do this, I am a Fedora user. This must be how delta rpms work right? Because why would they contain backwards data? They only contain new data, and the diff between old and new data.
- LudwigNagasena 5y agoThis seems… obvious? I am rather surprised they didn’t do that before. Am I missing something?
- m0zg 5y agoIt's not their size that was the problem. It's the frequency, and the fact that they take a long time to install and most of them seem to require a reboot. And sometimes they even reboot the machine when you don't want, potentially losing work.
- habibur 5y agoFirst read about this address map diff instead of binary diff compression in Chrome updates. And how they reduced binary update size by 90%. Guess Chrome updates still use this compression.
- BatteryMountain 5y agoSo.. not compression, just gitting updates? Am I missing something?
- Gargyle 5y agoWhich is still way more than most other os updaters need to download. Also its incredibly slow. Updating a bunch of computers with i9, 64gb ram, nvme and gigabit uplink from an older iso install to recent still took half a day and 4 restarts on average.
- jatone 5y agojesus my updates take < 30 seconds on linux though I update frequently. but even on my systems where I don't upgrade for over a years they only take about 10-15minutes.
- coding_lobster 5y agoInitially I read the headline as "How Minecraft reduced Windows 11 update size by 40%" and I was so intrigued. Oh well.
- j3th9n 5y agoXcode could learn a lot from this.
- chenzhekl 5y agoThe biggest problem for me is not the download size but the long installation time. Hope Microsoft can solve that point as well.
- DharmaPolice 5y agoMaybe it just shows their overwhelming focus on the cloud but I don't see how anyone at Microsoft can have looked at their OS update system (particularly for Windows Server) and think "Yup, that's acceptable." Our Windows Server 2016 VM's take over two hours to apply the monthly security patches even on a basic web or application server. The Sysadmin subreddit had a thread recently where some were claiming some of their 2016 machines take over four hours. Not only are our Linux servers dramatically faster to update there's also more information displayed on progress for those inclined to look at console outputs. This leads to a lot less awkward pauses where you try to work out if it's permanently stuck or just thinking.
- tatersolid 5y agoAre you patching all your machines at the same time, overloading your SAN? Are you running webroot or similar horrifying security software? We generally see <5 minute downtimes on all our Server 2016 and 2019 VMs, never more than 10 minutes.
- cyberpsybin 5y agoHow about they reduce the bloatware and spyware that is Windows11 by some percent. And how about not taking control of my computer to brute force updates?
- zaptheimpaler 5y agoOk cool now make it not bug out on simple things like night light being stuck to on even after I disable it. Best solution from the web was - "here try deleting these registry keys" (didn't work), "try this other weird unrelated thing hidden somewhere" (didn't work) and finally, from the community forums "reinstall windows I guess lmao" like what the fuck??? I was trying to disable this one setting and now I need to spend hours debugging this or reinstalling the entire OS? But hey solving cool engineering problems is more fun than fixing bugs I guess.. Windows, not even once.
- marcodiego 5y agoPosted a few days ago: https://news.ycombinator.com/item?id=28920081 https://news.ycombinator.com/item?id=28920081
- SigmundA 5y agoWish Apple would do something for macOS every little security update is over 2 gigs...
- ChrisRR 5y agoNow can they reduce the installation time? It always surprises me at how I can update everything on linux in a matter of seconds, but Windows takes exponentially longer
- naomeux 5y agoThis is a great method overall, they definitely optimized the total size needed for the update.
- theknocker 5y agoCool, it could have been 100%