3 ms·

Would a bad driver update potentially have had similar consequences? (real question) If so, this feels like a red herring.

by ern 2y ago

smallnamespace 2y ago
Drivers undergo a vetting process by Microsoft (at least, the ones that are signed and offered as updates) = quality control through their control of the channel. This has been the case at least since the late 90s if I remember correctly.
- nullindividual 2y ago
  WHQL has had some pretty awful drivers pushed through it. Granted, this was quite some time ago, early 2000s-ish, but they've lead to BSODs etc with nVidia drivers.
  - zinekeller 2y ago
    Well, it seems GPU driver fraud were (and is?) rampant: a snippet from Defrauding the WHQL driver certification process (2004, https://devblogs.microsoft.com/oldnewthing/20040305-00/?p=40373 https://devblogs.microsoft.com/oldnewthing/20040305-00/?p=40...) Some unscrupulous drivers will detect that they are being run by WHQL and disable various features so they pass certification. Of course, they also run dog slow in the WHQL lab, but that’s okay, because WHQL is interested in whether the driver contains any bugs, not whether the driver has the fastest triangle fill rate in the industry. The most common cheat I’ve seen is drivers which check for a secret “Enable Dubious Optimizations” switch in the registry or some other place external to the driver itself. They take the driver and put it in an installer which does not turn the switch on and submit it to WHQL. When WHQL runs the driver through all its tests, the driver is running in “safe but slow” mode and passes certification with flying colors. The vendor then takes that driver (now with the WHQL stamp of approval) and puts it inside an installer that enables the secret “Enable Dubious Optimizations” switch. Now the driver sees the switch enabled and performs all sorts of dubious optimizations, none of which were tested by WHQL.
nullindividual 2y ago
Yes, of course. Plenty of storage and chipset drivers have caused no-boot scenarios.
gjsman-1000 2y ago
I’m afraid that one morning, we’ll wake up, Windows is unbootable globally, and it’s because the desktop Widgets had a hardcoded certificate expire, causing an infinite taskbar crash. Which actually happened in one of the Windows 11 preview releases. It might not be Widgets next time - but there’s enough stupid in Windows that it could be anything from SmartScreen to Defender to Copilot to Edge. It could also be just something widespread and stupid, like Intel or NVIDIA’s driver analytics imploding.
- fallingsquirrel 2y ago
  I'm reminded of the time the Windows 11 Insiders taskbar was crashing for everyone because it failed to display an ad. https://www.neowin.net/news/how-an-ad-from-microsoft-broke-the-windows-11-start-menu-and-taskbar/ https://www.neowin.net/news/how-an-ad-from-microsoft-broke-t...
toast0 2y ago
Windows update has a process to rollback bad driver updates after a handful of crashes, and I've seen it work. I think it's about three crashes and it rolls back. But there's probably situations where it won't work, I'm guessing things like: driver A's update causes a crash in driver B; the updated driver crashes, but not immediately; the driver crashes, but only after writing bad data to disk. I also don't know if Microsoft's security software can rollback from bad definition updates.
- ronsor 2y ago
  > driver A's update causes a crash in driver B I can't see why this wouldn't work. A should be in B's stack trace, outside of esoteric interactions. The rollback didn't work for the Crowdstrike problem because despite being a ".sys" file, the file that was updated wasn't a driver.
- throw0101d 2y ago
  > Windows update has a process to rollback bad driver updates after a handful of crashes, and I've seen it work. Does that include "boot-start" drivers? > A boot-start driver is a driver for a device that must be installed to start the Microsoft Windows operating system. * https://learn.microsoft.com/en-us/windows-hardware/drivers/install/installing-a-boot-start-driver https://learn.microsoft.com/en-us/windows-hardware/drivers/i... CS designated their software as one, and when it crashed so did the whole OS to the point that the rollback process doesn't seem to be able to kick-off.
  - toast0 2y ago
    > Does that include "boot-start" drivers? I'm not sure... I don't actually know what driver it was that failed; although I suspect it was a video driver. Computer made some noises, blue screened, restarted, repeat N times, then came up, and after login said something about a bad update and may have told me which device, but I forgot. I was across the room for most of it.

Would a bad driver update potentially have had similar consequences? (real question) If so, this feels like a red herring.

by ern 2y ago

Would a bad driver update potentially have had similar consequences? (real question) If so, this feels like a red herring.

smallnamespace 2y ago

Drivers undergo a vetting process by Microsoft (at least, the ones that are signed and offered as updates) = quality control through their control of the channel. This has been the case at least since the late 90s if I remember correctly.

nullindividual 2y ago
WHQL has had some pretty awful drivers pushed through it. Granted, this was quite some time ago, early 2000s-ish, but they've lead to BSODs etc with nVidia drivers.
- zinekeller 2y ago
  Well, it seems GPU driver fraud were (and is?) rampant: a snippet from Defrauding the WHQL driver certification process (2004, https://devblogs.microsoft.com/oldnewthing/20040305-00/?p=40373 https://devblogs.microsoft.com/oldnewthing/20040305-00/?p=40...) Some unscrupulous drivers will detect that they are being run by WHQL and disable various features so they pass certification. Of course, they also run dog slow in the WHQL lab, but that’s okay, because WHQL is interested in whether the driver contains any bugs, not whether the driver has the fastest triangle fill rate in the industry. The most common cheat I’ve seen is drivers which check for a secret “Enable Dubious Optimizations” switch in the registry or some other place external to the driver itself. They take the driver and put it in an installer which does not turn the switch on and submit it to WHQL. When WHQL runs the driver through all its tests, the driver is running in “safe but slow” mode and passes certification with flying colors. The vendor then takes that driver (now with the WHQL stamp of approval) and puts it inside an installer that enables the secret “Enable Dubious Optimizations” switch. Now the driver sees the switch enabled and performs all sorts of dubious optimizations, none of which were tested by WHQL.

nullindividual 2y ago

Yes, of course. Plenty of storage and chipset drivers have caused no-boot scenarios.

gjsman-1000 2y ago

I’m afraid that one morning, we’ll wake up, Windows is unbootable globally, and it’s because the desktop Widgets had a hardcoded certificate expire, causing an infinite taskbar crash. Which actually happened in one of the Windows 11 preview releases. It might not be Widgets next time - but there’s enough stupid in Windows that it could be anything from SmartScreen to Defender to Copilot to Edge. It could also be just something widespread and stupid, like Intel or NVIDIA’s driver analytics imploding.

fallingsquirrel 2y ago
I'm reminded of the time the Windows 11 Insiders taskbar was crashing for everyone because it failed to display an ad. https://www.neowin.net/news/how-an-ad-from-microsoft-broke-the-windows-11-start-menu-and-taskbar/ https://www.neowin.net/news/how-an-ad-from-microsoft-broke-t...

toast0 2y ago

Windows update has a process to rollback bad driver updates after a handful of crashes, and I've seen it work. I think it's about three crashes and it rolls back. But there's probably situations where it won't work, I'm guessing things like: driver A's update causes a crash in driver B; the updated driver crashes, but not immediately; the driver crashes, but only after writing bad data to disk. I also don't know if Microsoft's security software can rollback from bad definition updates.

ronsor 2y ago
> driver A's update causes a crash in driver B I can't see why this wouldn't work. A should be in B's stack trace, outside of esoteric interactions. The rollback didn't work for the Crowdstrike problem because despite being a ".sys" file, the file that was updated wasn't a driver.
throw0101d 2y ago
> Windows update has a process to rollback bad driver updates after a handful of crashes, and I've seen it work. Does that include "boot-start" drivers? > A boot-start driver is a driver for a device that must be installed to start the Microsoft Windows operating system. * https://learn.microsoft.com/en-us/windows-hardware/drivers/install/installing-a-boot-start-driver https://learn.microsoft.com/en-us/windows-hardware/drivers/i... CS designated their software as one, and when it crashed so did the whole OS to the point that the rollback process doesn't seem to be able to kick-off.
- toast0 2y ago
  > Does that include "boot-start" drivers? I'm not sure... I don't actually know what driver it was that failed; although I suspect it was a video driver. Computer made some noises, blue screened, restarted, repeat N times, then came up, and after login said something about a bad update and may have told me which device, but I forgot. I was across the room for most of it.