5 ms·

> they need guidance on basic QA practices Microsoft has a loooong history of botched (security) updates, so I'm not hopeful they can teach Crowdstrike much.

by holsta 2y ago

SoftTalker 2y ago
Yes, quite the epitome of throwing stones from a glass house.
Rinzler89 2y ago
Do you happen to have a list of that "loooong history" of botched (security) updates? I can only find a couple of examples after googling, which a bit smaller than a "loooong history" you're talking about, so unless Microsoft is paying Google to delete results, maybe you're mistaken.
- SoftTalker 2y ago
  This is a company whose OS could not even be installed on a live network without getting rooted within a few minutes. Anybody who was paying attention knew that you didn't use any new Windows release until at least the first service pack had come out. Granted that was a while back but painful memories die hard.
  - Rinzler89 2y ago
    >This is a company whose OS could not even be installed on a live network without getting rooted within a few minutes. That was WIndows XP 20 years ago. Please bring arguments about modern Window 11 security which is the current up to date product they're selling and supporting not scenarios that haven't happened in 20 years.
    - clwg 2y ago
      First thing that comes to mind is that Recall stuff from a month ago, they also release updates[0] that crash machines. [0] https://www.tomsguide.com/news/windows-11-update-causing-blue-screen-of-death-how-to-fix-it https://www.tomsguide.com/news/windows-11-update-causing-blu...
      - TeMPOraL 2y ago
        Recall actually is a brilliant idea, and I dreamed of something like it for a long time, and so did plenty people here. It's just not something you can trust a third-party business with, whether it's a fly-by-night startup or an international megacorporation known to be openly promiscuous with advertisers. This is basically "take a screenshot every 30 seconds and compile it into a timelapse", but on steroids, and the same appeal, and arguments wrt. who gets to run it on whose machines, all apply.
        dahdum 2y ago
        If you keep your business and personal computing separate, Recall looks amazing.
        clwg 2y ago
        The functionality does seem intriguing, that doesn't change it's security profile which was poorly thought out and implemented.
      - feyman_r 2y ago
        Ignoring Windows Insider reports is bad. However, how many endpoints having issues (out of a billion+) is ‘acceptable’ after an update? We live in a news hype cycle so clearly even the one wrong failure will make it up somewhere. However, without metrics that show BSoDs from patches (which MS will likely never share), it’s hard to see if things have improved or regressed. If they regressed, someone up in their leadership chain is hopefully following the constructive discussion here.
    - Eduard 2y ago
      for a loooong history, you have to look in the past
      - Rinzler89 2y ago
        Ah, well, if only things of the past were useful today, I'd still have hair, and probably millions made form right investments, but unfortunately, it's what's happening today that actually matters.
        echoangle 2y ago
        So you asked for proof of a long history and are now surprised that the examples are all from the past?
        Rinzler89 2y ago
        How does that impact the present? If it's no longer as vulnerable today, why would I care about the past? The point is learning from mistakes and fixing them so that doesn't happen again.
        echoangle 2y ago
        If it doesn’t matter to you, why did you ask? Are you just trying to win an argument or are you being intellectually honest? Because you asked for proof of the long history someone claimed. You could have just said “the long history doesn’t matter because I only care about the current state”. That’s fine and valid, but don’t ask questions and then shift the goalposts if you don’t like the answers.
        Dylan16807 2y ago
        A "loooong history" needs to have a timespan of many years. So yes it would start in the past, but it then has to continue for a long time. Pointing out that a company was bad 20 years ago isn't enough. You need to show they were also bad 15 years ago, and 10 years ago, and 5 and/or 25 years ago. So complaining that the only evidence was so far in the past is valid. The original goalposts were not reached. (Well, someone in another part of the thread eventually listed every google result for a windows update making anything crash, but that doesn't really establish that microsoft is "botching" updates at a level significantly above background noise, which I think was the original intent.)
    - tacticus 2y ago
      The company that let every db server have global admin creds and 0 logging on their cloud platform? That didn't run their own enhanced visibility on their own cloud platform.
    - lightedman 2y ago
      Vulnerabilities present in 2000 are showing up still in modern Windows versions. https://www.csoonline.com/article/564499/3-leaked-nsa-exploits-work-on-all-windows-versions-since-windows-2000.html https://www.csoonline.com/article/564499/3-leaked-nsa-exploi... You have no idea the cruft and technical debt Windows has in order to maintain its backwards compatibility.
  - TeMPOraL 2y ago
    That's a bit disingenuous, though. That was, as 'Rinzler89 points out, some 20 years ago. Back then, any Linux distro would've definitely been much safer option, because after installing you couldn't even connect it to the network, because it had no support for your cable modem or wireless card, and that's assuming you didn't fuck up your MBR with LiLo for the 20th time. Ask me how I know. Both OS families have changed much since that time.
    - rvnx 2y ago
      Oh sweet, this laptop has a PCMCIA Wi-Fi card! That'd be cool if one day I can get the laptop running on battery and not just on sector. Let me just setup it. Wait a second, how do I wake up the screen again and get out of this hibernation stage ? Why are all the fans stuck in 100% now ? Errr, first let's see if I can get the trackpad working.
    - lupusreal 2y ago
      On please, if it were that tough then teenage me never would have managed it. 20 years ago, e.g. 2004 (I first installed it in 2001), installing Linux and getting networked was already user friendly. The only hitch I ever had was figuring out ndiswrapper, but my ethernet cards all worked "out of the box" and installers handled the bootloader without users even having to know what a bootloader was. It's not like 20 years ago was the 90s or something, and the dark days of Windows lasted well into the 00s.
    - commercialnix 2y ago
      In 2002 I wasn't yet even out of middle school when I had a Linux distro running all key hardware components "just working". At that time at my school we were taught how to search the web, so I searched the web and looked up what hardware worked. Very simple. All I had to pitch to my parents was, "this system shares its code and encourages me to study it and learn code", which made clear to them what I was asking for wasn't just another video game console. Soon after I had a refurb laptop (fortunately not x86) and a curated WiFi card that ran Linux (and soon after, BSD) with everything "just working". When I see someone complain about unsupported/unsupportable chips in comments on online forums, especially one dubbed "Hacker News", I am puzzled how I in my middle school years acted out a pattern that is objectively smarter* than what I read in such comments. I also happen to first-hand know I am for sure not the only one with this vantage point. Those who comment about unsupported/unsupportable chips as if it is somehow an open source kernel's fault might want to take a moment to consider how others, and how many others, are viewing such drivel. For every one of us who take the time to point this out, there are 10,000 of us experiencing utter contempt, like as if we just got an unexpected whiff of some hot garbage. [*]And, I honestly don't think I'm even that smart.
  - feyman_r 2y ago
    Agree.I also remember those days when it was so hard to get Linux to just boot up and get your display working correctly- it was almost like a rite of passage. It was just proving grounds for how much of an expert you were and the number of hours you spent in front of the PC, just to get things working. My point is, good and bad memories will always stand out.
  - FireBeyond 2y ago
    And at that time Linux shipped with Telnet running...
- system2 2y ago
  Anyone who worked in IT knows this, it is not something rare. Literally every month, for example one from last month: https://www.techradar.com/computing/windows/windows-11-update-is-reportedly-causing-some-pcs-to-crash-or-run-very-sluggishly https://www.techradar.com/computing/windows/windows-11-updat... This is the main reason every IT professional I know disables auto updates of windows and manually trigger updates after testing (hopefully) on multiple dummy machines on the network. I personally remember booting to safe mode to remove Windows updates to rescue the computers more than I can count.
  - Rinzler89 2y ago
    Examples like that one I also found, but that's not really a "looooong list". If people can only show one single example as an argument it's kind of a moot point.
    - system2 2y ago
      You'd experience at least 3-5 per year if you work in IT. There really is a long list but since it is not my argument, I won't list them after searching for an hour. The list starts early 2000s, not recent. EDIT: Whatever, I will do the search for you since you cannot use google: https://www.pcgamer.com/an-odd-bug-in-this-months-windows-10-update-is-crashing-some-pcs/ https://www.pcgamer.com/an-odd-bug-in-this-months-windows-10... https://www.windowslatest.com/2023/10/22/windows-11-october-2023-update-crashing-games-and-file-explorer/ https://www.windowslatest.com/2023/10/22/windows-11-october-... https://www.bleepingcomputer.com/news/microsoft/windows-10-emergency-updates-released-to-fix-printing-crashes/ https://www.bleepingcomputer.com/news/microsoft/windows-10-e... https://www.windowslatest.com/2023/02/09/microsoft-confirms-a-windows-11-bug-is-crashing-some-apps-on-intel-pcs/ https://www.windowslatest.com/2023/02/09/microsoft-confirms-... https://www.windowslatest.com/2023/07/16/windows-11-kb5028185-issues-install-fails-crashes-pcs-and-other-bugs/ https://www.windowslatest.com/2023/07/16/windows-11-kb502818... These are just the last quarter of 2023. There is over 2000 news but I won't link them Use keywords: Windows Update, Crash, and use the date option on google go before 2023.
      - 2y ago
- GordonS 2y ago
  There's only been a few really bad ones, but Microsoft botch Windows updates quite regularly.
  - Rinzler89 2y ago
    >but Microsoft botch Windows updates quite regularly OK, please show us the proof then. If it's as regularly indeed like you claim then it must be documented somewhere as a greppable list. Tech blogs would have a field day getting traffic on their site by keeping track and documenting on such regular mistakes if they exist.
    - Brybry 2y ago
      It's frequent enough that people pay money for AskWoody[1] to tell them when it's safe to patch or what patches to skip. [1] https://www.askwoody.com/ms-defcon-system/ https://www.askwoody.com/ms-defcon-system/
      - Rinzler89 2y ago
        Quote, from the website: "In general, I apply Windows Defender updates as soon as they’re available. Why? Microsoft hasn’t screwed up any of them too badly. You’re better off applying those updates than letting them slide for a week or two."
        Brybry 2y ago
        Yep, Microsoft does a good job with Windows Defender (antivirus) updates. It's the other Windows Updates that they botch frequently enough to make people wary of patching immediately.
    - oxygen_crisis 2y ago
      Here's >100 of them in the past ~8 months: https://www.manageengine.com/patch-management/resources/microsoft-known-issues.html https://www.manageengine.com/patch-management/resources/micr...
- mrj 2y ago
  Well, from the news this morning: https://www.forbes.com/sites/daveywinder/2024/07/27/microsoft-confirms-it-broke-windows-as-30-minute-reboots-hit-after-update/ https://www.forbes.com/sites/daveywinder/2024/07/27/microsof...
drdec 2y ago
>> they need guidance on basic QA practices > Microsoft has a loooong history of botched (security) updates, so I'm not hopeful they can teach Crowdstrike much. Experience is the best teacher
- justinclift 2y ago
  Is MS doing it properly these days though? If they are, then you could be right. :)
- psychoslave 2y ago
  Attention to teacher is not equal between learners, trying to thoroughly assimilate the lesson is not everyone move, self challenging oneself with actual tests to ensure skill acquisition is rare, and going through the whole rabbit hole to figure out what untold assumptions the teacher leverage on and understanding the limits of these suggestions is the way only a few exceptional beings will follow.
cogman10 2y ago
And they've learned a lot from it. For example, MS no longer universally deploys updates across the world, they have a slower rollout to avoid just such an incident.
- sunaookami 2y ago
  Yeah now one million users loose access to their computer instead of 100 million!
  - fragmede 2y ago
    yes? that's 100x better! at the end of the day, internal testing just isn't going to catch every single permutation of customer configuration, so there's always a risk that something bad goes out. if you're that big, you'd start with .01% of the fleet instead of 1% of the fleet, so it's 100_000 before you get to 1_000_000, before going to 100% but neither Apple or Google have figured out a better way than that. It's industry standard at this point.

> they need guidance on basic QA practices Microsoft has a loooong history of botched (security) updates, so I'm not hopeful they can teach Crowdstrike much.

by holsta 2y ago

> they need guidance on basic QA practices Microsoft has a loooong history of botched (security) updates, so I'm not hopeful they can teach Crowdstrike much.

SoftTalker 2y ago

Yes, quite the epitome of throwing stones from a glass house.

Rinzler89 2y ago

Do you happen to have a list of that "loooong history" of botched (security) updates? I can only find a couple of examples after googling, which a bit smaller than a "loooong history" you're talking about, so unless Microsoft is paying Google to delete results, maybe you're mistaken.

SoftTalker 2y ago
This is a company whose OS could not even be installed on a live network without getting rooted within a few minutes. Anybody who was paying attention knew that you didn't use any new Windows release until at least the first service pack had come out. Granted that was a while back but painful memories die hard.
- Rinzler89 2y ago
  >This is a company whose OS could not even be installed on a live network without getting rooted within a few minutes. That was WIndows XP 20 years ago. Please bring arguments about modern Window 11 security which is the current up to date product they're selling and supporting not scenarios that haven't happened in 20 years.
  - clwg 2y ago
    First thing that comes to mind is that Recall stuff from a month ago, they also release updates[0] that crash machines. [0] https://www.tomsguide.com/news/windows-11-update-causing-blue-screen-of-death-how-to-fix-it https://www.tomsguide.com/news/windows-11-update-causing-blu...
    - TeMPOraL 2y ago
      Recall actually is a brilliant idea, and I dreamed of something like it for a long time, and so did plenty people here. It's just not something you can trust a third-party business with, whether it's a fly-by-night startup or an international megacorporation known to be openly promiscuous with advertisers. This is basically "take a screenshot every 30 seconds and compile it into a timelapse", but on steroids, and the same appeal, and arguments wrt. who gets to run it on whose machines, all apply.
      - dahdum 2y ago
        If you keep your business and personal computing separate, Recall looks amazing.
      - clwg 2y ago
        The functionality does seem intriguing, that doesn't change it's security profile which was poorly thought out and implemented.
    - feyman_r 2y ago
      Ignoring Windows Insider reports is bad. However, how many endpoints having issues (out of a billion+) is ‘acceptable’ after an update? We live in a news hype cycle so clearly even the one wrong failure will make it up somewhere. However, without metrics that show BSoDs from patches (which MS will likely never share), it’s hard to see if things have improved or regressed. If they regressed, someone up in their leadership chain is hopefully following the constructive discussion here.
  - Eduard 2y ago
    for a loooong history, you have to look in the past
    - Rinzler89 2y ago
      Ah, well, if only things of the past were useful today, I'd still have hair, and probably millions made form right investments, but unfortunately, it's what's happening today that actually matters.
      - echoangle 2y ago
        So you asked for proof of a long history and are now surprised that the examples are all from the past?
        Rinzler89 2y ago
        How does that impact the present? If it's no longer as vulnerable today, why would I care about the past? The point is learning from mistakes and fixing them so that doesn't happen again.
        echoangle 2y ago
        If it doesn’t matter to you, why did you ask? Are you just trying to win an argument or are you being intellectually honest? Because you asked for proof of the long history someone claimed. You could have just said “the long history doesn’t matter because I only care about the current state”. That’s fine and valid, but don’t ask questions and then shift the goalposts if you don’t like the answers.
        Dylan16807 2y ago
        A "loooong history" needs to have a timespan of many years. So yes it would start in the past, but it then has to continue for a long time. Pointing out that a company was bad 20 years ago isn't enough. You need to show they were also bad 15 years ago, and 10 years ago, and 5 and/or 25 years ago. So complaining that the only evidence was so far in the past is valid. The original goalposts were not reached. (Well, someone in another part of the thread eventually listed every google result for a windows update making anything crash, but that doesn't really establish that microsoft is "botching" updates at a level significantly above background noise, which I think was the original intent.)
  - tacticus 2y ago
    The company that let every db server have global admin creds and 0 logging on their cloud platform? That didn't run their own enhanced visibility on their own cloud platform.
  - lightedman 2y ago
    Vulnerabilities present in 2000 are showing up still in modern Windows versions. https://www.csoonline.com/article/564499/3-leaked-nsa-exploits-work-on-all-windows-versions-since-windows-2000.html https://www.csoonline.com/article/564499/3-leaked-nsa-exploi... You have no idea the cruft and technical debt Windows has in order to maintain its backwards compatibility.
- TeMPOraL 2y ago
  That's a bit disingenuous, though. That was, as 'Rinzler89 points out, some 20 years ago. Back then, any Linux distro would've definitely been much safer option, because after installing you couldn't even connect it to the network, because it had no support for your cable modem or wireless card, and that's assuming you didn't fuck up your MBR with LiLo for the 20th time. Ask me how I know. Both OS families have changed much since that time.
  - rvnx 2y ago
    Oh sweet, this laptop has a PCMCIA Wi-Fi card! That'd be cool if one day I can get the laptop running on battery and not just on sector. Let me just setup it. Wait a second, how do I wake up the screen again and get out of this hibernation stage ? Why are all the fans stuck in 100% now ? Errr, first let's see if I can get the trackpad working.
  - lupusreal 2y ago
    On please, if it were that tough then teenage me never would have managed it. 20 years ago, e.g. 2004 (I first installed it in 2001), installing Linux and getting networked was already user friendly. The only hitch I ever had was figuring out ndiswrapper, but my ethernet cards all worked "out of the box" and installers handled the bootloader without users even having to know what a bootloader was. It's not like 20 years ago was the 90s or something, and the dark days of Windows lasted well into the 00s.
  - commercialnix 2y ago
    In 2002 I wasn't yet even out of middle school when I had a Linux distro running all key hardware components "just working". At that time at my school we were taught how to search the web, so I searched the web and looked up what hardware worked. Very simple. All I had to pitch to my parents was, "this system shares its code and encourages me to study it and learn code", which made clear to them what I was asking for wasn't just another video game console. Soon after I had a refurb laptop (fortunately not x86) and a curated WiFi card that ran Linux (and soon after, BSD) with everything "just working". When I see someone complain about unsupported/unsupportable chips in comments on online forums, especially one dubbed "Hacker News", I am puzzled how I in my middle school years acted out a pattern that is objectively smarter* than what I read in such comments. I also happen to first-hand know I am for sure not the only one with this vantage point. Those who comment about unsupported/unsupportable chips as if it is somehow an open source kernel's fault might want to take a moment to consider how others, and how many others, are viewing such drivel. For every one of us who take the time to point this out, there are 10,000 of us experiencing utter contempt, like as if we just got an unexpected whiff of some hot garbage. [*]And, I honestly don't think I'm even that smart.
- feyman_r 2y ago
  Agree.I also remember those days when it was so hard to get Linux to just boot up and get your display working correctly- it was almost like a rite of passage. It was just proving grounds for how much of an expert you were and the number of hours you spent in front of the PC, just to get things working. My point is, good and bad memories will always stand out.
- FireBeyond 2y ago
  And at that time Linux shipped with Telnet running...
system2 2y ago
Anyone who worked in IT knows this, it is not something rare. Literally every month, for example one from last month: https://www.techradar.com/computing/windows/windows-11-update-is-reportedly-causing-some-pcs-to-crash-or-run-very-sluggishly https://www.techradar.com/computing/windows/windows-11-updat... This is the main reason every IT professional I know disables auto updates of windows and manually trigger updates after testing (hopefully) on multiple dummy machines on the network. I personally remember booting to safe mode to remove Windows updates to rescue the computers more than I can count.
- Rinzler89 2y ago
  Examples like that one I also found, but that's not really a "looooong list". If people can only show one single example as an argument it's kind of a moot point.
  - system2 2y ago
    You'd experience at least 3-5 per year if you work in IT. There really is a long list but since it is not my argument, I won't list them after searching for an hour. The list starts early 2000s, not recent. EDIT: Whatever, I will do the search for you since you cannot use google: https://www.pcgamer.com/an-odd-bug-in-this-months-windows-10-update-is-crashing-some-pcs/ https://www.pcgamer.com/an-odd-bug-in-this-months-windows-10... https://www.windowslatest.com/2023/10/22/windows-11-october-2023-update-crashing-games-and-file-explorer/ https://www.windowslatest.com/2023/10/22/windows-11-october-... https://www.bleepingcomputer.com/news/microsoft/windows-10-emergency-updates-released-to-fix-printing-crashes/ https://www.bleepingcomputer.com/news/microsoft/windows-10-e... https://www.windowslatest.com/2023/02/09/microsoft-confirms-a-windows-11-bug-is-crashing-some-apps-on-intel-pcs/ https://www.windowslatest.com/2023/02/09/microsoft-confirms-... https://www.windowslatest.com/2023/07/16/windows-11-kb5028185-issues-install-fails-crashes-pcs-and-other-bugs/ https://www.windowslatest.com/2023/07/16/windows-11-kb502818... These are just the last quarter of 2023. There is over 2000 news but I won't link them Use keywords: Windows Update, Crash, and use the date option on google go before 2023.
    - 2y ago
GordonS 2y ago
There's only been a few really bad ones, but Microsoft botch Windows updates quite regularly.
- Rinzler89 2y ago
  >but Microsoft botch Windows updates quite regularly OK, please show us the proof then. If it's as regularly indeed like you claim then it must be documented somewhere as a greppable list. Tech blogs would have a field day getting traffic on their site by keeping track and documenting on such regular mistakes if they exist.
  - Brybry 2y ago
    It's frequent enough that people pay money for AskWoody[1] to tell them when it's safe to patch or what patches to skip. [1] https://www.askwoody.com/ms-defcon-system/ https://www.askwoody.com/ms-defcon-system/
    - Rinzler89 2y ago
      Quote, from the website: "In general, I apply Windows Defender updates as soon as they’re available. Why? Microsoft hasn’t screwed up any of them too badly. You’re better off applying those updates than letting them slide for a week or two."
      - Brybry 2y ago
        Yep, Microsoft does a good job with Windows Defender (antivirus) updates. It's the other Windows Updates that they botch frequently enough to make people wary of patching immediately.
  - oxygen_crisis 2y ago
    Here's >100 of them in the past ~8 months: https://www.manageengine.com/patch-management/resources/microsoft-known-issues.html https://www.manageengine.com/patch-management/resources/micr...
mrj 2y ago
Well, from the news this morning: https://www.forbes.com/sites/daveywinder/2024/07/27/microsoft-confirms-it-broke-windows-as-30-minute-reboots-hit-after-update/ https://www.forbes.com/sites/daveywinder/2024/07/27/microsof...

drdec 2y ago

>> they need guidance on basic QA practices > Microsoft has a loooong history of botched (security) updates, so I'm not hopeful they can teach Crowdstrike much. Experience is the best teacher

justinclift 2y ago
Is MS doing it properly these days though? If they are, then you could be right. :)
psychoslave 2y ago
Attention to teacher is not equal between learners, trying to thoroughly assimilate the lesson is not everyone move, self challenging oneself with actual tests to ensure skill acquisition is rare, and going through the whole rabbit hole to figure out what untold assumptions the teacher leverage on and understanding the limits of these suggestions is the way only a few exceptional beings will follow.

cogman10 2y ago

And they've learned a lot from it. For example, MS no longer universally deploys updates across the world, they have a slower rollout to avoid just such an incident.

sunaookami 2y ago
Yeah now one million users loose access to their computer instead of 100 million!
- fragmede 2y ago
  yes? that's 100x better! at the end of the day, internal testing just isn't going to catch every single permutation of customer configuration, so there's always a risk that something bad goes out. if you're that big, you'd start with .01% of the fleet instead of 1% of the fleet, so it's 100_000 before you get to 1_000_000, before going to 100% but neither Apple or Google have figured out a better way than that. It's industry standard at this point.