4 ms·

I found the article unbearable and just a convoluted way to say: this incident would have had a lot less impact if CrowdStrike had less customers or more compet

I found the article unbearable and just a convoluted way to say: this incident would have had a lot less impact if CrowdStrike had less customers or more competitors. A real page filler without any insight or solutions, just look at this paragraph, completely void of anything useful. > This time, the digital cataclysm was caused by well-intentioned people who made a mistake. That meant the fix came relatively quickly; CrowdStrike knew what had gone wrong. But we may not be so lucky next time. If a malicious actor had attacked CrowdStrike or a similarly essential bit of digital infrastructure, the disaster could have been much worse. Gee, the damage from an honest mistake (what does the author even base that on) is most likely easier to fix than the damage done by a malicious actor with bad intent. I feel so enlightened!

hsbauauvhabzb 2y ago
> mistake Funny way of spelling ‘complete failure in functional testing before going to prod causing billions of dollars worth of damage likely due to criminal level negligence’.
- tuatoru 2y ago
  Billions is probably an underestimate. When you factor on the time spent in meetings discussing endlessly what to do about this in corporations around the world, besides the cancelled flights and banking and hospital and 911 mayhem, a trillion isn't indefensible. Who needs North Korean hackers?
  - hsbauauvhabzb 2y ago
    If crowdstrikes internal competency failed to detect a consistent bsod, I doubt they’d notice North Korea dropping malware into their product solarwinds style.
- Galanwe 2y ago
  I mean, we've seen impacts of these unstaged updates in the deeper layers of banks, airports and hospitals. It's unfair that CrowdStrike gets more than half the blame here. Not to mention, probably half these companies were actually forced to install this. Getting SOC2 is essentially letting a senior-branded KPMG or EY intern tell you how to enforce security policies based on their checklist. There is far more to blame in this incident than CrowdStrike themselves.
  - aiisjustanif 2y ago
    Compliance is one thing but a cybersecurity incident response team is very blind without an EDR not to mention you would have less protection from insider threat.
- antonvs 2y ago
  That's true. But it's also true that companies depending on more or less random third parties to provide them with kernel modules for a monolithic kernel - and then giving them control over auto-updating those kernel modules - is beyond dumb.
- HeatrayEnjoyer 2y ago
  Undoubtedly killing people too: 911 networks failed. Hospitals were crippled. Short of getting your hands on a nuclear football it's difficult to imagine an act with a higher effort:devastation ratio.
  - RCitronsBroker 2y ago
    You’re not wrong about the end result, but the breakdown of systems this complex goes deeper than placing the blame on some CrowdStrike employee. Whoever thought up the great idea to allow auto-update-able kernel modules for something as mission critical as emergency response or healthcare deserves just as much blame. I’ve worked in healthcare for my whole career, this is madness. Not that their process is without flaw, but can we remind ourselves of how stringently we assess medical devices? I cannot imagine it’s controversial to say that emergency response equipment is every bit as critical as a insulin pump. If they fail, someone dies.
    - hsbauauvhabzb 2y ago
      But at the same time, auditing every update to an assurance level beyond ‘it didn’t bsod in test’ is incredibly hard. I don’t disagree with anything you’ve said, but I’d be very interested in solving the problem of actually auditing constant updates from vendors.
      - withinboredom 2y ago
        Even a rudimentary “delay autoupdate by two weeks” would have saved lives here. Let everyone else update first.
      - exe34 2y ago
        staged releases. don't cripple all your systems in one go. hot backups that you only update after the main system isn't dead from an update.
      - com 2y ago
        Automated CI/CD - many of us already do this hundreds of times a day. If you’re an emergency call centre, join a consortium of similar orgs and standardise tech and do it properly. Defer updates. Most things can wait 8-12 hours. Even more can wait 3 weeks (did this for all but security-critical npm package updates in one place). Demand legal changes to ensure fair liability for failure to undertake basic measures by service providers for paid software and services. Demand proper liability for C-suites not ensuring that actual risk management is in place instead of stupid box-ticking. Design better software. Seriously, the kinds of half-baked stuff that costs so much is incredible. It doesn’t take longer, and it doesn’t cost more to do things right, the only change is that management needs to be engaged with outcomes and have skin in the game. Execs should run the risk of going to jail for egregious failures.
notepad0x90 2y ago
Crowdstrike does not have a shortage of competitors, there are plenty in this field, at least a dozen. They just happen to be the best, primarily because of their intel, and you wouldn't believe this but because their sensor has always been very lightweight and easy to manage. The article is barely scratching the surface, supply chain security is a lot more scary than this.
- mylastattempt 2y ago
  My last comment was solely made to point out that this article is a garbage article. As for competitors and the dangers of supply chain attacks, I'm with you.
- khafra 2y ago
  > They just happen to be the best, primarily because of their intel, and you wouldn't believe this but because their sensor has always been very lightweight and easy to manage. I think an important part of their dominance is that they're the zero configuration EDR. Buy Carbon Black, and you'll need to know something about what's installed in your environment, what you do and don't want running. Buy SentinelOne, and you need to know that plus you have to learn a complex product. But with CS, you pretty much just install it, then get that sweet, sweet green checkmark on your security audit.
  - notepad0x90 2y ago
    You nailed it. It is a sweet green check mark but it also works well. The main reason CISO's get it so that if they ever do get compromised they can say they purchased the best security product out there. but there are still a certain portion of their customers that get targeted by APT's, think defense contractors, government orgs like the NRO, oil and gas, finance, power companies that have a large budget and that checkbox is the least of their worries.
xqt40 2y ago
You are right about that. The company seems to have bad QA process and that is not something that can be improved easily as it requires change in culture and management.

hsbauauvhabzb 2y ago

> mistake Funny way of spelling ‘complete failure in functional testing before going to prod causing billions of dollars worth of damage likely due to criminal level negligence’.

tuatoru 2y ago
Billions is probably an underestimate. When you factor on the time spent in meetings discussing endlessly what to do about this in corporations around the world, besides the cancelled flights and banking and hospital and 911 mayhem, a trillion isn't indefensible. Who needs North Korean hackers?
- hsbauauvhabzb 2y ago
  If crowdstrikes internal competency failed to detect a consistent bsod, I doubt they’d notice North Korea dropping malware into their product solarwinds style.
Galanwe 2y ago
I mean, we've seen impacts of these unstaged updates in the deeper layers of banks, airports and hospitals. It's unfair that CrowdStrike gets more than half the blame here. Not to mention, probably half these companies were actually forced to install this. Getting SOC2 is essentially letting a senior-branded KPMG or EY intern tell you how to enforce security policies based on their checklist. There is far more to blame in this incident than CrowdStrike themselves.
- aiisjustanif 2y ago
  Compliance is one thing but a cybersecurity incident response team is very blind without an EDR not to mention you would have less protection from insider threat.
antonvs 2y ago
That's true. But it's also true that companies depending on more or less random third parties to provide them with kernel modules for a monolithic kernel - and then giving them control over auto-updating those kernel modules - is beyond dumb.
HeatrayEnjoyer 2y ago
Undoubtedly killing people too: 911 networks failed. Hospitals were crippled. Short of getting your hands on a nuclear football it's difficult to imagine an act with a higher effort:devastation ratio.
- RCitronsBroker 2y ago
  You’re not wrong about the end result, but the breakdown of systems this complex goes deeper than placing the blame on some CrowdStrike employee. Whoever thought up the great idea to allow auto-update-able kernel modules for something as mission critical as emergency response or healthcare deserves just as much blame. I’ve worked in healthcare for my whole career, this is madness. Not that their process is without flaw, but can we remind ourselves of how stringently we assess medical devices? I cannot imagine it’s controversial to say that emergency response equipment is every bit as critical as a insulin pump. If they fail, someone dies.
  - hsbauauvhabzb 2y ago
    But at the same time, auditing every update to an assurance level beyond ‘it didn’t bsod in test’ is incredibly hard. I don’t disagree with anything you’ve said, but I’d be very interested in solving the problem of actually auditing constant updates from vendors.
    - withinboredom 2y ago
      Even a rudimentary “delay autoupdate by two weeks” would have saved lives here. Let everyone else update first.
    - exe34 2y ago
      staged releases. don't cripple all your systems in one go. hot backups that you only update after the main system isn't dead from an update.
    - com 2y ago
      Automated CI/CD - many of us already do this hundreds of times a day. If you’re an emergency call centre, join a consortium of similar orgs and standardise tech and do it properly. Defer updates. Most things can wait 8-12 hours. Even more can wait 3 weeks (did this for all but security-critical npm package updates in one place). Demand legal changes to ensure fair liability for failure to undertake basic measures by service providers for paid software and services. Demand proper liability for C-suites not ensuring that actual risk management is in place instead of stupid box-ticking. Design better software. Seriously, the kinds of half-baked stuff that costs so much is incredible. It doesn’t take longer, and it doesn’t cost more to do things right, the only change is that management needs to be engaged with outcomes and have skin in the game. Execs should run the risk of going to jail for egregious failures.

notepad0x90 2y ago

Crowdstrike does not have a shortage of competitors, there are plenty in this field, at least a dozen. They just happen to be the best, primarily because of their intel, and you wouldn't believe this but because their sensor has always been very lightweight and easy to manage. The article is barely scratching the surface, supply chain security is a lot more scary than this.

mylastattempt 2y ago
My last comment was solely made to point out that this article is a garbage article. As for competitors and the dangers of supply chain attacks, I'm with you.
khafra 2y ago
> They just happen to be the best, primarily because of their intel, and you wouldn't believe this but because their sensor has always been very lightweight and easy to manage. I think an important part of their dominance is that they're the zero configuration EDR. Buy Carbon Black, and you'll need to know something about what's installed in your environment, what you do and don't want running. Buy SentinelOne, and you need to know that plus you have to learn a complex product. But with CS, you pretty much just install it, then get that sweet, sweet green checkmark on your security audit.
- notepad0x90 2y ago
  You nailed it. It is a sweet green check mark but it also works well. The main reason CISO's get it so that if they ever do get compromised they can say they purchased the best security product out there. but there are still a certain portion of their customers that get targeted by APT's, think defense contractors, government orgs like the NRO, oil and gas, finance, power companies that have a large budget and that checkbox is the least of their worries.

xqt40 2y ago

You are right about that. The company seems to have bad QA process and that is not something that can be improved easily as it requires change in culture and management.