23 ms·
Took down our entire emergency department as we were treating a heart attack. 911 down for our state too. Nowhere for people to be diverted to because the other
by jmcgough 2y ago
Took down our entire emergency department as we were treating a heart attack. 911 down for our state too. Nowhere for people to be diverted to because the other nearby hospitals are down. Hard to imagine how many millions of not billions of dollars this one bad update caused.
- ibeff 2y ago[flagged]
- deutschlerner 2y ago[flagged]
- dang 2y agoIt's against the site guidelines to post like this, and we have to ban accounts that to it repeatedly, so if you'd please review https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and stick to the rules when posting here, we'd appreciate it.
- mcs5280 2y ago[flagged]
- bhaney 2y ago> Surely none of the medical devices needed to treat a heart attack are Windows PCs connected to the internet? Wouldn't that be nice
- jmcgough 2y agoWithout access to Epic we can't place med orders, look up patient records, discharge patients from the hospital, enter them into our system, really much of anything. Every provider in the emergency department is on their computer placing orders and doing work when not interacting with a patient. Like most hospitals in this country, our entire workflow depends on Epic. We couldn't even run blood tests because the lab was down too. The STEMI was stabilized, it's more that it was scary to lose every machine in the department at once while intubating a crashing patient. You're flying blind in a lot of ways.
- ibeff 2y ago[flagged]
- basch 2y agoIf the computer system was down, and medicine was needed to save a life, would either some protocol dictate grabbing the medicine and dealing with the paperwork or consequence later? If protocol didn’t allow for discussion, would staff start breaking protocol to save life?
- TestingWithEdd 2y agoYou can skip paperwork but what if the patient is allergic to a medicine and you need to check medical records? Or you need to call for a surgeon but VoIP is down? Etc…
- p_l 2y agoMy father's coworker died from being in hospital for observation after few scratches in car accident because they were accidentally given medication they were allergic to. So, yeah. The paperwork can save lives too and not sadly red tape is bad. Otherwise you may go o hospital to pickup your friend and be told to wait for coroner
- xmdx 2y agoI'm guessing they were being treated over the phone as the systems went down. I've been through a similar situation, the person on the phone will give step by step instructions while waiting for an ambulance to arrive. Sounds like with the systems being down the call would have been cut off which sounds horrible.
- jmcgough 2y agoNo, treating in person. But we can't function as a department without computers. You call cardiology (on another floor) and none of their computers are working to be able review the patients records. You could take the EKG printout and run it to them, but we're just telling them lab results from what we can remember before our machines all bluescreened. The lab's computers were down so we can't do blood tests. Nursing staff knows what to do next by looking at the board or their computer. Without that you're just a room full of people shouting things at each other, and definitely can't see the 3-4x patients an hour you're expected to. Doctors and midlevels rely on epic to place med orders too.
- A4ET8a8uTh0 2y ago[dead]
- epolanski 2y agoMay I say that starting from "treating a heart attack" and ending up worrying about millions lost in productivity sounds a bit "wrong"?
- plonk 2y agoBillions in losses means a somewhat worse life for a huge number of people and potentially much worse healthcare problems down the line, the NHS was affected
- jmcgough 2y agoI just had a ten hour hospital shift from hell, apologies if my writing is lacking. I can't think of a better way to try to measure the scope of the damage caused by this.
- yard2010 2y agoTake care of yourself. you're making the world a better place. You deserve better supportive technology, not this shit show.
- TeMPOraL 2y agoStarting with an ER story kind of set up the expectation that you'll be "measuring the scope of the damage" in lives lost, not dollars. Though I guess at large enough scale, they're convertible. Regardless, thanks for your report; seeing it was very sobering. I hope you can get some rest, and that things will soon return to normalcy.
- monkmartinez 2y agoJust completed a standing 24 due to this outage. My B-Shift brothers and sisters had to monitor the radios all night for their units to be called for emergencies. I heard every dispatch that went out. We were back in the 1960's with paper and pen for everything, no updates on nature of call, no address information, nothing... find out when you show up and hope the scene is secure. It was wild as it was coupled to a relatively intense Monsoon storm.
- 2y ago
- plonk 2y agoDid the person survive?
- jmcgough 2y agoWe have limited visibility into this in the emergency department. You stabilize the patient and admit them to the hospital, then they become internal medicine or ICU's patient. Thankfully most of the work was done and consults were called prior to the outage, but they were in critical condition.
- perfectstorm 2y agoi'll admit i have no idea what i'm talking about but isn't there some Plan B options? something that's more manual? or are surgeons too reliant on computers?
- nubinetwork 2y agoI don't know about surgeons, but nursing and labs have paper fallback policies... they can backload the data later.
- regularfry 2y agoIt's often the case that the paper fallbacks can't handle anywhere near the throughput required. Yes, there's a mechanism there, but it's not usable beyond a certain load.
- Bjartr 2y agoI think it's eventually manageable for some subset of medical procedures, but the transition to that from business as usual is a frantic nightmare. Like there's probably a whole manual for dealing with different levels of system failure, but they're unlikely to be well practiced. Or maybe I'm giving these institutions too much credit?
- zamadatix 2y agoThere are plan B options like paper charting, downtime procedures, alternative communication methods and so on. So while you can write down a prescription and cut a person open you can't manually do things pull up the patient's medical history for the last 10 years in a few seconds, have an image read remotely when there isn't a radiologist available on site, or electronically file for the meds to just show up instantly (all depending on what the outage issue is affecting of course). For short outages some of these problems are more "it caused a short rush on limited staff" than "things were falling apart". For longer outages it gets to be quite dangerous and that's where you hope it's just your system that's having issues and not everyone in the region so you can divert. If the alternatives/plan b's were as good or better than the plan a's then they wouldn't be the alternatives. Nobody is going to have half a hospital's care capacity sit as backup when they could use that year round to better treat patients all the time, they just have plans of last resort to use when what they'd like to use isn't working. (worked healthcare IT infrastructure for a decade)
- swyx 2y agoi mean not just dollars but lives also right? do we have a way to track that?
- beeboobaa3 2y agoAnd how many lifes lost?
- kyledrake 2y agoI remember a fed speaker in the 90s at Alexis hotel Defcon trying to rationalize their weirdly over-aggressive approach to enforcement by mentioning how hackers would potentially kill people in hospitals, fast forward to today and it's literally the "security" software vendor that's causing it.
- emodendroket 2y agoNot like hackers haven’t done the same.
- Quarrelsome 2y agonothing like this scale. These machines are full blue screen and completely inoperable.
- toomuchtodo 2y agoThe problem is concentration risk and incentives. Everyone is incentivized to follow the herd and buy Crowdstrike for EDR because of sentiment and network effects. You have to check the box, you have to be able to say you're defending against this risk (Evolve Bank had no EDR, for example), and you have to be able to defend your choice. You've now concentrated operational risk in one vendor, versus multiple competing vendors and products minimizing blast radius. No one ever got fired for buying Crowdstrike previously, and you will have an uphill climb internally attempting to argue that your org shouldn't pick what the bubble considers the best control. With that said, Microsoft could've done this with Defender just as easily, so be mindful of system diversity in your business continuity and disaster recovery plans and enterprise architecture. Heterogeneous systems can have inherent benefits.
- mango7283 2y agoIf you have a networked hybrid heterogeneous system though now you have weakest link issue, since lateral movement can now happen after your weaker perimeter tool is breached
- jodrellblank 2y ago> "Took down our entire emergency department as we were treating a heart attack." Not questioning that it happened, but this was a boot loop after a content update. So if the computers were off and didn't get the update, and you booted them, they would be fine. And if they were on and you were using them, they wouldn't be rebooting, and it would be fine. How did it happen that you were rebooting in the middle of treating a heart attack? [Edit: BSOD -> auto reboot]
- sleepydog 2y agoNot OP, but some (most? many?) machines receiving the update crashed with a BSOD. So that's how they could enter the boot loop.
- jodrellblank 2y agoI just realised I had read that, but 4 minutes later and it's too late to delete my comment now; Thanks, yes it makes sense.
- lupire 2y agoWhat happens when a computer gets rebooted as part of daily practice or because of the update, and then it becomes unusable, and then the treatment team needs to use it hours later?
- jodrellblank 2y agoI dunno, but they'd know about it hours earlier in time to switch to paper, or pull out older computers, or something - in that scenario it wouldn't have happened "as we were treating a heart attack" and they would have had time to prepare.
- TeMPOraL 2y agoBeyond the BSOD that happened in this case, in general this is not true with Windows: > And if they were on and you were using them, they wouldn't be rebooting, and it would be fine. Windows has been notorious for forcing updates down your throat, and rebooting at the least appropriate moments (like during time-sensitive presentations, because that's when you stepped away from the keyboard for 5 minutes to set up the projector). And that's in private setting. Corporate setting, the IT department is likely setting up even more aggressive and less workaround-able reboot schedule. Things like this is exactly why people hate auto-updates.
- davycro 2y agoThe system crashed while my coworker was running a code (aka doing CPR) in the ER last night. Healthcare IT is so bad at baseline that we are somewhat prepared for an outage while resuscitating a critical patient.
- smsm42 2y agoNow this is an unusual meeting of two meanings of "running a code".
- iamtheworstdev 2y agothere's a great meme out there that says something like: Everyone on my floor is coding! \n Software PMs: :-D \n Doctors: :-O
- jmcgough 2y agoWhen you're a software engineer turned doctor you get sent that by all of your friends xD
- dwatson92 2y agoExcuse my ignorance, but what systems are needed for CPR?
- pksebben 2y agoMy guess is the system that notifies the next caretaker in the chain that someone is currently receiving CPR. if it works, there's a lot more to be done to get the patient to stable.
- SamuelAdams 2y agoI used to work in healthcare IT. Running a code is not always only CPR. Different medications may be pushed (injected into the patient) to help stabilize them. These medications are recorded via a bar code and added to the patients chart in Epic. Epic is the source of truth for the current state of the patient. So if that is suddenly unavailable that is a big problem.
- ww520 2y agoI saw one of the surgery videos recently. The doctor was saying, "Alexa, turn on suction." It boggled my mind. There could be so many points of failure.
- senortumnus 2y agoFwiw this is not typical, we don’t have alexa/siri type smart devices in any OR I work in, and suction is turned on off with a button and a dial.
- ww520 2y agoIt's in a Maryland clinic doing plastic surgery. Edit: Found it. https://www.youtube.com/watch?v=nS9nLvGMLH0&t=947s https://www.youtube.com/watch?v=nS9nLvGMLH0&t=947s
- nullbyte 2y agoALEXA, TURN OFF THE SUCTION! ALEXA!! “Loive from NPR news in Washington“
- CamperBob2 2y agoI don't suppose there was a doctor or nurse named Alexa involved?
- importantbrian 2y agoWife is a nurse. They eventually go 2 computers working for her unit. I don't think it impacted patients already being treated, but they couldn't get surgeries scheduled and no charting was being done. Some of the other floors were in complete shambles.
- SarahWSJ 2y agoHi, as I noted to another commenter, I'm a journalist looking to speak with people who've been impacted by the outage. I'm wondering if I could speak with your wife. My email is sarah.needleman@wsj.com. Thanks.
- importantbrian 2y agoSure I’ll pass your email along to her and see if she wants to do that.
- willmadden 2y agoI'm guessing hundreds of billions if you could somehow add it all up. I can't believe they pushed updates to 100% of Windows machines and somehow didn't notice a reboot loop. Epic gross negligence. Are their employees really this incompetent? It's unbelievable. I wonder where MSFT and Crowdstrike are most vulnerable to lawsuits?
- lanstin 2y agoThis outage seems to be the natural result of removing QA by a different team than the (always optimistic) dev team as a mandatory step for extremely important changes. And neglecting canary type validations. The big question is will businesses migrate away from such a visibly incompetent organization. (Note I blame the overall org; I am sure talented individuals tried their best inside a set of procedures that asked for trouble.)
- spaceywilly 2y agoSo there was apparently an Azure outage prior to this big one. One thing that is a pretty common pattern in my company when there are big outages is something like this: 1. Problem A happens, it’s pretty bad 2. A fix is rushed out very quickly for problem A. It is not given the usual amount of scrutiny, because Problem A needs to be fixed urgently. 3. The fix for Problem A ends up causing Problem B, which is a much bigger problem. tl;dr don’t rush your hotfixes through and cut corners in the process, this often leads to more pain
- SoftTalker 2y agoTheir terms of use undoubtedly disclaim any warranty, fitness for purpose, or liability for any direct or incidental consequences of using their product. I am LMFAO at the entire situation. Somewhere, George Carlin is smiling.
- fsloth 2y agoIf you’ve ever been forced to use a PC with Crowdstrike it’s not amazing at all. I’m amazed incident of this scale didn’t happen earlier. Everything about it reeks of incompetence and gross negligence. It’s the old story of the user and purchaser being different parties-the software needs to be only good enough to be sold to third parties who never neeed to use it. It’s a half-baked rootkit part of performative cyberdefence theatrics.
- HaZeust 2y agoYup - my mom went into the ER for stroke symptoms last night and was put under MRI. The MRI imaging could NOT be sent to the off-site radiologist and they had to come in -- turned out the MRI outputs weren't working at all. We were discharged at midnight by the doctor, the nurse didn't come into our exam room to tell us until 4am. I can't imagine the mess this has caused.
- davidw 2y agoI hope she's ok.
- olwmc 2y agoI wish your mother the best <3
- HaZeust 2y agoThank you <3
- jmcgough 2y agoHope she's okay. For better or worse, our entire emergency department flow is orchestrated around epic. If we can't even see the board, nurses don't know what orders to perform, etc.
- deleted 2y ago[deleted]
- Red_Leaves_Flyy 2y agoIf it’s so critical that nurses are left standing around clueless then if it goes down entire teams of people should be going to prison for manslaughter. Or, we could build robust systems that can tolerate indefinite down time. Might cost more, might need more staff. Pick one. I’ll always pick the one that saves human lives when systems go down.
- pca006132 2y ago
- rsync 2y ago"Took down our entire emergency department as we were treating a heart attack. 911 down for our state too." Why would Windows systems be anywhere near critical infra ? Heart attacks and 911 are not things you build with Windows based systems. We understood this 25 years ago.
- Wytwwww 2y agoWell you can use stupid broken software with any OS, not just Windows. Isn't CrowdStrike Falcon available on Linux, is there any reason why couldn't they have introduced a similar bug and similar consequences there?
- tw04 2y agoNone. There are a bunch of folks here who clearly haven’t spent a day in enterprise IT proclaiming Linux would’ve saved the day. 30 seconds of research would’ve lead them to discover crowdstrike also runs on Linux and has created similar problems on Linux in the past.
- rowanG077 2y agoOh could you link me the source of the claim that all linux clients of crowdstrike went down all at once? I'm very interested to hear it.
- p_l 2y agoIt's even better when you get told about the magical superiority of apple for that... ... Except Apple pretty much pushes you to run such tools just to get reasonable management key alone things like real-time integrity monitoring of important files (Crowdstrike in $DAYJOB[-1] is how security knew to ask whether it was me or something else that edited PAM config for sudo on corporate Mac)
- nikau 2y agoEnterprise mac always follows the same pattern, users proclaim its superiority while its off the radar, then it gets mcaffee, carbon black, airlock, and a bunch of other garbage tooling installed and runs as poorly as enterprise Windows. The best corporate dev platform at moment is WSL2 - most of the activity inside the WSL2 vm isn't monitored by the windows tooling so performance is fast. Eventually security will start to mandate agents inside the WSL2 instance, but at the moment most orgs dont.
- drhelix 2y agoWhy is the emergency department using windows?
- et2o 2y agoBecause essentially every large hospital in the USA does?
- niutech 2y agoThis is the result of vendor lock-in and the lesson for all businesses not to use Microsoft servers. Linux/*BSD are rock-solid and open source.
- shados 2y agoIt's not just that. A large portion of IT people who work in these industries find Windows much easier to administer. They're very resistant to switching out even if it was possible and everything the company needed was available elsewhere. Even if they did switch, they'd then want to install all the equivalent monitoring crap. If such existed, it would likely be some custom kernel driver and it could bring a unix system to its knees when shit goes wrong too.
- marcthe12 2y agoI mean crowdstrike has a linux equivalent which broke rhel recently by triggering kernel panic
- bell-cot 2y agoHigh-end hospital-management software is not simple stuff, to roll your own. And the (very few) specialty companies which produce such software may see no reason to support a variety of OS's.
- michaelmrose 2y agoA follow up question is why is the one OS chosen the one historically worst at security.
- charles_f 2y ago> Hard to imagine how many millions of not billions of dollars this one bad update caused. I mean, if the problem is that hospitals can't function anymore, money is hardly the biggest problem
- charles_f 2y ago> Took down our entire emergency department What does this mean? Did the power go down? Is all the equipment connected? Or is it the insurance software that can't run do nothing gets done? Maybe you can't access patient files anymore but is that taking down the whole thing?
- jmcgough 2y agoEvery computer entered a bluescreen loop. We are dependent on Epic for placing orders, for nursing staff to know what needs to be done, for viewing records, for transmitting and interpreting radiology machines. It's how we know the current state of the department and where each patient (out of 50+ people we are simultaneously treating) is at. Our equipment still works but we're flying blind and having to shout orders at each other and have no way to send radiology images to other doctors for consultation.
- just_1_comment 2y agoYeah in Radiology we depend on Epic and a remote reading service called VRAD. VRAD runs on AWS and went down just after 0130 hrs EST. Without Epic & VRAD we were pretty helpless.
- jmcgough 2y agoCan't imagine how stressful this must have been for Radiology. I had two patients waiting on CT read with expectation to discharge if no acute findings. Had to let them know we had no clear estimate for when that would be, and might not even know when the read comes back if we can't access epic. Have a family member in crit care who was getting a sepsis workup on a patient when this all happened. They somehow got plain film working offline after a bit of effort.
- atoav 2y agoNot to be that guy, but I often say software engineering as a field should have harsher standards of quality and certainly liability for things like these. You know like civil engineers, electrical engineers and most people whose work could kill people if done wrongly. Usually when I write this devs get all defensive and ask me what the worst thing is that could happen.. I don't know.. Could you guarantee it doesn't involve people dying? Dear colleagues, software is great because one persons work multiplies. But it is also a damn fucking huge responsibility to ensure you are not inserting bullshit into the multiplication.
- whydoyoucare 2y agoI believe instances like this will push people to reconsider the lax stance. Humans in general have a hard time regulating something abstract. The fact that people can be killed is well-known since the 80s', see https://en.wikipedia.org/wiki/Therac-25 https://en.wikipedia.org/wiki/Therac-25
- suzzer99 2y agoI once worked on some software that generated PDFs of lab reports for drug companies monitoring clinical trials. These reports had been tested, but not exhaustively. We got a new requirement to give doctors access to print them on demand. Before this, doctors only read dot matrix-printed reports that had been vetted for decades. With our XSL-FO PDF generator, it was possible that a column could be pushed outside the print boundary, leading a doctor to see 0.9 as 0. I assume in a worst worst case scenario, this could lead to an misdiagnosis, intervention, and even a patient's death. I was the only one in the company who cared about doing a ton more testing before we opened the reports to doctors. I had to fight hard for it, then I had to do all the work to come up with every possible lab report scenario and test it. I just couldn't stand the idea that someone might die or be seriously hurt by my software. Imagine how many times one developer doesn't stand up in that scenario.
- atoav 2y agoThis is why I made that point, similar to you I would not stand for having my code in something that I can't stand behind, especially if it potentially harms people. But it should not hinge on us convincing people.
- sizzle 2y agoContact a lawyer if this affected her health please. Any delays in receiving Stroke care can have injured her more I imagine. Any docs here?
- bookofjoe 2y agoYes. Reading and learning.
- psychlops 2y agoI can't even begin to imagine the cost of proving the health effects and attempting to win the case.
- jmcgough 2y agoER worker here. It really depends on the details. If she was C-STAT positive with last known normal within three hours, you assume stroke, activate the stroke team, and everything moves very quickly. This is where every minute counts, because you can do clot busting to recover brain function. The fact that she was discharged without an overnight admit suggests to me that the MRI did not show a stroke, or perhaps she was outside the treatment window when she went to the hospital.
- sizzle 2y agoWhat if it was a cerebral bleed?
- WhyNotHugo 2y agoIt's honestly terrifying that someone would opt for Windows in systems critical to medical emergencies. I hope organisations start revisiting some of these insane decisions.
- afavour 2y agoEh. There are a great many problems that could befall a medical emergency systems that are unrelated to OS. Like power loss. I think the core problem here really is a lack of redundancy.
- wannacboatmovie 2y agoI've had updates break Linux machines. Just a few weeks ago I had an OpenBSD box render itself completely unbootable after nothing more than a routine clean shutdown. Turns out their paranoid-idiotic "we re-link the kernel on every boot" coupled with their house-of-cards file system corrupted the kernel, then overwrote the backup copy when I booted from emergency media - which doesn't create device nodes by default so can't even mount the internal disks without more cryptic commands. Give me the Windows box, please.
- type0 2y agoyou are comparing a broken bicycle to a trainwreck
- sgarland 2y agoCounter anecdote: I’ve been using Linux for 20 years, nearly half of that professionally. The only time I’ve broken a Linux box where it wasn’t functional was mixing Debian unstable with stable, and I was still able to fix it. I’ve had hardware stop working because I updated the kernel without checking if it removed support, but a. that’s easily reversible b. Linux kept working fine, as expected. I’ll also point out, as I’m sure you know, that the BSDs are not Linux.
- dgan 2y agoFunny, i broke my Debian twice (on two separate laptops) by doing exactly that, mixing stable with testing. I was kinda obliged to use "testing" because Dell XPS would miss critical drivers. I switched to opensuse afterwards
- ayakang31415 2y agoIf true, this is insane that critical facilities like hospital do not have decentralized security system.
- mandevil 2y agoCrowdstrike is on every machine in the hospital because hospitals and medical centers became a big target for ransomware a few years ago. This forced medical centers to get insured against loss of business and getting their data back. The insurance companies that insure companies against ransomware insist on putting host based security systems onto every machine or they won't cover losses. So Crowdstrike (or one of their competitors) has to run on every machine.
- ensignavenger 2y agoI wonder if those same insurance policies are going to pay out due to the losses from this event?
- chiph 2y agoDepends on what the policy (contract) says. But there's a good argument that your security vendor is inside the wall of trust at a business, and so not an external risk.
- golergka 2y agoIn a sense, it looks like these insurance company's policies work a little bit like regulation. Except that it's not monopolistic (different companies are free to have different rules), and when shit hits the fan, they actually have to put their money where their mouth is. Despite this horrific outage, in the end it sounds like a much better and anti-fragile system than a government telling people how to do things.
- ensignavenger 2y agoA little bit, probably slightly better. But insurance companies don't want to eliminate risk (if they did that, no one would buy their product). They instead want to quantify, control and spread the risk by creating a risk pool. Good, competent regulation would be aimed at eliminating, as much as reasonably possible, the risk. Instead, insurance company audits are designed to eliminate the worst risk and put everyone into a similar risk bucket. After spending money on an insurance policy and passing an audit, why would a company spend even more money and effort? They have done "enough".
- hassiktir 2y agoI dont understand how this isnt bigger news? Local emergency services were basically nonfunctioning for better part of the day along with the heat wave and various events, seems like a number of deaths (locally at least, specific to what I know for my mid sized US city) will be indirectly attributable to this.
- jmcgough 2y agoIt's entirely possible (likely, even) that someone died from this, but it's hard to know with critically ill patients whether they would have survived without the added delays.
- b800h 2y agoOn aggregate it is. How many deaths over the average for these conditions did we see?
- hassiktir 2y agoWe are in the process of calculating this but need this 24H period to roll over so we can benchmark the numbers against a similar 24H period. Its hard to tell if the numbers we get back will even be reliable given a lot of the statistics back from today from what I can tell have been via emails or similar.
- tucnak 2y agoSo what?
- callalex 2y agoGive it like, a week before bothering to ask such questions...
- jjav 2y ago> Took down our entire emergency department as we were treating a heart attack. It makes my blood boil to be honest that there is no liability for what software has become. It's just not acceptable. Companies that produce software with the level of access that Crowdstrike has (for all effective purposes a remote root exploit vector) must be liable for the damages that this access can cause. This would radically change how much attention they pay to quality control. Today they can just YOLO-push barely tested code that bricks large parts of the economy and face no consequences. (Oh, I'm sure there will be some congress testimony and associated circus, but they will not ever pay for the damages they caused today.) If a person caused the level and quantity of damage Crowdstrike caused today they would be in jail for life. But a company like Crowdstrike will merrily go on doing more damage without paying any consequence.
- throwaway7356 2y ago> Companies that produce software What about companies that deploy software with the level of quality that Crowdstrike has? Or Microsoft 365 for that matter. That seems to be the bigger issue here; after all Crowdstrike probably says it is not suitable for any critical systems in their terms of use. You shouldn't be able to just decide to deploy anything not running away fast enough on critical infrastructure. On the other hand, Crowdstrike Falcon Sensor might be totally suitable for a non-critical systems, say entertainment systems like the Xbox One.
- rvnx 2y agoCrowdStrike https://www.crowdstrike.com https://www.crowdstrike.com › resources › infographics Learn how CrowdStrike keeps your critical areas of risk such as endpoints, cloud workloads, data, and identity, safe and your business running
- HarryTom 2y ago[dead]
- fransje26 2y ago> Hard to imagine how many millions of not billions of dollars this one bad update caused. And even worse, possibly quite a few deaths as well. I hope (although I will not be holding my breath), that this is the wake-up call we need to realise that we cannot have so much of our critical infrastructure rely on the bloated OS of company known for its buggy, privacy-intruding, crapware riddled software. I'm old enough to remember the infamous blue-screen-of-death Windows 98 presentation. Bugs exist but that was hardly a glowing endorsement of high-quality software.. This was long ago, yet it is nigh on impossible to believe that the internal company culture has drastically improved since then, with regular high-profile screw-ups reminding us of what is hiding under the thin veneer of corporate of respectability. Our emergency systems don't need windows, our telephone systems don't need windows, our flight management systems don't need windows, our shop equipment systems don't need windows, our HVAC systems don't need windows, and the list goes on, and on, and on. Specialized, high-quality OSes with low attack surfaces are what we need to run our systems. Not a generic OS stuffed with legacy code from a time when those applications were not even envisaged. Keep-it-simple-stupid -KISS-is what we need to go back to, our lives literally depend on it. With the mutli-billion dollars screw-up that happened yesterday, and an as-of-yet unknown number of deaths, it's impossible to argue that the funds are unavailable to develop such systems. Plurality is what we need, built on top of strong standards for compatibility and interoperability.
- fluoridation 2y agoOK, but this was a bug in an update of a kernel module that just happened to be deployed on Windows machines. How many OSs are there that can gracefully recover from an error in kernel space? If every machine that crashed had been running, say, Linux and the update had been coded equivalently, nothing would've changed. Perhaps rather than an indictment on Windows, this is a call to re-evaluate microkernels, at least for critical systems and infrastructure.
- fransje26 2y agoIt was not a call to replace windows systems with linux, but to replace it with specialised OSes that do less, with better stability guarantees. And building something around microkernels would definitely not be a bad starting point.