6 ms·
The OS doesn't matter, the question should be why is critical infrastructure online and allowed to receive OTA updates from third parties.
by lnrd 2y ago
The OS doesn't matter, the question should be why is critical infrastructure online and allowed to receive OTA updates from third parties.
- teej 2y agoThe OS absolutely matters
- deleted 2y ago[deleted]
- sixothree 2y agoThe alternatives aren't in a position fill the roles needed for the tasks at hand.
- rbanffy 2y agoWhat are the tasks in question?
- jeremycarter 2y agoThis is true. Linux large fleet management is still missing some features large enterprises demand. Do they need all those features, idk, but they demand them if they're switching from Windows.
- xvector 2y agoNo, what is stopping a similarly designed EDR from causing the same problem on Linux?
- lallysingh 2y agoFrom a comment above, Linux has features (ebpf) that key crowdstrike stay out of the kernel. The old "everyone else is just as bad" adage is bullshit. Some OSs are better suited than others.
- fragmede 2y agoFrom a comment elsewhere, a CS update took out Linux machines earlier this year.
- marcosdumay 2y agoThe culture around the OS matters. But this is a 3rd party software with ring-0 access to all of your computers deciding to break them. The technical features of the OS absolutely do not matter.
- rbanffy 2y agoThe question is whether other OSs would require it to have kernel mode privileges. People run complicated stuff in kernel mode for performance, because the switch to/from userspace is expensive. Guess what’s also expensive? A global outage is expensive. Much more than taking the performance hit a better, more isolated, design would avoid.
- marcosdumay 2y agoEDS run in kernel mode for access, not performance. They monkey-patch your syscalls.
- aylons 2y agoIt matters as in it makes it easy for this kind of issue to cause this much damage with little to no recourse for a fast correction. Not that Linux or whatever are all immune, but it definitely matters.
- meibo 2y agoYou should look into what a kernel driver is. You can panic a Linux kernel with 2 lines of code just as you can panic a Windows kernel, they just got lucky that this fault didn't occur in their Linux version. And to be honest, I don't think recovering from this would be that much easier for non-technical folk on a fully encrypted Linux machine, not that it's particularly hard on Windows, it's just a lot of machines to do it on.
- rbanffy 2y agoIn Linux it could be implemented as an eBPF thing while most of the app runs in userspace. And, for specialised uses, such as airline or ER systems, a cut-down specialised kernel with a minimal userland would not require the kind of protection Crowdstrike provides. I’m sure the NSA wasn’t affected by this.
- ashayh 2y agoebpf works in Windows as well.
- bg24 2y ago+1 With so much of tooling and products, it should come down to - What am I running and their current security state - Supply chain of any change that's happening - Test/Stage/Rollout any change - do not trust the vendor, as they do not know your infrastructure By allowing OTA update, they assumed that the vendor has tested all permutations.
- cpill 2y agoOf course the OS matters! Windows is a nasty ball of patches in order to maintain backward compatibility with the 80s. Linux and OSX don't have to maintain all the nasty hacks to keep this backward compatibility. Also, Crowdstrike is a security (patch) company because Windows security sucks to the point they have, by default, real-time virus protection running constantly (runs my CPU white hot for half the day, can you imagine the global impact on the environment?!). It's so bad on security that its given birth to a whole industry to fix it i.e. Crowdstrike. Every time I pass a bluescreen in a train station or advertisement I'd like "hA! you deserve that for choosing Windows".
- cyberax 2y ago> Of course the OS matters! Windows is a nasty ball of patches in order to maintain backward compatibility with the 80s. Linux and OSX don't have to maintain all the nasty hacks to keep this backward compatibility. Just don't tell that to Linus Torvalds :) Because Linux absolutely does maintain compatibility with old ABI from 90-s.
- rbanffy 2y ago> Just don't tell that to Linus Torvalds :) Because Linux absolutely does maintain compatibility with old ABI from 90-s. That’s nothing. IBM’s z/OS maintains compatibility with systems dating all the way back to the 60’s. If they want to think they are reading a stack of punch cards, the OS is happy to fool them.
- rbanffy 2y agoIBM’s z/OS maintains compatibility with the 60’s, and machines running it continued to process billions of transactions every second without taking a break. The OS matters, as well as the ecosystem and, and this is most important, the developer and operations culture around it.
- tivert 2y ago> The OS doesn't matter, the question should be why is critical infrastructure online and allowed to receive OTA updates from third parties. Not exactly. I think the question is why is critical infrastructure getting OTA updates from third parties automatically deployed directly to PROD without any testing. These updates need to go to a staging environment first, get vetted, and only then go to PROD. Another upside of that it won't go to PROD everywhere all at once, resulting in such a worldwide shitshow.
- gtvwill 2y agoLol if they can't do staging to mitigate balls ups on the high availability infrastructure side (optus in aus earlier this year pushed a router config that took down 000 emergency for a good chunk of the nation) we got bugger all hope of big companies getting it further up the stack in software.
- worik 2y ago> why is critical infrastructure getting OTA updates from third parties automatically deployed directly to PROD without any testing. I am missing some details, perhaps From what I see this was an update from Crowdstrike. They are a first party, no? Was another party involved?
- kortilla 2y agoThey are a third party software provider
- pertymcpert 2y agoI think you have the priority backwards. We shouldn’t be relying on trusting the QA process of a private company for national security systems. Our systems should have been resilient in the face of Crowdstrike incompetence.
- tivert 2y ago> I think you have the priority backwards. We shouldn’t be relying on trusting the QA process of a private company for national security systems. Our systems should have been resilient in the face of Crowdstrike incompetence. I think you misunderstood me. I wasn't talking about Crowdstrike having a staging environment, I was talking about their customers. So 911 doesn't go down immediately once Crowdstrike pushes a bad update, because the 911 center administrator stages the update, sees that it's bad, and refuses to push it to PROD. I think that would even provide some resiliency in the fact of incompetent system administrators, because even if they just hit "install" on every update, they'll tend to do it at different times of day, which will slow the rollout of bad updates and limit their impact. And the incompetent admin might not hit "install" because he read the news that day.
- lambda 2y agoWhile Linux isn't a panacea, the OS does matter as Linux provides tools for security scanners like Crowdstrike to operate entirely in userspace, with just a sandboxed eBPF program performing the filtering and blocking within the kernel. And yes, CrowdStrike supports this mode of operation, which I'll be advocating we switch over to on Monday. So yeah, for this specific issue, Linux provides a specific feature that would have prevented this issue.
- taftster 2y agoRight! Windows should NEVER blue screen. Ever. From a third-party software. Maybe Windows doesn't provide the right ABI or whatever for CS, but come on, you should never be able to kernel panic Windows. That this blue screened is 100% Microsoft's fault. It's a mess all the way around.
- CaliforniaKarl 2y agoBPF-based CrowdStrike is relatively recent, partially because, from the Enterprise Linux perspective, kernel support is relatively recent. For example, BPF-based CrowdStrike works on Enterprise Linux 9 and Debian 12. I don't know if the necessary support was in EL 8 or Debian 11.
- killerstorm 2y agoOS absolutely does matter. Windows has an enormous attack surface because Microsoft doesn't care. There's a number of minimal operating systems without all bells and whistles. The reason they aren't as popular choice is the "OS doesn't matter". If OS is minimal it doesn't need OTA updates, let alone from a third party...
- nequo 2y agoIn this case it wasn’t an update to the OS but an update to something running on the OS supplied by an unrelated vendor. But if we entertain the idea that another OS would not need CrowdStrike or anything else that required updates to begin with, I have doubts. Even your CPU needs microcode updates nowadays.