10 ms·

When will people learn? 1. Stop putting mission critical systems on Windows, it's not the reliable OS it once was since MS has cut off most of its QA 2. AV so

• jmcgough 2y ago
  None of this advice is useful for massive organizations like banks and hospitals who got hit by this. They cannot switch off of windows for a number of reasons.
  • gloosx 2y ago
    They are not willing perhaps? Why cannot they?
    • cromka 2y ago
      Because reasons!
    • aenis 2y ago
      I'd attempt an answer. 1/ imagine running >1000 legacy applications, some never updated in 20 years 2/ imagine a byzantine mix of local data centers, VPCs in aws/gcp/azure 3/ imagine a IT departament run by a lot of people who have never learned anything new since they were hired That would be your typical large, boring entity such as a bank, public utility or many of the big public companies. Yeah, there is no law of physics preventing this, but it's actually nearly impossible to disentangle an organization from decades of mess.
      • qalmakka 2y ago
        That's why we've invented emulators, sandboxing, ... People have continued to run old management systems inside of virtual machines and similar solutions. You can sandbox it, reset it, do all kinds of wondrous things if you use modern technologies in an era-appropriate way. Run your old Windows software inside of a VM, or tweak it to run well on Wine if you have the source. The reason this mess happened is that all of those software are literally running a desktop OS in mission critical applications. I have worked as an embedded engineer for a while and I can't count the number of nonsensical stuff I've seen incompetent people running on unpatched, obsolescent Windows XP and 7 machines. This mess is 100% self inflicted.
      • gloosx 2y ago
        I think these are just technical excuses, but the real answer lies somewhere in the fields of politics and economics. If people in charge are to make a decision – then us tech nerds are going to migrate and refactor 1000 applications and update 20 years of byzantine code mess. I saw entities so large and boring they can barely move one step – changing rapidly and evolving once their economic stability is at stake, and this is a great example of such a disruption which can push them into chasm of change.
    • anigbrowl 2y ago
      Shareholders prefer profits to long-term investment. Thanks capitalism!
    • madaxe_again 2y ago
      Well, there’s this one app, written in VB6 using lots of DCOM that produces XML and XSLT transforms that only work in IE6, and the entire organisation depends on it, and the nephew who built it is now a rodeo clown and is unavailable for consultation. True story.
      • gloosx 2y ago
        He-he, entire organisation depending on IE6. I have good news and bad news...
    • account42 2y ago
      Because it would mess with the backroom deal the executive making the decision has with MS.
    • jmcgough 2y ago
      All of the hardware that's attached to workstations in our hospital are designed for windows. Certain departments have specific needs as well and depend on software that is Windows only. After decades of Windows it develops an insidious grasp that is difficult to escape, even moreso when your entire industry is dependent on Windows. Switching over to windows wouldn't just be extremely costly from an IT perspective but would require millions of dollars in new hardware. We are in the red in part because of the pandemic, existing problems in our industry accelerated by the last few years, and because a large percentage of our patients are on Medicare, which the fed govt shrinks fixed service payments for every year. I can't imagine convincing our administration to switch over to Linux across the hospital without a clear, obvious, and more importantly short-term financial payoff.
      • InsideOutSanta 2y ago
        I'm working for a company that has no Windows boxes at all, anywhere. Sure, some Windows software has no alternatives. We're running all of those programs in VMs. Does this make financial sense? Probably not in the short run, which is an issue for most companies nowadays. But in the long run? I think it's the right choice.
      • gloosx 2y ago
        It is not the hardware designed for windows but the driver code, which is most probably written in basic C, which most probably can be cross-compiled for usage outside Windows – so instead of millions of dollars in new hardware it is really thousands in porting the drivers and GUIs to the new platform. What works on windows in 90% cases is an easy porting job for the manufacturer, they just won't be doing it unless someone stops paying for windows version and be willing to pay for alternative platform port. Anyway, i totally agree with you. The convincing part here is short of clear and obvious for administration types. Until MS finally bricks it's OS and renders it totally unusable they can continue to do whatever shit they want and keep mocking their loyal customers forever.
  • chgs 2y ago
    Because they made a continuing choice over the years to write their systems to rely on windows?
  • the_gipsy 2y ago
    It's not a "3 step solution" that some IT guy can do now. It's a high level critique of using Windows and ignoring known problems.
  • mrweasel 2y ago
    There's nothing they can do right now, but my issue is that this will be forgotten when next update/purchasing round swings into action. Take Mærsk who couldn't operate their freight terminals due to a cyber attack and had the entire operation being dependent on a hard drive in a server that happened to be offline. Have they improved network separation? Perhaps. Have they limited their critical infrastructure to only run whitelisted application? I assure you they have not. They've probably just purchased a Crowdstrike license. Companies continuously fail to view their critical infrastructure as critical and severely underestimate risk.
    • Svip 2y ago
      Mærsk is kind of a bad example, because they made real security mitigations afterwards.[0] I cannot speak to whether they whitelist applications, but neither can you. [0] https://www.csoonline.com/article/567845/rebuilding-after-notpetya-how-maersk-moved-forward.html https://www.csoonline.com/article/567845/rebuilding-after-no...
  • qalmakka 2y ago
    That's the reason why I wrote, "stop putting" instead of "throw all of your PCs out of the window". Just like they migrated away from DOS they should start planning to migrate away from Windows to more modern, sandboxed solutions. There are ZERO reasons why a cash register shouldn't boot from a read-only filesystem, run AV, and so on.
• bruce343434 2y ago
  Linux isn't exactly reliable either... I'm sorry but that OS is barely capable of outputting a stable HDMI signal, god help you if you are on a laptop with external monitor. For 3 computers, 2 laptops, I've never _not_ had display bugs/oddities/issues. System upgrades always make me nervous because there is a very real chance of something getting fucked up and my screen staying black the next time it boots, having to go into a TTY, and manually fixing stuff up or booting the previous version that was still saved in GRUB. We can not get computers perfect. They are too complicated. That's true for anything in life. As soon as it gets too complicated, you're left in a realm of statistics and emergent phenomena. As much as I dislike windows enough to keep using Linux, I never had display issues on windows. To anyone compelled to reply with a text that contains "just" or "simply": simply just consider that if you are able to think of it in 10 seconds, then I have thought of it as well, and tried it too.
  • ben0x539 2y ago
    Yeah I mean you probably shouldn't put mission critical systems on a laptop with an external monitor either.
    • deleted 2y ago
      [deleted]
  • KronisLV 2y ago
    > For 3 computers, 2 laptops, I've never _not_ had display bugs/oddities/issues. System upgrades always make me nervous because there is a very real chance of something getting fucked up and my screen staying black the next time it boots, having to go into a TTY, and manually fixing stuff up or booting the previous version that was still saved in GRUB. I've had a Debian update break GRUB itself as well: https://blog.kronis.dev/everything%20is%20broken/debian-and-grub-are-broken https://blog.kronis.dev/everything%20is%20broken/debian-and-... I still use some Linux distros because when they work, they're pretty good, but when they don't, be prepared for a bunch of annoyances and debugging.
    • qalmakka 2y ago
• baq 2y ago
  This issue could easily happen on any other OS - Linux, macOS, BSDs - because it's a third party kernel driver which would be installed by the corporate IT regardless of anyone's opinion for compliance reasons. Your advice is incompatible with how the real world operates.
  • htrp 2y ago
    corporate IT heads need to roll for that to ever change. The romans used to make the architects stand under the arches they built, to enforce the idea of consequences for bad work.
  • monocasa 2y ago
    I've seen orgs get through soc2 and pci-dss without kernel anti virus. It's all about compensating controls.
    • baq 2y ago
      You should grab the folks who've done it and start authoring a book. You've got a 100x audience increase today!
    • bennyelv 2y ago
      Alas in the world of B2B, contracts from larger companies nearly always come with lists of specific requirements for security controls that must be implemented, which nearly always include requiring anti-virus. It just not as simple as commenters on this thread wish!
      • monocasa 2y ago
        The contracts are rarely specifying stuff like antivirus explicitly, but instead compliance with one or more of the security standards like PCI DSS. Those say you have to use antivirus, but they all have an escape hatch called a "compensating control" which is basically "we solved the problem this is trying to solve this other way that's more conducive to our overall security posture, and got the auditor to agree with us".
• KronisLV 2y ago
  > AV was never a necessity on UNIX, whatever MS bundles in is usually enough What prevents someone pushing a malicious package that takes my user data (that is accessible from a logged in session directly) and sends it somewhere? Especially in non-system repos, like Maven/NuGet/npm/pip/RubyGems and so on? What about the too widespread practice of piping shell scripts from the web, or applications with custom update mechanisms that might be compromised and pull in malicious code? I'm not saying that AV software would protect against all of these, but even if users don't do stupid things (which they absolutely will anyways, sooner or later), then there are still vectors of attack against any system. As for why *nix systems don't see that much malware, I've no idea, maybe because it's not as juicy of a target because of the lower count of desktop software installations (though the stuff that is on the systems might be more interesting to some, given the more tech savvy userbase), or maybe because a lot of the exploits focus on server software, like most CVEs. On Windows, I guess the built in AV software is okay, maybe with occasional additional scans by something like Malwarebytes, but that's situational.
  • bugbuddy 2y ago
    I would like to inform you that none of the AV product on the market will be able to protect you from piping a bad script from the web. Case closed.
    • darkwater 2y ago
      Crowdstrike agent is theoretically able to detect that what you just pipe-installed is now connecting to a known command and control server and can act accordingly.
      • pas 2y ago
        yes, as any competent classic old fart network-wide IPS/IDS endpoint security is a great utopia to strive for, but to get there we ought to starts with having secure by default endpoints.
    • yourusername 2y ago
      If your malicious script starts doing things like running well known payloads or trying to move laterally or access things it really shouldn't be trying to access AV will flag/block it.
• WatchDog 2y ago
  This isn't a windows issue. For what it's worth, I've had plenty of problems in the past with kernel panics from crowdstrike's macos system extension, although it was fairly random, nothing like today's issue.
• AdeptusAquinas 2y ago
  1. this is a crowdstrike issue not windows 2. plenty of malware and c2 systems happily operate off all systems, regardless of how hardened (or how unix) they are - IDS/IPS is a reactive way to try and mitigate this 3. you don't need third party software to compromise the unix kernel, you just need to wait a week or two until someone finds a bug in the kernel itself all that being said, this has solarwinds vibes. the push for these enterprise IDS systems needs to be weighted, the approach adjusted
• tamimio 2y ago
  Windows Defender is more than sufficient for most of these companies, but they need that false sense of security, or maybe they have excess budget to spare, or they are transferring the risk per their risk management plan.
  • bugbuddy 2y ago
    Bingo
  • benfortuna 2y ago
    Transferring the risk of malicious actors, but creating a resilience risk they are not owning.
• nox101 2y ago
  > Do not install third party software that runs in kernel mode. T You mean don't install Steam nor the Epic Store, nor many of the games. Note: I'm agreeing with you except that pretty much the only reason I have a Windows machine is for games. I do have Steam installed. I also have the Oculus software installed. I suspect both run in kernel mode. I have to cross my fingers that Valve and Facebook don't do bad things to me and don't leave too many holes. I don't install games that require admin. Oh, and I have Photoshop and I'm pretty sure Adobe effs with the system too >:(
  • Zr40 2y ago
    Steam does not have any kernel-mode components.
    • nox101 2y ago
      Steam asks for admin to be installed and asks for it again to install more features related to screen sharing. To me, any app that asks for admin is suspect.
      • Zr40 2y ago
        Admin privileges aren't the same thing as a kernel-mode driver. Steam does require admin to be installed, but it does not install a kernel-mode driver.
        • fragmede 2y ago
          are you sure about that? anti cheat stuff is pretty invasive these days
        • giantpotato 2y ago
          per microsoft admin to kernel is not a security boundary
      • computerfriend 2y ago
        Root and kernel are different levels.
• 2rsf 2y ago
  CrowdStrike Falcon is not an AV, Windows can be decently hardened and Microsoft did not "cut off most of its QA"
  • qalmakka 2y ago
    > Microsoft did not "cut off most of its QA" Windows RTMs used to be shipped in a usable state (albeit buggy) for more than a decade. You installed it from a CD and it worked fine, you installed patches every once in a while from a random Service Pack CD you got from somewhere. Modern Windows has had the habit of being so buggy after release in such horrendous ways that I can't imagine being able to use the same install CD for years. This is definitely putting less attention to detail in my view.
  • alt227 2y ago
    > Microsoft did not "cut off most of its QA" Yes they did. Hear about it first hand from an ex windows software tester: https://www.youtube.com/watch?v=S9kn8_oztsA https://www.youtube.com/watch?v=S9kn8_oztsA https://www.youtube.com/watch?v=lRV6PXB6QLk https://www.youtube.com/watch?v=lRV6PXB6QLk
  • tomashubelbauer 2y ago
    The slice of Microsoft stuff I worked at certainly did not have dedicated QA at the time I was there and used to have a QA team before, so there is some degree of truth to the statement. I can't speak for other Microsoft teams and offices. It was very disappointing for me, because I have had the opportunity to work with great QA staff before and in my current job and there is no way a developer dedicating 25 % of their time (which is what was suggested as a replacement for having dedicated QA) can do a job anywhere near as good.
  • szszrk 2y ago
    I have a feeling most commenters (not just here) don't really know what Falcon is and does, if EDR (and more?) keeps getting compared to a plain antivirus.
    • kasabali 2y ago
• cm2187 2y ago
  Not saying the microsoft QA is stellar but I also remember heartbleed.
• deutschlerner 2y ago
  > Stop putting mission critical systems on Windows, it's not the reliable OS it once was since MS has cut off most of its QA Dream on, that ship sailed long, long, long ago. It’s always funny to me reading comments on this site from users like this who have no idea how the real world operates
  • scheeseman486 2y ago
    Or doesn't operate, as it currently stands.
    • hulitu 2y ago
      > Or doesn't operate, as it currently stands. The definition of "operate" has changed over the years.
• systemvoltage 2y ago
  Why is this type of a comment upvoted to the top? It's a knee jerk reaction to the midwit zeitgeist. No real understanding of how the real world operates. No maturity or thoughtfulness. Totally misleading prognosis. Misleading advice.
• maeil 2y ago
  You missed the most important one: Have some kind of soaking/testing environment for production critical systems, especially if you're a big business. If you're hip, something like a proper blue/green setup (please chime in with best practices!). If you're legacy, do it all by hand if you must. Blindly enabling immediate internet-delivered auto-update on production systems will always allow a bad update to cause chaos. It doesn't matter how well you permission things off on your favourite Linux flavor. If an update is to be meaningful, the update can break the software. And clearly you're relying on the software, otherwise you wouldn't be using it.
• deleted 2y ago
  [deleted]
• hulitu 2y ago
  > Windows, it's not the reliable OS it once was And when exactly was Windows a reliable OS ? When it was turned off ? /s
• steven43126 2y ago
  Crowdstrike is not just "Antivirus" capability. Depending on the threats pertinent to the org they may require deep observability and the ability to perform threat hunting for new and emerging threats, or detect behaviour based signals, or move to block a new emerging threat. Not all threats require Administrator privileges! Not installing AV might be fine for a small number of assets in a low risk industry, but is bad advice for a larger more complex environment. If were unbiased here the apparent crowdstrike problem could occur on any OS and with any vendor where you have updates or configuration changes automatically deployed at scale.
• steve1977 2y ago
  > Linux has taught multiple times that drivers should be developed and included with the OS. I've had Linux GPU drivers fail multiple times due to system updates, to the point were I needed to roll back. I've had RHEL updates break systems in a way were even Red Hat support couldn't help me (I had to fix them myself). I don't see how Linux is any better in this regard than Windows to be honest. Also: > AV was never a necessity on UNIX Sure, why write a virus when you can just deploy your malware via official supply chains?
  • orwin 2y ago
    Do you have/need GPUs on your 'mission critical systems'? I would bet most of us don't. I quite agree with OP here. VMs are now quite lightweight (compared to available resources on machines at least) and I would rather use a light, hardened Linux as my base OS that will run windows VM and do snapshots for quick rollbacks. Actually, that's what I run on my own PC, and I think it would be the sanest way to operate.
  • rhim 2y ago
    Imagine a supply chain attack on a closed system and nobody finding out about it.