5 ms·
Another article blaming the upstream vendor and not bothering to put any onus on the horrible security practices of companies allowing auto updating of executab
by averageRoyalty 2y ago
Another article blaming the upstream vendor and not bothering to put any onus on the horrible security practices of companies allowing auto updating of executable code in production on critical systems.
This is unacceptable practice. I understand non tech media not getting it, but this lack of awareness from tech news is sad.
- axlee 2y agoYou want your antivirus to be autoupdating.
- mr90210 2y agoBy now I’d expect people including you to have a more sophisticated perspective about third-party software. Yesterday was a catastrophe and you are still stuck with such naive and simplistic view: you want your antivirus to be auto updating.
- mewpmewp2 2y agoHow can an anti virus software protect from new threats if it can't auto update as soon as new threat is there?
- averageRoyalty 2y agoIt's a trade off. That said, we're in an age where companies do 100+ pushes per day. Automate a build, run a test, then deploy rolling updates across the fleet. The options aren't "everyone auto updates or no updates for weeks", there's a balance point. It's very clear what choice most critical companies this week did though.
- nikau 2y agoCrazy idea, for critical systems dont give them blanket internet access, USB, email attachments...
- nerdjon 2y agoRealistically what is the alternative if you are running servers that could seriously be the target of an attack? I will give you that I highly doubt that a large number of these machines are anywhere near that critical nature, but there are some that will fall within that much risk. What do you do, just not update to handle new risks? A lot of systems going down is really bad, don't get me wrong. But is it worse that you could be breached depending on the data (and other services) those systems may have access too? To me this is a flaw in Crowdstrike but also Windows that this could happen in the first place, and a serious flaw on Crowdstrike's side that this somehow got out. And yes I do acknowledge that much of this is security theatre, but I also would not be surprised if it does sometimes work.
- averageRoyalty 2y agoTo be clear, you blame CrowdStrike, Windows (??) but not the companies who picked this software, configured it and wrote their own internal risk policies around a kernel level piece of software?
- nerdjon 2y agoMost of the blame here falls on Crowdstrike. Both from a software standpoint that it can cause a BSOD so easily and not be able to handle something like this happening. But also whatever failure happened to let that file get out. Some, minor, blame falls on Windows due to its ability to BSOD as easily as it does. As far as the companies, it is a tricky situation. Many of the companies have Crowdstrike enabled and automatic updates turned on to check some audit box. They have to keep the updates going out regularly. We are well past the point in tech that a company is solely responsible for their systems with external dependencies being the norm. Either with the shared security model with cloud services like AWS or a reliance on external API's and servers. You have to trust the vendor you are working with for whatever critically important system is going to do their job. Could you look back and say that maybe you chose the wrong vendor for a specific piece of software, but this could have happened to other vendors. Something that I am not entirely sure of is for those audit, compliance, etc requirements can they use an alternative update method. And this is something that would be different based on each compliance, but to the best of my knowledge for security software most want you to have automatic updates. If this was the case of all of these servers going down because of a major AWS outage would you really be saying the companies are to blame?
- tuxone 2y agoNot on production critical systems where there are human lives at stake. Last Friday is a pretty good example of what comes together with ungoverned ‘autoupdate’.
- mewpmewp2 2y agoSo let's imagine that it has to be updated manually. New threat appears and since it takes a while to manually update it means bad actors can act on it meanwhile, causing a similar or even worse disruption since it could have far more severe impact, because of the bad intents. Would that be better?
- averageRoyalty 2y ago"Immediate across the fleet" and "Entirely manual process" are not the only two options. HN rules say we must assume good faith, but there are obviously options in between, and all of them stop the issue that happened on Friday.
- mewpmewp2 2y agoWhat option would you pick if Crowdstrike found a vulnerability that could affect everyone involved?
- averageRoyalty 2y agoYour argument is the 0.01% of cases should dictate the other 99.99%s actions? I would pick automated testing and spread fleet deploys. There's no reason in any enterprise this should take more than 1-2 hours, which is a perfectly acceptable window of risk.
- mewpmewp2 2y agoI'm not fully sure what you mean by 0.01% cases? Where did you get those percentages? Businesses are under a constant barrage of cyber attacks, with goals to steal the data, encrypt it and then blackmail or sell all the data. Ransomware payouts exceeded $1 bil last year. And that doesn't include all the damage done besides the payouts. Edit: Supposedly global cost of cybercrime is expected to reach $20 trillion+ by 2027.
- averageRoyalty 2y agoMaybe if my antivirus has basic filtering of input values. But in a critical systems scenario, I want to validate in my testing stage first, or at least run a split rollout so that my entite fleet doesn't shit the bed.
- belorn 2y agoNot if you are running critical systems and the antivirus is not 100% guarantied to be safe and running in an isolated environment. Hospitals should not loose their ability to provide care to sick people, just because of an misconfiguration of an antivirus. That is as bad as airplanes crashing because of a lack of redundancy and management of risks.
- xyst 2y agolol, haven’t used AV in a long time. It’s security theater and it’s trivial for a malware dev to get around these programs. Only stops script kiddies, at best.
- 56745349 2y ago[dead]
- gruez 2y ago>companies allowing auto updating of executable code in production on critical systems. crowdstrike said the update was a "configuration file". https://www.crowdstrike.com/blog/technical-details-on-todays-outage/ https://www.crowdstrike.com/blog/technical-details-on-todays...
- averageRoyalty 2y agoYou're right, but I don't think it changes anything. Third party software bringing anything new in needs to go through a test phase. That could be automated and a simple 30 minute VM boot, regular tasks run, and a longer 24 hour one running in the background whilst you start rolling updates. Either way, it's unacceptable for critical services to be beholden to the validation of a third party upstream. The companies in question are responsible for that negligent handing off of ownership.
- gruez 2y ago>Third party software bringing anything new in needs to go through a test phase. That could be automated and a simple 30 minute VM boot, regular tasks run, and a longer 24 hour one running in the background whilst you start rolling updates. I don't think anyone is going to disagree that's engineering best practice and should theoretically be done, but how is microsoft going to enforce this? Do you want to force developers wanting to publish software for windows to undergo annual audits (soc-2 style) to confirm that all the engineering best practices are indeed being followed? Not even Apple is that strict.
- averageRoyalty 2y agoI don't understand why people keep bringing Microsoft into this. This testing is the responsibility of the company whose computer fleet it is. They have many upstream software vendors - often 10s or 100s - and should be doing this testing every time. You should never rely on the vendor to test (evidently). I'd go so far as to assume all vendor updates are hostile and build your test model against that. Automated testing of new software in companies with 10k+ desktops (which covers most affected companies here) should be as common as password policies or email attachment policies. If the vendor implements things in a way that doesn't allow this style of testing, they don't meet security requirements and another should be found.
- richbell 2y ago> the horrible security practices of companies allowing auto updating of executable code in production on critical systems. That's the irony of the situation. The criticality of the systems (arguably) necessitates real-time updates, otherwise they'd be vulnerable to threat actors.