10 ms·
It's clear we need a full security audit of the companies that provide software tools to the companies that provide software tools to IT outsourcing shops. Of c
by tolbish 5y ago
It's clear we need a full security audit of the companies that provide software tools to the companies that provide software tools to IT outsourcing shops. Of course that just leaves things open to the next security breach of companies that provide software tools to the companies that provide software tools to the companies that provide software tools to IT outsourcing shops.
- ElViajero 5y agoI say this quite often, or we get better as a profession or we are going to get related to death. No society will tolerate failures in major businesses including food providers to happen continuously. Or we do better, or the next generation of developers is going to have regulated programming languages, architectures, and tech stack. Or we show that we can deliver safely and resist business pressure to deliver fast and cut costs, or the government is going to do it for us.
- rapind 5y agoWho is “we” though. I think a lot of deva cut corners, a lot of managers insist on cutting corners (preferably without a paper trail pointing at them), and a lot of businesses hire inexperienced devs that wouldn’t even know where to start, because cheaper. I don’t know how you coordinate this without regulation and something like P.Eng type certification (which comes with it’s own problems).
- nine_k 5y agoI think you should certify systems in place, not engineers. People without credentials but with experience and care can produce a reasonably secure system. People with credentials but under time pressure and contradictory requirements, and willingness to cut corners, can produce an unsafe system. It's the system that fails. It can be inspected the way buildings are inspected.
- MrQuimico 5y agoI totally agree.
- rapind 5y agoIdeally I agree, but I’m not sure it will be effective. There needs to be concrete consequences, and there’s a lot of room to hide. If we required something like a P.Eng, which (at least theoretically) means you are responsible with real consequences (for say bad building design) then certified employees will be far more reluctant to attach their name to shoddy practices. This bubbles up to the employer. If a client requires you to have certified P.Eng sign off then there’s some reassurance it’s not just going to fall a part. I don’t think you need this for everything in a stack, but security wise it might be a good idea.
- mcguire 5y ago"I don’t think you need this for everything in a stack..." The problem there is that everything on the stack is a potential hole. The chain is only as strong as its weakest link.
- mcguire 5y agoCertification of people is cheaper than certification of systems built by random people. For fun: Try to get a commercial building built that doesn't have an architect and structural engineer (both of which are certified professionals) somewhere in its background.
- rapind 5y agoThis is a really good point. Because the stakes are so high it makes sense. No one would want to live in a building that's yolo designed. Certifying and insuring it after the fact though would be really hard because a lot of critical decisions are hidden away. We accept that possibility in SaaS though... for now. Most successful SaaS applications probably weigh in at 250k LOC or more (I know LOC isn't a great metric, but it serves as an idea of an application's complexity) and will use many libraries that themselves could be questionable. Seems like it would be pretty complicated to certify.
- lisper 5y agoJust FYI, the "or ... or ..." construct you are using here sounds odd to a native English speaker. The correct idiom is "either ... or ...", e.g. "Either we get better as a profession, or we are going to get related to death." (And I'm also guessing you meant "regulated" rather than "related".)
- jerf 5y agoIf "related" was a typo, it was a happy one. The root problem is that we're getting more and more connected, but those connections have low security. It's too easy for bad stuff to flow across those connections. We're getting related to death. Much as developers in the past couple of years have really started to grapple with the fact that as wonderful as software libraries are, dependencies have non-zero costs that come with their benefits, connections need to be seen as being having non-zero costs as well. I work in a similar situation myself and I know my philosophy over the past 10 years has very much gone from a permissive "well maybe I'll need this later" open-ended protocol design to a super-strict "this connection moves exactly this set of strongly-typed, verified messages over it and if anybody tries any funny business we slam it shut and scream bloody murder on the monitoring" philosophy. Especially challenging with the Web's support for messaging, which is great for being a web page but is flabby and bloated for a messaging solution, what with all these headers that do magical things in the web servers or anything in between. As our systems get more and more complicated, biological metaphors seems to be ever more useful, and I think we need to look to nature's highly defensive "programming" a bit more for inspiration. Chemistry has some different characteristics from our programming environment, some to nature's advantage and some to its disadvantage, so we can't blindly copy it, but living things take 'message security' very seriously. You don't last long if you just blindly trust everything out there.
- user3939382 5y agoWe have self regulation available via NIST 800-53, NIST 800-30, SOC2, mandatory in some contexts like FISMA, etc. What is now guidance can become mandatory audits with the stroke of a congressional pen. I sound like a broken record but it bears repeating: Most of these attacks are successful because companies neglect best practices. Whitelisting, security awareness training, UEBA, etc go a long way. I would hope the free market would prune companies without proper cybersecurity but regulatory capture means it probably won’t. Equifax and its executives are doing just fine.
- cosmodisk 5y agoThat's the problem that free market prefers companies that have 100 bigger budgets for marketing than security teams. Unless this changes, nothing will change.
- afrodc_ 5y agoThe markets aren't always rational but a slew of security flaws increasing the drain of revenue by having to pay out ransomware or losing all of your data will be a driving force behind improving security.
- mcguire 5y agoThe true skill of choosing ransoms is to make sure that it's high enough to compensate you for the risk you take, but low enough to the other options are unappealing to the victim. Large businesses can trivially set aside a few million dollars per year for ransoms. Small businesses don't have the ability to change the system.
- afrodc_ 5y agoThat's a very good point. In that same vein, I do wonder if the hostage takers are that aware of the implications of their actions without letting greed guide them, not being a cohesive organization themselves.
- 5y ago
- hindsightbias 5y agoIf software engineering were physical, it would be like a non-stop series of Surfside condos and 737 Max crashes. People just don’t take virtual things as seriously, unless they involve conspiracies. *edit: when I say "people", I mean the end-users who would otherwise demand change.,
- ErikVandeWater 5y agoExcept software is behind the ~40 million successful flights around the world each year.
- sidlls 5y agoThe vast majority of software is more like hipster react web dev garbage than serious engineering.
- bluGill 5y agoThe vast majority of software doesn't need serious engineering either. If Google breaks that isn't a big deal, even though almost everyone in the world will be annoyed.
- mcguire 5y agoWhich systems do not need serious engineering? How much money or damage is "no big deal"? https://en.wikipedia.org/wiki/Equifax https://en.wikipedia.org/wiki/Equifax https://en.wikipedia.org/wiki/List_of_data_breaches https://en.wikipedia.org/wiki/List_of_data_breaches https://nvd.nist.gov/vuln/search/results?form_type=Basic&results_type=overview&search_type=all&isCpeNameSearch=false&hyperlink_types=US-CERT+Vulnerability+Notes https://nvd.nist.gov/vuln/search/results?form_type=Basic&res...
- bluGill 5y agoData breaches don't kill people in general. Medical devices can. Your point is well taken, even though things are not as bad they are still bad.
- nyghtly 5y agoDoing the right thing is bad for short term profits. Thus, government regulation in our industries is inevitable. In fact, I would argue that it's the only way to ensure consistent reliability. There's a reason why Facebook is pushing for regulation. They know that it can't be stopped, and in that case they would prefer to be the one's writing the regulations.
- nine_k 5y agoBut government regulation very often is preferring short-term profits to doing the right thing. Just the profits are political. Think of all the security theater in airports, for instance. Think of all the to and fro in many policies as different officials get elected every few years.
- cartoonworld 5y agoIt's the CYA thats preferred over CIA that causes execs, share holders and customers to accept this dangerous malfeasance. It's far less the programmers than the business plans that demand minimal investment. These are classic externalities that serve to damage society at large.
- acomjean 5y agoI think we’ve lost control of our machines. At least when we attach them to internet. I’m not sure what the answer is but better security and a rethinking of user authorizations seems to be in order.
- pyrale 5y agoWe never had it. It was just not really useful to destroy someone else's toys. The key to the current spree of ransomware is the massively improved ability to monetize digital hostage-taking. I don't really understand how financial watchdogs have let this go through, but cryptos have become a massive loophole in kyc and anti-laundering regulations. Recent moves in that sector seem to hint that this party is about to end and, hopefully, will create enough friction to reduce ransomware activity.
- silexia 5y agoCryptocurrencies have been a disaster. The only place to actually use is a currency is in criminal activity. The amount of environmental damage caused by proof of work is massive. We need regulation.
- ChrisClark 5y ago>The only place to actually use is a currency is in criminal activity. This discredits your entire post. Your other points may be valid, but how can anyone be sure if this one is provably false?
- silexia 5y agoThe statement was a bit hyperbolic, but not by much. Have you ever used cryptocurrency for any routine transaction? How much total money have you spent in cryptocurrency if so? It's very newsworthy when people like Elon musk say they will accept cryptocurrency for the very reason that most businesses do not. And I would be surprised if more than a handful of Teslas have been sold via cryptocurrency.
- raesene9 5y agoIt does seem that more serious maintenance and auditing of these providers is needed. Looking at this thread on reddit https://old.reddit.com/r/msp/comments/ocggbv/crticial_ransomware_incident_in_progress/ https://old.reddit.com/r/msp/comments/ocggbv/crticial_ransom... which is by a company looking into the incident and how it happened, it seems that it was exploited vulnerabilities in Classic ASP files used as part of the solution. Having classic ASP pages hosted on a production system in 2021 seems like a pretty strong indication of a lack of codebase maintenance and auditing.
- giardini 5y agoraesene9:5 says >"Having classic ASP pages hosted on a production system in 2021 seems like a pretty strong indication of a lack of codebase maintenance and auditing."< AFAIK classic ASP pages can be as secure as those in any other framework. The vulnerabilities (most commonly SQL injection) are known and are addressable. I know it chafes at some (Microsoft marketers esp.?) that ASP pages are still around. But classic ASP is yet another example of that old adage "If it ain't broke, don't fix it!" I watched one organization assign an entire division of programmers to develop a moderately-sized ASP.NET application: they went through orientation and training in ASP.NET and then planned, designed, coded and rolled out ...nothing! After two years there was literally nothing! A perceptive manager in another division approached his sole ASP developer and asked if she could write some ASP code to "demo" that same project. She looked at the specifications and then quietly wrote an entire system! Weeks later, when the department heads saw her "demo", they thought the ASP.NET developers had completed it. Imagine their surprise to find that what they were looking at was done, not by the ASP.NET group funded millions of dollars but by someone in another division: a single focused classic ASP developer quietly working downstairs. Nonetheless IIRC ASP pages reach end-of-life support by Microsoft in 2025 so it might make sense to migrate. But b/c classic ASP was written in a manner consistent with early web standards (CGI/APACHE) migration of classic ASP to one of the various classic ASP-like frameworks in PHP, Perl, Ruby et al would likely be easier, faster and cheaper. Migration would be mostly translation and much, if not all, could be automated. In contrast, moving classic ASP to ASP.NET would be more fraught with problems. The underlying models of the WWW are inconsistent.
- p49k 5y agoIs that good enough? It seems like we have reached a point where people can perpetually find zero-days in any system and that we need to completely start over - from chip design to the highest level programming languages - with security in mind.
- kube-system 5y agoThat's not enough either; the humans overseeing the redesign will also make human errs.
- nine_k 5y agoIt's not like we don't have safe enough CPUs, safe enough languages and programming techniques, or understanding how to architect safe systems and usage practices. It just costs a very significant amount of money. Many businesses just don't see the lowered risk to be worth the expense.
- oezi 5y agoTurtles all the way down. The only way to get secure software is to increase liability of parties involved. My suggestion: Start with the ability of any customer to return any purchase (hardware/software) which contains software with a disclosed but unfixed CVE after 90 days without a patch. If this doesn't get rid of the Internet of Shit, I don't know what will. Next, set a minimum damage rate of 100 USD per user for each data-breach that involves personal information and 1000 USD for any special kinds of personal information (credit cards, etc) and 10000 USD for any protected health information breach.
- Hermitian909 5y ago> Start with the ability of any customer to return any purchase (hardware/software) which contains software with a disclosed but unfixed CVE after 90 days without a patch. If this doesn't get rid of the Internet of Shit, I don't know what will. Does this not also just kill tech? CVEs pop up decades after products have died. Now every tech product is just one unsupported CVE away from losing _all_ lifetime revenue. I just can't see how anyone would ever invest again... edit: to clarify further the fact that any CVE triggers this, no matter how small, seems egregious to me. The idea of there being no lifetime on the liability seems wild given how CVEs are often the result of other developers breaking ABIs. Imagine a profitable product that was last sold 10 years ago having it's full lifetime revenue refunded because of some change in glibc.
- oezi 5y agoWell, some limits of support should be given: - How about 5 years minimum for hardware? And as much as the vendor wants to promise. - How about that requiring that vendors at least allow for customers to pay for extended support for another 5 years by paying 20% of the initial price per year. It is just ridiculous that currently many devices are insecure 3 months out of the gate.
- korethr 5y agoWhy not try to combine it with some right-to-repair-friendly stuff? If, after the cessation of support, you release any and all source code and documentation needed for any person competent in the relevant sciences and arts to maintain the device and repair any CVEs, you're off the hook for liability. I had played around with the idea of requiring support for 3/5/7/10/whatever years after the cessation of sale, kind of like how car manufacturers are required to offer parts support for 10 years after sale, but I can see that causing enough overhead that many tech devices simply would never get made.
- IncludeSecurity 5y agoWe do security audits for a living. In a nut shell, here's why things are so screwed up IMHO: 1) Most of these companies have had audits, but they're being done by 3rd rate or very inexperienced external consultants. 2) The companies limit the scope of the tests. Real hackers don't give a shit about your scope of work, they have no rules, only goals. 3) Even when a test is properly done the exec management looks for silver bullet product solutions instead of changing across people/process/technology My company solves #1, but we can't do anything about #2 or #3 :-/
- user-the-name 5y agoWhat audit would have found a zero-day vulnerability?
- raesene9 5y agoIf https://old.reddit.com/r/msp/comments/ocggbv/crticial_ransomware_incident_in_progress/ https://old.reddit.com/r/msp/comments/ocggbv/crticial_ransom... is correct a compentent web application security review (white box or black box) which was correctly scoped to include the affected files would likely have found the SQLi and authentication bypass issues (mentioned in update 12) Without seeing the codebase in question, you can't be sure, but having been a web app pentester for 10+ years, these are the kind of issues that were found regularly, and whenever I saw classic ASP in tests, they were the kind of issues I'd be looking for, knowing the inherent weaknesses in the platform.
- dogman144 5y ago"The attackers were able to exploit zero-day vulnerabilities in the VSA product to bypass authentication and run arbitrary command execution," the Miami-headquartered company noted in the incident analysis. "This allowed the attackers to leverage the standard VSA product functionality to deploy ransomware to endpoints. There is no evidence that Kaseya's VSA codebase has been maliciously modified." This is very likely not the full story, unless the 0day in VSA was somehow wormable. That "deployment" is doable through overly permissive IAM and everything else that enables privesc. There are two parts to these vulns. Whatever gets the foothold, and whatever allows privilege escalation. Audits do a great job in catching the misconfigs that allow privesc. The tragic thing about these attacks is often the blast radius can be contained fairly easily by asking the right questions... If you're someone who has passed these audits, or done these audits, it becomes pretty easy to see how many unforced errors go into these catastrophic attacks.
- dogman144 5y agoHaha it's clear to basically every competent security team that this audit has to occur, but it usually is not clear to product, engineering and operations.
- kolbe 5y agoI’m beginning to think this line of advocacy is a red herring. If we’ll-endowed entities like nation-state backed ransomeware attackers want to completely ransack companies with nine figure budgets, they will always win. To me, this is more of a foreign policy issue. I’d say the damages caused by an attack like this can be requantified in loss of American life, and treat it like such. What should we do if Russia were killing 20 US civilians every few weeks?