7 ms·
"This Is How They Tell Me the World Ends" tackles an important question: What causes the vulnerability of our modern world to "cyberattacks"? What causes that
by Animats 6y ago
"This Is How They Tell Me the World Ends" tackles an important question: What causes the vulnerability of our modern world to "cyberattacks"?
What causes that vulnerability? Lack of liability for software vendors. If Microsoft had to pay the costs of vulnerabilities, we'd have far more secure systems.
There's one area of the industry where companies are held financially responsible for their mistakes - gambling systems. A few percent of revenue goes to paying for mistakes. For GTech, before they were acquired by a non-US company, you could see the numbers in their annual report. It's not a killer.
- tptacek 6y agoSince nobody knows how to reliably ship secure commercial software, liability will mostly have the effect of making it difficult to start new software businesses. Liability would make more sense to me if we could converge on a common understanding of a secure development process, but working on that has taken up most of my career and I don't think it's even on the horizon. The industry is still debating memory safety.
- devonkim 6y agoWe have things like cybersecurity insurance that's required to be carried for vendors when working with the government, but much greater investment in both private and public sectors into secure coding and operational practices (namely by making commonly insecure things much more secure) would be helpful to at least some trends away from the current dominant mentality of "ship it first, ship it fast" dominating the software business. I don't think it makes much sense for a random one-off script written by some lone developer starting a company to be subjected to all the alphabet soup of regulation, but I don't think letting everyone get away with security breaches forever is a good idea nor is just throwing up our hands and going "oh well, we're going to keep getting break-ins" for another 50 years.
- ampdepolymerase 6y agoGalois, if you are listening, there is a billion dollar opportunity for building a provably secure SAP/Salesforce.
- mcguire 6y ago"Liability would make more sense to me if we could converge on a common understanding of a secure development process, but working on that has taken up most of my career and I don't think it's even on the horizon." That seems unlikely to happen as long as it is in commercial software businesses' best financial interests that it not happen.
- Veserv 6y agoThat is only a problem if you create mandatory fixed-cost liability requirements. You can solve the problem by allowing companies to opt-in to liability requirements for some benefit. As an example, we could require that any company that wants to advertise security must do so in the form of a number specifying how much they will pay their customers in the event of a breach. So, they would not be able to advertise something like: "We have a secure cloud database.". They would instead be required to say: "We have a cloud database with $10M security." and in the event of a breach be forced to pay out $10M to its customers. So, if a company is small and unable to afford any security, they can just set the number at $0 which means it is the customer's problem. However, if the company is big and people have security expectations, they would need to specify a reasonable number that would alleviate those concerns. As long as the number is clearly communicated and you are not allowed to fraudulently or misleadingly advertise a different number, then customer's would be able to make an informed decision with respect to the security level and liability they are accepting from their vendors. Obviously there are some complexities with respect to properly communicating this information. For instance, for a consumer-facing company you would probably want a per-consumer number instead of an aggregate number. As an example, if Apple were to claim $300M for the iPhone, that might seem like a large number to an average person, but that would only amount to ~$1.50 per iPhone sold in a year. You also need to prevent misleading advertising that might attempt to divert from the quantitative liability they are actually accepting. However, the base scheme of mandatory labelling requirements along with opt-in liability should allow for a solution that is hard to bypass while being flexible enough to support both small and large businesses.
- joe_the_user 6y agoSo, if a company is small and unable to afford any security, they can just set the number at $0 which means it is the customer's problem. And if a company is large and low margin, they can do this too so nothing changes. A few companies will be out there promising things, like today, and maybe they can deliver and maybe you can sue them if they don't, like today.
- setr 6y agoWell, right now everyone and everything claims to be secure; this puts a number on that claim, and gives you companies explicitly claiming that they’re not secure (at least, they wouldn’t bet on it), which is quite a bit more information than you have today. Being able to actually get your money back... is another problem. But obviously a toothless regulation is toothless, and changes nothing, so its not a useful assumption to make
- ImprobableTruth 6y agoWhat's the issue with requiring a formal spec and being liable for deviations from it? Though obviously this would have to be restricted to certain areas.
- Aerroon 6y agoBecause chances are the the formal spec itself won't be perfect. But even if it is you still have to worry about every layer below you in the software stack on top of the hardware. You might do everything perfectly at great cost and it all gets rendered moot by spectre/meltdown/rowhammer etc.
- ImprobableTruth 6y ago>Because chances are the the formal spec itself won't be perfect It's not about creating 'perfect software', but about holding developers accountable. If a customer accepts a contract with a faulty spec, that's on them, not the developer. >But even if it is you still have to worry about every layer below you in the software stack on top of the hardware. You might do everything perfectly at great cost and it all gets rendered moot by spectre/meltdown/rowhammer etc. So what? It's not your fault. There's only one party that should be held liable for spectre, Intel.
- wglb 6y agoBut it aldo affects mainframes, no?
- ImprobableTruth 6y agoWell, it also affects certain AMD and ARM processors as far as I know, I was just talking about a specific example. If software can be exploited because of a bug in hardware, I think it only makes sense to assign liability to the hardware company.
- Aerroon 6y ago
- aidenn0 6y ago> Since nobody knows how to reliably ship secure commercial software, liability will mostly have the effect of making it difficult to start new software businesses. Do you literally mean "nobody" here? As in if I wanted to hire you to put a team together to reliably ship secure commercial software, you couldn't do it either? I think we have, as an industry, tacitly agreed that it's better to ship cheap, insecure software than it is to ship expensive, secure software. A lot of the innovation in software has occurred specifically because it is so inexpensive. It seems clear to me that, so far, this tradeoff has been a net benefit, but at some point we may want to trade the inexpensiveness and innovation for some security, particularly has we start to put software into more things that can kill and maim us.
- snaxxx 6y ago> Do you literally mean "nobody" here? As in if I wanted to hire you to put a team together to reliably ship secure commercial software, you couldn't do it either? It's fundamentally impossible given that you won't control everything about the operating environment of the software, and can't see into the future. I mean, even if you could create a perfect piece of software with no bugs present at the time, that's still not enough, because vulnerabilities in dependent components will arise which were unknown at the time. For example, if you'd written the perfect web browser a few years back, how would you prevent vulnerability to CPU-level attacks such as Spectre and Meltdown discovered later on? Or consider if you write the perfect wireless networking firmware, but then end up being subject to vulnerabilities in the protocol standards (WEP was a good example of this)? This is on top of software development being such a complex process that even the best of its practitioners write code strewn with bugs.
- aidenn0 6y ago> It's fundamentally impossible given that you won't control everything about the operating environment of the software... Why not? Given the cost of writing secure software, for all but the highest volumes, you can ship it as a dedicated app on its own hardware. > For example, if you'd written the perfect web browser a few years back, how would you prevent vulnerability to CPU-level attacks such as Spectre and Meltdown discovered later on? Given that we are talking about liability here. Presumably Spectre and Meltdown would be primarily liabilities for hardware vendors. Software shipped after the vendors recommend software mitigations would shift that, but I don't think > Or consider if you write the perfect wireless networking firmware, but then end up being subject to vulnerabilities in the protocol standards (WEP was a good example of this)? First we must put aside the fact that WEP raised red-flags for cryptographers far before an exploit was found, and the fact that those concerns were dismissed with a hand wavy "People can tap into Ethernet lines too, this is just 'Wired Equivalent Privacy'" Now, I think that if you write software that faithfully implements a protocol, advertise it as implementing that protocol, and it is exploited not in-spite-of but because it did so, what exactly are you liable for? Even knowing that WEP, WPA, WPA2 and WPA3 are all flawed protocols, I'm sure there will be a WPA4 some day that is also flawed. If the only thing we had to worry about in WiFi stacks were protocol flaws, attackers would have a much harder time of things. No flaw in the WEP protocol allowed gaining execution access on the WiFi adapter, much less the host computer (though given the bus designs of the time, the former inevitably led to the latter).
- tshaddox 6y agoNo one knows how to make a perfectly safe car, yet there are some car safety problems for which car manufacturers are liable and others for which they are not. And those things can change over time, particularly as we learn about new manufacturing techniques and features which can improve safety! I see no reason why the same can't be applied to software. It's probably true that we can't feasibly create software and computer systems that are completely free from security vulnerabilities, but we absolutely have discovered some vulnerabilities which can be protected against using certain software development techniques, and I see no reason why software vendors ought not be liable for damages caused by their failure to follow these established techniques. Just like with cars, there is always going to be some tradeoffs between cost and security/safety, and exactly where to draw the line is something reasonable people can debate.
- pvg 6y agoThis sort of analogy, just like its popular variants (e.g. 'how come we can build bridges that don't collapse but windows crashes', etc) falls apart under very superficial scrutiny. For one thing, the failure semantics and modes of physical things are vastly simpler. It's not really a fruitful approach to this discussion.
- tshaddox 6y agoThe point of an analogy is to point out that certain aspects of two things are comparable in a certain context. It's true that there are significant differences between cars and software, like, say, the presence of wheels, but those aren't relevant to the specific point I was making. What is it exactly that makes the development of regulation of car safety so different from the development and (potential) regulation of software security?
- pvg 6y agoIt's not the presence of wheels, it's the difficulty of even defining what software operating 'correctly' means. This is a fairly well-understood aspect of software in general, not just the security part - hence everything from the important (e.g. halting problem) to the silly ('software will eat the world'). Software represents models we make up and on top of that, ones we interpret. Cars are not like that. You don't say your car made a mistake when it refuses to start one cold morning. There's essentially universal agreement on the 'semantics' of cars and bridges.
- creamytaco 6y ago> Since nobody knows how to reliably ship secure commercial software Secure software lies on a spectrum. We know memory unsafe languages are dangerous yet critical software is still being written in them. Liability would drastically change the domain in terms of discarding practices we've known for decades are deadends, and raise the bar significantly.
- wglb 6y agoYet the SolarWinds had nothing to do with memory unsafety. Nor did it for the equifax hack.
- coldtea 6y ago>Since nobody knows how to reliably ship secure commercial software, liability will mostly have the effect of making it difficult to start new software businesses. Doesn't sound so bad of a tradeoff, enforced for certain classes of software. Like one expects liability for medical, banking, flight control, etc. software, it could be extended to OSes, server programs, and anything handling user accounts... Software with less impact if it's not secure, like e.g. a media player or a text editor, could continue to ship free from liability.
- quasirandom 6y ago> Since nobody knows how to reliably ship secure commercial software, liability will mostly have the effect of making it difficult to start new software businesses. I think you would agree that having critical string parsing logic written in C and shipped in an opaque binary is both negligent and more dangerous than a reasonable person would expect. Traditional products liability claims focus on those kinds of questions: was there negligent conduct? is the product more dangerous than a reasonable person would expect? Attaching some kind of liability is a good way to encourage the industry to settle on standards of non-negligent conduct. If you provide vendors an opt-out mechanism (e.g. disclose your source and there's no liability), it will not dramatically interfere with the cadence of development. Common law is very well suited to establishing standards in a dynamic environment. It got us through the first industrial revolution.
- google234123 6y agoIf MSFT had to pay the cost of vulnerabilities then Windows would cost a lot more. Also, who would pay for Linux vulnerabilities? Would anyone contribute to open source if they were liable for any damage caused by their contribution?
- colejohnson66 6y agoThat’s a good point about Linux. The GPL boilerplate already includes: > This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. The author including that line is saying “despite what you think this software may do (creating implied warranties and the like), it doesn’t matter. I can’t be held liable for your usage of this code/software.” Would forced liability mean open source licenses like the GPL wouldn’t be able to have those sentences mean anything?
- joe_the_user 6y agoYou need a way to generally enforce an appropriate effort to security. This seems extremely important. I don't see liability, after-the-fact-punishment, only with software vendors, being the way to achieve this. Numerous threads on HN lately have discussed the way that "connect everything to everything" approaches wind-up inherently insecure and a wide range of organizations have no incentive to stop them. Water systems have no need to connect to the Internet even indirectly and neither automobile remote start system but organization managing these things have no incentive to take this stuff seriously. This is what's called an "externality" and a difficult one. Externalities are generally best dealt with be direct regulation but security is a bit different externalities because defining best practicing isn't. I could imagine a "security institute", only focused on defense and independent enough to be trustworthy to most institutions. But I'm not all that optimistic such a thing could be created.
- DennisP 6y agoIt would also help if the NSA put a priority on helping vendors close security holes, over exploiting those holes.
- snaxxx 6y agoThey already do this to a certain extent. Sometimes it's after the vulnerability has already been exploited by them or a sister agency. But other times it's soon after discovery.
- Animats 6y agoAt one time the NSA did that. The way it worked at one point in the 1980s is that a vendor could submit an OS for evaluation. NSA employees would test it, and if they could break in, they'd tell the vendor how they got in. The vendor got a chance to resubmit the system. One chance. If it failed on round 2, it was rejected. Vendors hated it. I've seen a vendor group document asking NSA not to publicly certify systems at the highest levels because it make them look bad. They got DoD to go for a system where the testing was outsourced to private labs, and you could keep going back for a retest over and over again until you "passed". Amusingly, computer security in those days wasn't a high priority at NSA for a bureaucratic reason. Computer security was at FANX, not NSA HQ at Fort Meade. FANX is the "Friendship Annex", a few miles from NSA HQ, near what used to be called Friendship Airport and is now BWI. FANX had all the minor functions of NSA - personnel, procurement, training, etc., and was considered a dumping ground for losers. Putting computer security out there doomed it. What really killed security is that from 1940 to 1980, DoD was the major consumer of advanced computer and electronics technology. By 1990, the commercial sector was dominant. Here are the early security standards books.[1] [1] https://i.ibb.co/CJTTzDL/ivorytowerspeedtest.png https://i.ibb.co/CJTTzDL/ivorytowerspeedtest.png
- dockd 6y agoIf we make the software vendor responsible, who pays for open source vulnerabilities?
- no-dr-onboard 6y ago>"If Microsoft had to pay the costs of vulnerabilities" Yeah, ok but lets trade out MSFT for FOSS, Firefox or small iOS app developers. At some point, we should ask to see if we are inadvertently advocating for scaring people into doing the right thing. Worse yet, for something that they couldn't see or anticipate. What about the standards makers? Are they going to be held accountable too? When is the last time you've read OWASP articles? Have you seen the state of that sort of literature? Its a mess. Have you seen how arcane and arbitrary the NIST-800 series is? What is the limit to this sort of "accountability"? At the risk of being even more trite, ideologically motivated statements like this look cool only because its on the internet.