3 ms·

You said: “If a single simple mistake has the potential to cause devastating liability” Liability is in proportion to actual harm. A single simple mistake can

by Veserv 2y ago

You said: “If a single simple mistake has the potential to cause devastating liability” Liability is in proportion to actual harm. A single simple mistake can only incur devastating liability if it causes devastating harm (as in harm actually occurred). If it can not and does not cause devastating harm, then you will not incur devastating liability and you are already protected. Again, you are applying some sort of bizarre uniform standard for defect remediation that ignores all nuance around intended use, intended users, usage context, harm, potential for harm, etc. That is not how anything works. Toys have different standards than planes which have different standards than bridges. The world is not unary. As for the questions of cars you posed. Those fall into slightly more nuanced principles. One of those principles is that liability is in proportion to/limited by guarantees either explicit or implicit. The limitations of the physical security of car windows are clearly communicated, obvious, and well-understood to the average consumer. A company guaranteeing that their windows are immune to being broken would be taking on the liability for their guarantee that is in excess of normal expectations. Software security and reliability is not just not clearly communicated, it is deceptively communicated. Do you believe that the average Crowdstrike customer believed that Crowdstrike could take out their business? Or were Crowdstrike’s sales and marketing continuously assuring their customers that Crowdstrike will work flawlessly despite the fine print disclaiming all liability? If the customer actually believed it could or would fuck up royally, then sure, no liability. Customer’s fault for using a product they believed to be a piece of shit. But would Crowdstrike have ever sold a single seat if their customer’s believed that? Fuck no. They made a implicit and likely explicit guarantee of operability and they are liable for it. Liability around dangerous operation is again, about clear understanding of operation guarantees. It is obvious to the most casual observer the dangers of a chainsaw and how one might be safely used. It is the job of the manufacturer to make such boundaries clear, to mitigate unknowing unsafe operation, and to mitigate unsafe operation where feasible. To go back to Crowdstrike, you can not contend that it was used “dangerously”. The defect was in failure to perform correctly, not inappropriate usage. Dangerous usage might be enabling a feature that automatically deletes any program that starts sending on the network, but you forgot that your critical server starts sending notifications to your team when under heavy load. That is likely your damages to bear. But if it was actually a feature that deletes any “anomalous program”, then that depends on the guarantees provided. Making careful tradeoffs about the distribution of damages and liability is a well-worn path. Throwing your hands up in the air because your problems are harder and more dangerous so you should be held to lower standards is absurd. And please, you make it sound like software is built to some high standard comparable to your average physical product and that it is unfair to hold it to higher standards. You and I both know software is generally held to exactly no standard. Demanding it be held to even normal standards is not some sort of unfair imposition. It should probably be held to higher standards due to the risks of correlated failures due to network connectivity as you point out, but even just getting to normal standards would be a massive and qualitative improvement.

thayne 2y ago
> You said: “If a single simple mistake has the potential to cause devastating liability” I think you are misinterpreting my statement. I meant devastating to the software creator. Which is not necessarily the same thing as devastating to the victim of an attack. > A single simple mistake can only incur devastating liability if it causes devastating harm (as in harm actually occurred) This is absolutely not true. Even if you ignore any possible inequities that exist in the justice system (like say that civil courts tend to favor the party that spends more on lawyers), the same absolute amount in dollars has a very different impact on different parties. For example suppose a single developer or early bootstrapped startup makes a library or application (possibly open source), and a big bank uses that library then a vulnerability in it is used as part of an attack that causes a few million in damages. That few million would be a relatively small amount to the big bank, but would likely bankrupt an individual or small startup, and would thus be "devastating". > Again, you are applying some sort of bizarre uniform standard for defect remediation that ignores all nuance around intended use, intended users, usage context, harm, potential for harm, etc. I don't know why you think that. > One of those principles is that liability is in proportion to/limited by guarantees either explicit or implicit. The limitations of the physical security of car windows are clearly communicated, obvious, and well-understood to the average consumer. A company guaranteeing that their windows are immune to being broken would be taking on the liability for their guarantee that is in excess of normal expectations. Right. And most software doesn't come with any garantee that it is bug free, or even free of security bugs. Afaik, software doesn't currently have some kind of exemption from liability. But it is very common to push as much liability on the user of the software as possible with terms of service or licensing terms. In fact almost all open source licenses have disclaimers saying that the copyright holder makes no garantees about the quality of the software. > Software security and reliability is not just not clearly communicated, it is deceptively communicated. That's a broad generalization, but I agree that there are many cases where that is true. But if what sales and marketing says differs from the actual terms of the contract or ToS, wouldn't that be more a problem of false advertising? And security is an imprecise and relative term. And there are cases where software contracts do have garantees of reliability, and sometimes security, and promise to compensate damages if those garantees aren't met. But you usually have to be willing to pay a lot of money for an enterprise contract to get that. Could this situation be improved? Absolutely! I think if your ToS push liability onto the custemer, at the very least that should be made much more clear to the customer (and the same for many other things hidden in the fine print), and then maybe market forces would push more companies to make stronger garantees to win customers. But that problem is hardly unique to software. Lots of companies hide stuff like "you take full responsibility, and agree not to sue us" in their fine print. > Do you believe that the average Crowdstrike customer believed that Crowdstrike could take out their business? The crowdstrike issue is unrelated. As you admiited later, there was no malicious actor involved. And there were significant problems with their deployment process. I absolutely do think they should be held liable. But that is very different from something like, say, a buffer overflow due to a simple mistake that slipped through rigorous review, testing, fuzzing and maybe even a pen test. > you can not contend that it was used “dangerously”. I don't. > Throwing your hands up in the air because your problems are harder and more dangerous so you should be held to lower standards is absurd. That isn't at all what I am saying. I'm saying that developers shouldn't be held responsible for the actions of criminals just because those criminals used an unknown weakness of the software to commit the crime. Doing so is holding software up to a higher standard than most other products. Now just as with physical products, I think there are exceptions, like if your product is sold or marketed specifically to prevent specific actions of criminals, and fails to do so. Ironically, blaming developers cybercrime is throwing up your hands because your problems are hard. Specifically, stopping cybercrime with law enforcement is extremely difficult, in part because the criminals are often beyond the jurisdiction the applicable law enforcement. But maybe putting some government funding towards increasing security of critical components, especially open source ones, or initiatives to rewrite those components in memory safe languages, would be more effective than pointing fingers? > You and I both know software is generally held to exactly no standard. Um, just like physical products the standards for software varies widely. What I am opposed to is applying some kind of blanket liability to all software products. Because a game shouldn't be held to the same standard as an enterprise firewall, or anti-malware solution. And there absolutely are standards and certifications for security and reliability in software. ISO 27001, SOC 2, PCI, FedRAMP, HIPAA, just to name a few. And if you sell to certain organizations, like governments, financial institutions, health care providers, etc. you will need one or more of those.

by Veserv 2y ago

thayne 2y ago

> You said: “If a single simple mistake has the potential to cause devastating liability” I think you are misinterpreting my statement. I meant devastating to the software creator. Which is not necessarily the same thing as devastating to the victim of an attack. > A single simple mistake can only incur devastating liability if it causes devastating harm (as in harm actually occurred) This is absolutely not true. Even if you ignore any possible inequities that exist in the justice system (like say that civil courts tend to favor the party that spends more on lawyers), the same absolute amount in dollars has a very different impact on different parties. For example suppose a single developer or early bootstrapped startup makes a library or application (possibly open source), and a big bank uses that library then a vulnerability in it is used as part of an attack that causes a few million in damages. That few million would be a relatively small amount to the big bank, but would likely bankrupt an individual or small startup, and would thus be "devastating". > Again, you are applying some sort of bizarre uniform standard for defect remediation that ignores all nuance around intended use, intended users, usage context, harm, potential for harm, etc. I don't know why you think that. > One of those principles is that liability is in proportion to/limited by guarantees either explicit or implicit. The limitations of the physical security of car windows are clearly communicated, obvious, and well-understood to the average consumer. A company guaranteeing that their windows are immune to being broken would be taking on the liability for their guarantee that is in excess of normal expectations. Right. And most software doesn't come with any garantee that it is bug free, or even free of security bugs. Afaik, software doesn't currently have some kind of exemption from liability. But it is very common to push as much liability on the user of the software as possible with terms of service or licensing terms. In fact almost all open source licenses have disclaimers saying that the copyright holder makes no garantees about the quality of the software. > Software security and reliability is not just not clearly communicated, it is deceptively communicated. That's a broad generalization, but I agree that there are many cases where that is true. But if what sales and marketing says differs from the actual terms of the contract or ToS, wouldn't that be more a problem of false advertising? And security is an imprecise and relative term. And there are cases where software contracts do have garantees of reliability, and sometimes security, and promise to compensate damages if those garantees aren't met. But you usually have to be willing to pay a lot of money for an enterprise contract to get that. Could this situation be improved? Absolutely! I think if your ToS push liability onto the custemer, at the very least that should be made much more clear to the customer (and the same for many other things hidden in the fine print), and then maybe market forces would push more companies to make stronger garantees to win customers. But that problem is hardly unique to software. Lots of companies hide stuff like "you take full responsibility, and agree not to sue us" in their fine print. > Do you believe that the average Crowdstrike customer believed that Crowdstrike could take out their business? The crowdstrike issue is unrelated. As you admiited later, there was no malicious actor involved. And there were significant problems with their deployment process. I absolutely do think they should be held liable. But that is very different from something like, say, a buffer overflow due to a simple mistake that slipped through rigorous review, testing, fuzzing and maybe even a pen test. > you can not contend that it was used “dangerously”. I don't. > Throwing your hands up in the air because your problems are harder and more dangerous so you should be held to lower standards is absurd. That isn't at all what I am saying. I'm saying that developers shouldn't be held responsible for the actions of criminals just because those criminals used an unknown weakness of the software to commit the crime. Doing so is holding software up to a higher standard than most other products. Now just as with physical products, I think there are exceptions, like if your product is sold or marketed specifically to prevent specific actions of criminals, and fails to do so. Ironically, blaming developers cybercrime is throwing up your hands because your problems are hard. Specifically, stopping cybercrime with law enforcement is extremely difficult, in part because the criminals are often beyond the jurisdiction the applicable law enforcement. But maybe putting some government funding towards increasing security of critical components, especially open source ones, or initiatives to rewrite those components in memory safe languages, would be more effective than pointing fingers? > You and I both know software is generally held to exactly no standard. Um, just like physical products the standards for software varies widely. What I am opposed to is applying some kind of blanket liability to all software products. Because a game shouldn't be held to the same standard as an enterprise firewall, or anti-malware solution. And there absolutely are standards and certifications for security and reliability in software. ISO 27001, SOC 2, PCI, FedRAMP, HIPAA, just to name a few. And if you sell to certain organizations, like governments, financial institutions, health care providers, etc. you will need one or more of those.