7 ms·

Standards for Software Liability: Jim Dempsey, Lawfare, UC Berkeley Law

mustache_kimono 3y ago
Basically the argument, as I see it, the feds are demanding a liability regime, as software producers shouldn't be able to disclaim all liability for the software they sell, and the courts are better situated than regulatory agencies to decide liability questions. However, we don't have the time to let a software liability common law regime develop over decades, given the threat presented now. Therefore we should adopt the products liability, design defect analysis, which he explains as: In fact, it turns on two reasonableness inquiries: Was there a “reasonable” alternative design that the manufacturer could have used to avoid the vulnerability, and, without that alternative, was the actual design used “unreasonably dangerous”? Imagine asking the question -- are there reasonable alternative designs for OpenSSL? For C and C++? For Linux? Re: OpenSSL and C and C++ memory vulnerabilities, I'd say maybe, see: Rust and rustls! Re: Linux, I would have to say no. Is there an alternative which doesn't have the same issues? Not really. The author's summary liability regime, in light of the NIST guidance, would be as follows: 1. A rules-based approach would define a floor—the minimum legal standard of care for software—focused on specific product features or behaviors to be included or avoided. 2. However, a list of known coding weaknesses cannot suffice alone. Software is so complex and dynamic that a liability regime also needs to cover design flaws that are not so easily boiled down. For these, I propose a standard based on the defects analysis common to products liability law. 3. But this liability should not be unlimited or unpredictable. As the Biden administration’s National Cybersecurity Strategy recognizes, developers deserve a safe harbor that shields them from liability for hard-to-detect flaws above the floor. For that, I would turn to a set of robust coding practices.
- alilleybrinker 3y ago
  I think this reading is _mostly_ right. To make it a little clearer: 1. A rules-based approach to define a minimum standard, likely based around existing descriptions of software weaknesses where the rules basically say "you aren't allowed to have this type of software weakness" (the doc specifically gives path traversal as an example). HOWEVER, liability would only be established if a weakness is actually exploited, to help avoid the problem of people proactively scanning for weaknesses and then troll-suing as a way to make money. 2. A process-based safe harbor to avoid liability for exploitation of weaknesses NOT on the rules-based "no no list." Basically, if you can show you followed the safe harbor process requirements, you get the benefit of the safe harbor. This is to protect and encourage organizations to go above and beyond the floor established by the rules-based standard.
  - mustache_kimono 3y ago
    > HOWEVER, liability would only be established if a weakness is actually exploited, to help avoid the problem of people proactively scanning for weaknesses and then troll-suing as a way to make money. Agreed. Torts require damages.
    - alilleybrinker 3y ago
      Yeah, the doc acknowledges this concept is "drawing on tort law principles." In this case the author, Jim Dempsey, is proposing a rules-based framework for establishing a floor regardless of what underlying system is used (warranty, torts, etc.). So he's not _inherently_ assuming a tort law framing, and is instead making a recommendation for how it should operate regardless of whether the resulting system is based around tort law.
      - mustache_kimono 3y ago
        > So he's not _inherently_ assuming a tort law framing, and is instead making a recommendation for how it should operate regardless of whether the resulting system is based around tort law. Not to get too in the weeds, but breaches of implied warranties were once/began as torts. But, yes, I guess I was a little glib. I should have said -- it would be sensible to require damages for these types of actions. Although I'm not sure I agree with the author's formulation: > HOWEVER, liability would only be established if a weakness is actually exploited, to help avoid the problem of people proactively scanning for weaknesses and then troll-suing as a way to make money. I think we should encourage customers to look for vulnerabilities. Damages analyses can take into account depreciation, reasonableness of damages/troll suing, etc. If you have 100s of devices you must replace because the vendor won't patch, and your patch request is reasonable (high consequence vulnerability) within 5 years of purchase, I think damages accounting for depreciation make sense, without proof of exploitation. EDIT: Implied 2nd quoted portion was the parent commenter's notion when it was a summary of the author's position.
        alilleybrinker 3y ago
- gavinhoward 3y ago
  > Re: ...C and C++ memory vulnerabilities, I'd say maybe, see: Rust In general, yes. However, I hate Rust; my software wouldn't exist if I had to write it in Rust. [1] So I hope there would be a way for me to justify using C, along with a plan to minimize memory vulnerabilities. In my case, my plan is to fuzz, use every one of the fuzz paths found as a test case, and ensure all of those test cases come out clean in sanitizers and Valgrind. Oh, and I'll also do error injection, like SQLite [2], to exercise even more paths, and those paths will also have to come out clean. [1]: https://gavinhoward.com/2023/02/why-i-use-c-when-i-believe-in-memory-safety/ https://gavinhoward.com/2023/02/why-i-use-c-when-i-believe-i... [2]: https://www.sqlite.org/testing.html#anomaly_testing https://www.sqlite.org/testing.html#anomaly_testing
  - alilleybrinker 3y ago
    At the moment there's an effort from the Office and the National Cyber Director (ONCD), CISA, NSF, DARPA, and OMB to encourage a move to memory safe languages, including but not limited to Rust (see [1]). This included an RFI for which there were many responses, including a number of major tech companies (Microsoft, Google, others). The general consensus supports a move to memory safe languages, BUT many of the respondents also caution about the need to prioritize and carefully manage such a transition, and the need to continue to support and assure code written in memory unsafe languages using other assurance techniques like fuzzing, integration testing, unit testing, static code analysis, and more. Hopefully that helps explain where things stand at the moment regarding at least US movement on this topic, and also to assuage concerns about a unilateral mandate that things get rewritten in Rust (or another language). [1]: https://www.whitehouse.gov/oncd/briefing-room/2023/08/10/fact-sheet-office-of-the-national-cyber-director-requests-public-comment-on-open-source-software-security-and-memory-safe-programming-languages/ https://www.whitehouse.gov/oncd/briefing-room/2023/08/10/fac...
    - gavinhoward 3y ago
      I know. I also know that despite some having caution, there are those that would speed ahead with RIIR.
  - 3y ago
Turing_Machine 3y ago
I can't think of a better way to destroy FOSS, and to drive non-free software completely off-shore. P.S. where should the liability fall for: This site can’t provide a secure connection www.lawfaremedia.org sent an invalid response.
- mustache_kimono 3y ago
  > I can't think of a better way to destroy FOSS, and to drive non-free software completely off-shore. But products liability is for products? If you aren't selling anything how would you be liable? If you and I have no business relationship, what is the basis of liability? I don't see a liability regime touching internal use, personal use, or non-commercial (read: FOSS) development. However, when you decide to sell a service or a good, that's when liability attaches. > This site can’t provide a secure connection www.lawfaremedia.org sent an invalid response. Works for me?
  - nickff 3y ago
    You can be liable for harms caused by things you give away; you can even be liable for harm caused by things stolen from you (i.e. cars).
    - mustache_kimono 3y ago
      > You can be liable for harms caused by things you give away; you can even be liable for harm caused by things stolen from you (i.e. cars). Not in traditional tort law. A criminal stealing your car is an intervening event. You didn't cause the harm. But for the sake of argument let's say you find a precedent. Please explain how that liability would work in the context of products liability. First ask yourself the threshold question: What is a product? A product is not your FOSS on GitHub.
      - nickff 3y ago
        This law firm made a handy chart for determining owner liability for harms caused by a stolen vehicle: https://www.mwl-law.com/wp-content/uploads/2018/02/OWNER-LIABILITY-FOR-STOLEN-VEHICLE-CHART.pdf https://www.mwl-law.com/wp-content/uploads/2018/02/OWNER-LIA... >"Please explain how that liability would work in the context of products liability. First ask yourself the threshold question: What is a product? A product is not your FOSS on GitHub. " I think that a court could easily determine a FOSS contributor was liable if they knew of a way in which the use of their software could cause harm to a user or 'victim' of the software. I am not sure the word 'product' does any work here.
pylua 3y ago
This would be a great win for the lawyers. T The better alternative would to be to completely rework the software developing discipline to be as rigorous as civil engineering from the top down. Requiring proper licensing, and security clearances for accessing U.S. customer data.
- bee_rider 3y ago
  Software is tricky in this regard because of the dependency mess. If you throw your “little garden path” program up on GitHub and somebody decides to use it to build their “highway,” it really ought to be on them, not you. Edit: I suspect any reasonable law would account for that, and these are the folks who do the Lawfare podcast, right? It is pretty good. I haven’t finished the paper yet but I’d be surprised if they missed this.
  - carols10cents 3y ago
    > these are the folks who do the Lawfare podcast, right? Yep, and they had a podcast episode with the author of this paper: https://www.lawfaremedia.org/article/the-lawfare-podcast-jim-dempsey-on-standards-for-software-liability https://www.lawfaremedia.org/article/the-lawfare-podcast-jim...
  - alilleybrinker 3y ago
    Cool thing to note: this argument was made a lot in responses to the Office of the National Cyber Directors' recent Request for Input on improving open source software security. That said, there's a state of tension on this topic right now. The European Union's draft Cyber Resilience Act has included language across multiple versions that would in at least some cases assign liability to producers of open source software. They've tried to modify the language to exclude non-commercial open source projects, but there's been a lot of wrangling over the exact definition of "commercial."
    - bee_rider 3y ago
      Honestly, there should be tension and arguing there. For example, Google should (IMO) very clearly be treated as selling products in the cases of Android and Chrome despite the fact that these technically are related to open source projects. I’m quite glad they aren’t ignoring the complexity.
gavinhoward 3y ago
He wants a standard of care, and I have one. [1] He wants a liability regime, and I have one. [2] Yes, we need to start accepting liability for our software, and as he says, it's better to get one now before one is thrust upon us. Bonus points if it funds FOSS. [1]: https://gavinhoward.com/2022/10/we-must-professionalize-programming-to-preserve-society-and-computing-freedom/#the-professional-programmers-standard-of-care https://gavinhoward.com/2022/10/we-must-professionalize-prog... [2]: https://gavinhoward.com/2023/11/how-to-fund-foss-save-it-from-the-cra-and-improve-cybersecurity/ https://gavinhoward.com/2023/11/how-to-fund-foss-save-it-fro...
- deleted 3y ago
  [deleted]
zer00eyz 3y ago
The elephant in the room: If software becomes a product does that mean it's no longer speech? How do you have liability for something that is copyright? What about the first amendment in the us?
- AnimalMuppet 3y ago
  IANAL. So this isn't a leading question; I really don't know. What's the standard for liability for something like a manual on how to repair your own car brakes, where if you mess it up you might get killed? How about medical advice online, where if you follow it, it might kill you? How about a manual on how to prepare fugu? How about if I give what I think is good psychological advice, and they commit suicide? If I write any of the above, and it's wrong, and someone dies because it was wrong, how liable am I under current law? How about if I write something that someone takes as investment advice, and they follow it and lose a lot of money? How about if I write something that looks like legal advice, and they follow it and end up in jail? What is the actual current state of the law on these things in the US?
  - beaeglebeached 3y ago
    My prediction is non commercial software becomes anonymous, and commercial software gets sold from some other low tax professional-dense location like Dubai or Singapore. If you look at 3d printed gun plans and code, 1A has arguably helped with keeping it on the net but the authors are mostly under psuedonyms.
  - zer00eyz 3y ago
    IANAL ... but I like reading the law, I like that the law is protectionist in spite of the founding fathers wishes (a digression) https://en.wikipedia.org/wiki/United_States_free_speech_exceptions https://en.wikipedia.org/wiki/United_States_free_speech_exce... Intent matters in the case of the first amendment. Or you limit how you can license software, that's tort, but someone will argue that your limiting what I can do with my speech at that point in time... This change looks like the "Nuclear option" for something that is, fairly common in society.
  - 3y ago

Basically the argument, as I see it, the feds are demanding a liability regime, as software producers shouldn't be able to disclaim all liability for the software they sell, and the courts are better situated than regulatory agencies to decide liability questions. However, we don't have the time to let a software liability common law regime develop over decades, given the threat presented now. Therefore we should adopt the products liability, design defect analysis, which he explains as: In fact, it turns on two reasonableness inquiries: Was there a “reasonable” alternative design that the manufacturer could have used to avoid the vulnerability, and, without that alternative, was the actual design used “unreasonably dangerous”? Imagine asking the question -- are there reasonable alternative designs for OpenSSL? For C and C++? For Linux? Re: OpenSSL and C and C++ memory vulnerabilities, I'd say maybe, see: Rust and rustls! Re: Linux, I would have to say no. Is there an alternative which doesn't have the same issues? Not really. The author's summary liability regime, in light of the NIST guidance, would be as follows: 1. A rules-based approach would define a floor—the minimum legal standard of care for software—focused on specific product features or behaviors to be included or avoided. 2. However, a list of known coding weaknesses cannot suffice alone. Software is so complex and dynamic that a liability regime also needs to cover design flaws that are not so easily boiled down. For these, I propose a standard based on the defects analysis common to products liability law. 3. But this liability should not be unlimited or unpredictable. As the Biden administration’s National Cybersecurity Strategy recognizes, developers deserve a safe harbor that shields them from liability for hard-to-detect flaws above the floor. For that, I would turn to a set of robust coding practices.

alilleybrinker 3y ago
I think this reading is _mostly_ right. To make it a little clearer: 1. A rules-based approach to define a minimum standard, likely based around existing descriptions of software weaknesses where the rules basically say "you aren't allowed to have this type of software weakness" (the doc specifically gives path traversal as an example). HOWEVER, liability would only be established if a weakness is actually exploited, to help avoid the problem of people proactively scanning for weaknesses and then troll-suing as a way to make money. 2. A process-based safe harbor to avoid liability for exploitation of weaknesses NOT on the rules-based "no no list." Basically, if you can show you followed the safe harbor process requirements, you get the benefit of the safe harbor. This is to protect and encourage organizations to go above and beyond the floor established by the rules-based standard.
- mustache_kimono 3y ago
  > HOWEVER, liability would only be established if a weakness is actually exploited, to help avoid the problem of people proactively scanning for weaknesses and then troll-suing as a way to make money. Agreed. Torts require damages.
  - alilleybrinker 3y ago
    Yeah, the doc acknowledges this concept is "drawing on tort law principles." In this case the author, Jim Dempsey, is proposing a rules-based framework for establishing a floor regardless of what underlying system is used (warranty, torts, etc.). So he's not _inherently_ assuming a tort law framing, and is instead making a recommendation for how it should operate regardless of whether the resulting system is based around tort law.
    - mustache_kimono 3y ago
      > So he's not _inherently_ assuming a tort law framing, and is instead making a recommendation for how it should operate regardless of whether the resulting system is based around tort law. Not to get too in the weeds, but breaches of implied warranties were once/began as torts. But, yes, I guess I was a little glib. I should have said -- it would be sensible to require damages for these types of actions. Although I'm not sure I agree with the author's formulation: > HOWEVER, liability would only be established if a weakness is actually exploited, to help avoid the problem of people proactively scanning for weaknesses and then troll-suing as a way to make money. I think we should encourage customers to look for vulnerabilities. Damages analyses can take into account depreciation, reasonableness of damages/troll suing, etc. If you have 100s of devices you must replace because the vendor won't patch, and your patch request is reasonable (high consequence vulnerability) within 5 years of purchase, I think damages accounting for depreciation make sense, without proof of exploitation. EDIT: Implied 2nd quoted portion was the parent commenter's notion when it was a summary of the author's position.
      - alilleybrinker 3y ago
gavinhoward 3y ago
> Re: ...C and C++ memory vulnerabilities, I'd say maybe, see: Rust In general, yes. However, I hate Rust; my software wouldn't exist if I had to write it in Rust. [1] So I hope there would be a way for me to justify using C, along with a plan to minimize memory vulnerabilities. In my case, my plan is to fuzz, use every one of the fuzz paths found as a test case, and ensure all of those test cases come out clean in sanitizers and Valgrind. Oh, and I'll also do error injection, like SQLite [2], to exercise even more paths, and those paths will also have to come out clean. [1]: https://gavinhoward.com/2023/02/why-i-use-c-when-i-believe-in-memory-safety/ https://gavinhoward.com/2023/02/why-i-use-c-when-i-believe-i... [2]: https://www.sqlite.org/testing.html#anomaly_testing https://www.sqlite.org/testing.html#anomaly_testing
- alilleybrinker 3y ago
  At the moment there's an effort from the Office and the National Cyber Director (ONCD), CISA, NSF, DARPA, and OMB to encourage a move to memory safe languages, including but not limited to Rust (see [1]). This included an RFI for which there were many responses, including a number of major tech companies (Microsoft, Google, others). The general consensus supports a move to memory safe languages, BUT many of the respondents also caution about the need to prioritize and carefully manage such a transition, and the need to continue to support and assure code written in memory unsafe languages using other assurance techniques like fuzzing, integration testing, unit testing, static code analysis, and more. Hopefully that helps explain where things stand at the moment regarding at least US movement on this topic, and also to assuage concerns about a unilateral mandate that things get rewritten in Rust (or another language). [1]: https://www.whitehouse.gov/oncd/briefing-room/2023/08/10/fact-sheet-office-of-the-national-cyber-director-requests-public-comment-on-open-source-software-security-and-memory-safe-programming-languages/ https://www.whitehouse.gov/oncd/briefing-room/2023/08/10/fac...
  - gavinhoward 3y ago
    I know. I also know that despite some having caution, there are those that would speed ahead with RIIR.
- 3y ago

Turing_Machine 3y ago

I can't think of a better way to destroy FOSS, and to drive non-free software completely off-shore. P.S. where should the liability fall for: This site can’t provide a secure connection www.lawfaremedia.org sent an invalid response.

mustache_kimono 3y ago
> I can't think of a better way to destroy FOSS, and to drive non-free software completely off-shore. But products liability is for products? If you aren't selling anything how would you be liable? If you and I have no business relationship, what is the basis of liability? I don't see a liability regime touching internal use, personal use, or non-commercial (read: FOSS) development. However, when you decide to sell a service or a good, that's when liability attaches. > This site can’t provide a secure connection www.lawfaremedia.org sent an invalid response. Works for me?
- nickff 3y ago
  You can be liable for harms caused by things you give away; you can even be liable for harm caused by things stolen from you (i.e. cars).
  - mustache_kimono 3y ago
    > You can be liable for harms caused by things you give away; you can even be liable for harm caused by things stolen from you (i.e. cars). Not in traditional tort law. A criminal stealing your car is an intervening event. You didn't cause the harm. But for the sake of argument let's say you find a precedent. Please explain how that liability would work in the context of products liability. First ask yourself the threshold question: What is a product? A product is not your FOSS on GitHub.
    - nickff 3y ago
      This law firm made a handy chart for determining owner liability for harms caused by a stolen vehicle: https://www.mwl-law.com/wp-content/uploads/2018/02/OWNER-LIABILITY-FOR-STOLEN-VEHICLE-CHART.pdf https://www.mwl-law.com/wp-content/uploads/2018/02/OWNER-LIA... >"Please explain how that liability would work in the context of products liability. First ask yourself the threshold question: What is a product? A product is not your FOSS on GitHub. " I think that a court could easily determine a FOSS contributor was liable if they knew of a way in which the use of their software could cause harm to a user or 'victim' of the software. I am not sure the word 'product' does any work here.

pylua 3y ago

This would be a great win for the lawyers. T The better alternative would to be to completely rework the software developing discipline to be as rigorous as civil engineering from the top down. Requiring proper licensing, and security clearances for accessing U.S. customer data.

bee_rider 3y ago
Software is tricky in this regard because of the dependency mess. If you throw your “little garden path” program up on GitHub and somebody decides to use it to build their “highway,” it really ought to be on them, not you. Edit: I suspect any reasonable law would account for that, and these are the folks who do the Lawfare podcast, right? It is pretty good. I haven’t finished the paper yet but I’d be surprised if they missed this.
- carols10cents 3y ago
  > these are the folks who do the Lawfare podcast, right? Yep, and they had a podcast episode with the author of this paper: https://www.lawfaremedia.org/article/the-lawfare-podcast-jim-dempsey-on-standards-for-software-liability https://www.lawfaremedia.org/article/the-lawfare-podcast-jim...
- alilleybrinker 3y ago
  Cool thing to note: this argument was made a lot in responses to the Office of the National Cyber Directors' recent Request for Input on improving open source software security. That said, there's a state of tension on this topic right now. The European Union's draft Cyber Resilience Act has included language across multiple versions that would in at least some cases assign liability to producers of open source software. They've tried to modify the language to exclude non-commercial open source projects, but there's been a lot of wrangling over the exact definition of "commercial."
  - bee_rider 3y ago
    Honestly, there should be tension and arguing there. For example, Google should (IMO) very clearly be treated as selling products in the cases of Android and Chrome despite the fact that these technically are related to open source projects. I’m quite glad they aren’t ignoring the complexity.

gavinhoward 3y ago

He wants a standard of care, and I have one. [1] He wants a liability regime, and I have one. [2] Yes, we need to start accepting liability for our software, and as he says, it's better to get one now before one is thrust upon us. Bonus points if it funds FOSS. [1]: https://gavinhoward.com/2022/10/we-must-professionalize-programming-to-preserve-society-and-computing-freedom/#the-professional-programmers-standard-of-care https://gavinhoward.com/2022/10/we-must-professionalize-prog... [2]: https://gavinhoward.com/2023/11/how-to-fund-foss-save-it-from-the-cra-and-improve-cybersecurity/ https://gavinhoward.com/2023/11/how-to-fund-foss-save-it-fro...

deleted 3y ago
[deleted]

zer00eyz 3y ago

The elephant in the room: If software becomes a product does that mean it's no longer speech? How do you have liability for something that is copyright? What about the first amendment in the us?

AnimalMuppet 3y ago
IANAL. So this isn't a leading question; I really don't know. What's the standard for liability for something like a manual on how to repair your own car brakes, where if you mess it up you might get killed? How about medical advice online, where if you follow it, it might kill you? How about a manual on how to prepare fugu? How about if I give what I think is good psychological advice, and they commit suicide? If I write any of the above, and it's wrong, and someone dies because it was wrong, how liable am I under current law? How about if I write something that someone takes as investment advice, and they follow it and lose a lot of money? How about if I write something that looks like legal advice, and they follow it and end up in jail? What is the actual current state of the law on these things in the US?
- beaeglebeached 3y ago
  My prediction is non commercial software becomes anonymous, and commercial software gets sold from some other low tax professional-dense location like Dubai or Singapore. If you look at 3d printed gun plans and code, 1A has arguably helped with keeping it on the net but the authors are mostly under psuedonyms.
- zer00eyz 3y ago
  IANAL ... but I like reading the law, I like that the law is protectionist in spite of the founding fathers wishes (a digression) https://en.wikipedia.org/wiki/United_States_free_speech_exceptions https://en.wikipedia.org/wiki/United_States_free_speech_exce... Intent matters in the case of the first amendment. Or you limit how you can license software, that's tort, but someone will argue that your limiting what I can do with my speech at that point in time... This change looks like the "Nuclear option" for something that is, fairly common in society.
- 3y ago