4 ms·

Disclosing zero-days so the vendor can patch them and declare "mission accomplished" is such a waste. "Penetrate and Patch" is about as effective for software

by Veserv 2y ago

Disclosing zero-days so the vendor can patch them and declare "mission accomplished" is such a waste. "Penetrate and Patch" is about as effective for software security as it is for bulletproof vests. If you randomly select 10 bulletproof vests for testing, shoot each 10 times and get 10 holes each, you do not patch those holes and call it good. What you learned from your verification process is that the process that lead to that bulletproof vest is incapable of consistently delivering products that meet the requirements. Only development process changes that result in passing new verification tests give any confidence of adequacy. Absent actively, or likely actively, exploited vulnerabilitys, the government should organize vulnerabilitys by "difficulty" and announce the presence of, but not disclose the precise nature of, vulnerabilitys and demand process improvement until vulnerabilitys of that "difficulty" are not longer present as indicated by fixing all "known, but undiclosed" vulnerabilitys of that "difficulty". Only that provides initial supporting evidence that the process has improved enough to categorically prevent vulnerabilitys of that "difficulty". Anything less is just papering over defective products on the government's dime.

JumpCrisscross 2y ago
> the government should organize vulnerabilitys by "difficulty" and announce the presence of, but not disclose the precise nature of, vulnerabilitys and demand process improvement until vulnerabilitys of that "difficulty" are not longer present as indicated by fixing all "known, but undiclosed" vulnerabilitys of that "difficulty" For this amount of bureaucracy, the government should just hire all coders and write all software.
- Veserv 2y ago
  You appear to misunderstand what I am saying: 1) Government already has vulnerabilitys. 2) Government identifies vulnerabilitys they already own by "difficulty to discovery". 3) Government selects the lowest "difficulty to discover" vulnerabilitys they already own. 4) Government announces products with known "lowest difficulty to discover" vulnerabilitys are vulnerable, but does not disclose them. 5) Government keeps announcing those products continue to be the most "insecure" until all vulnerabilitys they already own at that level are fixed. 6) Repeat.
  - JumpCrisscross 2y ago
    What you're suggesting requires creating a massive federal bureaucracy to continuously survey the product landscape. It then requires the private sector to duplicate that work. This is stupid.
    - Veserv 2y ago
      I have no idea how you came to that conclusion. The government has vulnerabilitys in stock that they already verify function as intended. They already regularly verify these vulnerabilitys continue to function with each update cycle. They already quantify these vulnerabilitys by ease-of-discovery, impact, etc. so they can prioritize utilization. They already determine vulnerabilitys to disclose. Assuming that the vulnerabilitys they disclose are not entirely "actively under exploit", the only difference in my proposed policy is that they do not disclose the details to the vendors so they can paper over them. Instead, they publicly announce the presence of vulnerabilitys and then keep verifying as they already do until the vulnerabilitys no longer function. You seem to think I am arguing that the government should create a new organization to look for vulnerabilitys in all software everywhere and then act as I stated.
      - JumpCrisscross 2y ago
        > they publicly announce the presence of vulnerabilitys and then keep verifying as they already do until the vulnerabilitys no longer function Yes. This is difficult. Particularly given you need to do it fairly and thus comprehensively.
        Veserv 2y ago
        Then you are arguing with a strawman. I never proposed they do it across all products, only the products that they already have vulnerabilitys in that they already seek to disclose. You can not change the parameters of my proposal to explicitly require a gigantic bureaucracy then argue that it is a poor idea because the gigantic bureaucracy you added is a problem. You could have argued that my proposal is untenable because it would be "unfair" and the only way to make it fair would be to do it comprehensively which would be too hard. To which I would state: 1) That means we as a society care more about being "fair" to companies with inadequate software security than demanding adequate software security. Could be the case, but then everything other than roll over and accept it is off-the-table. 2) The government already requires certification against the Common Criteria for many software products in use by the government. You could restrict this policy to just systems that are used and require certification before use. Thus being applied "fairly" to government procurement and incentivizing improvements in security for procured systems. 3) This should actually just be general policy for everybody, but only the government, currently, has enough leverage to really pull off publicly announcing a problem and the vendor not being able to just shove it under the rug. And, even if you disagree with those points, I am also making the point that the current policy of disclosing the vulnerability so the vendor can make a point-patch to resolve it instead of fixing their overall security process is a failed security policy at the societal level. We need mechanisms to encourage overall security process improvement. Currently, the only thing that does that is the exponentially increasing amount and severity of hacks, and it would be nice to get ahead of it instead of being purely reactionary.
tptacek 2y ago
"Penetrate and patch" is a term of art Marcus J. Ranum tried to popularize 15-20 years ago, as part of an effort to vilify independent security research. Ranum was part of an older iteration of software security that was driven by vendor-sponsored cliques. The status quo ante of "penetrate and patch" that he was subtextually supporting is not something that most HN people would be comfortable with.
- Veserv 2y ago
  "Penetrate and Patch" as a failed security process is distinct from vilifying independent security research, and it should be obvious from my post as I point out that penetration testing is a integral part of the testing and verification process. It tells you if your process and design have failed, but it is not a good development process itself.
  - tptacek 2y ago
    Again and tediously: the only reason anyone would use that sequence of words would be to invoke Ranum's (in)famous "Six Dumbest Ideas In Computer Security" post, one of which was, in effect, "the entire modern science of software security". I recommend, in the future, that if you want to pursue a security policy angle in discussions online with people, you avoid using that term.
    - Veserv 2y ago
      I am invoking it. “Penetrate and Patch” is illustrative and catchy and the arguments presented, for that one in particular, are largely theoretically, empirically, and even predictively supported. In fact, all of points except “Hacking is Cool”, are largely well supported. The common thread being that all the other points are about “designing systems secure against common prevailing threat actors” (i.e “defense”) and only that point is about “development of adversarial capabilitys” (i.e. offense) which they wrongfully undervalue and even associate criminality with; misunderstanding the value of verification processes. And besides, the entire modern science of software security is, objectively, terrible at “designing systems secure against common prevailing threat actors”. What it is pretty good at is the “development of adversarial capabilitys” which has so far vastly outstripped the former and demonstrates quite clearly that prevailing “defense” is grossly inadequate by multiple orders of magnitude.

by Veserv 2y ago

JumpCrisscross 2y ago

> the government should organize vulnerabilitys by "difficulty" and announce the presence of, but not disclose the precise nature of, vulnerabilitys and demand process improvement until vulnerabilitys of that "difficulty" are not longer present as indicated by fixing all "known, but undiclosed" vulnerabilitys of that "difficulty" For this amount of bureaucracy, the government should just hire all coders and write all software.

Veserv 2y ago
You appear to misunderstand what I am saying: 1) Government already has vulnerabilitys. 2) Government identifies vulnerabilitys they already own by "difficulty to discovery". 3) Government selects the lowest "difficulty to discover" vulnerabilitys they already own. 4) Government announces products with known "lowest difficulty to discover" vulnerabilitys are vulnerable, but does not disclose them. 5) Government keeps announcing those products continue to be the most "insecure" until all vulnerabilitys they already own at that level are fixed. 6) Repeat.
- JumpCrisscross 2y ago
  What you're suggesting requires creating a massive federal bureaucracy to continuously survey the product landscape. It then requires the private sector to duplicate that work. This is stupid.
  - Veserv 2y ago
    I have no idea how you came to that conclusion. The government has vulnerabilitys in stock that they already verify function as intended. They already regularly verify these vulnerabilitys continue to function with each update cycle. They already quantify these vulnerabilitys by ease-of-discovery, impact, etc. so they can prioritize utilization. They already determine vulnerabilitys to disclose. Assuming that the vulnerabilitys they disclose are not entirely "actively under exploit", the only difference in my proposed policy is that they do not disclose the details to the vendors so they can paper over them. Instead, they publicly announce the presence of vulnerabilitys and then keep verifying as they already do until the vulnerabilitys no longer function. You seem to think I am arguing that the government should create a new organization to look for vulnerabilitys in all software everywhere and then act as I stated.
    - JumpCrisscross 2y ago
      > they publicly announce the presence of vulnerabilitys and then keep verifying as they already do until the vulnerabilitys no longer function Yes. This is difficult. Particularly given you need to do it fairly and thus comprehensively.
      - Veserv 2y ago
        Then you are arguing with a strawman. I never proposed they do it across all products, only the products that they already have vulnerabilitys in that they already seek to disclose. You can not change the parameters of my proposal to explicitly require a gigantic bureaucracy then argue that it is a poor idea because the gigantic bureaucracy you added is a problem. You could have argued that my proposal is untenable because it would be "unfair" and the only way to make it fair would be to do it comprehensively which would be too hard. To which I would state: 1) That means we as a society care more about being "fair" to companies with inadequate software security than demanding adequate software security. Could be the case, but then everything other than roll over and accept it is off-the-table. 2) The government already requires certification against the Common Criteria for many software products in use by the government. You could restrict this policy to just systems that are used and require certification before use. Thus being applied "fairly" to government procurement and incentivizing improvements in security for procured systems. 3) This should actually just be general policy for everybody, but only the government, currently, has enough leverage to really pull off publicly announcing a problem and the vendor not being able to just shove it under the rug. And, even if you disagree with those points, I am also making the point that the current policy of disclosing the vulnerability so the vendor can make a point-patch to resolve it instead of fixing their overall security process is a failed security policy at the societal level. We need mechanisms to encourage overall security process improvement. Currently, the only thing that does that is the exponentially increasing amount and severity of hacks, and it would be nice to get ahead of it instead of being purely reactionary.

tptacek 2y ago

"Penetrate and patch" is a term of art Marcus J. Ranum tried to popularize 15-20 years ago, as part of an effort to vilify independent security research. Ranum was part of an older iteration of software security that was driven by vendor-sponsored cliques. The status quo ante of "penetrate and patch" that he was subtextually supporting is not something that most HN people would be comfortable with.

Veserv 2y ago
"Penetrate and Patch" as a failed security process is distinct from vilifying independent security research, and it should be obvious from my post as I point out that penetration testing is a integral part of the testing and verification process. It tells you if your process and design have failed, but it is not a good development process itself.
- tptacek 2y ago
  Again and tediously: the only reason anyone would use that sequence of words would be to invoke Ranum's (in)famous "Six Dumbest Ideas In Computer Security" post, one of which was, in effect, "the entire modern science of software security". I recommend, in the future, that if you want to pursue a security policy angle in discussions online with people, you avoid using that term.
  - Veserv 2y ago
    I am invoking it. “Penetrate and Patch” is illustrative and catchy and the arguments presented, for that one in particular, are largely theoretically, empirically, and even predictively supported. In fact, all of points except “Hacking is Cool”, are largely well supported. The common thread being that all the other points are about “designing systems secure against common prevailing threat actors” (i.e “defense”) and only that point is about “development of adversarial capabilitys” (i.e. offense) which they wrongfully undervalue and even associate criminality with; misunderstanding the value of verification processes. And besides, the entire modern science of software security is, objectively, terrible at “designing systems secure against common prevailing threat actors”. What it is pretty good at is the “development of adversarial capabilitys” which has so far vastly outstripped the former and demonstrates quite clearly that prevailing “defense” is grossly inadequate by multiple orders of magnitude.