5 ms·

So what you are saying is that EAL >=5 does not imply secure, it is the formal methods that imply secure? Well good news, that is not a mistake. EAL >=5 implie

by Veserv 3y ago

So what you are saying is that EAL >=5 does not imply secure, it is the formal methods that imply secure? Well good news, that is not a mistake. EAL >=5 implies formal methods of various degrees. The standard demands the thing that works, amazing. It might be too hard for consumer companies to achieve, but that does not make the process a farce as long as there are companies that can certify useful functionality according to the process. And, although you seem to think the functionality that can be certified is limited, I doubt you consider secure cryptographic co-processors useless or unimportant.

tptacek 3y ago
In that paragraph, the words "of various degrees" is load-bearing. I don't think you're making a coherent case for the security or insecurity of an iPhone with any of this stuff. Real security in complex products simply has nothing to do with the CCTL process. Which, I'm sorry to say again, is farcical. If it's not clear: I've had the displeasure of working with this process in my career, most of which I've spent in vulnerability research.
- Veserv 3y ago
  Yes, various degrees; we are discussing three distinct levels. Why on earth would they all demand the same degree of formal methods? Up to EAL4 all you need is a informal specification; what is basically useless paperwork. It is only at EAL5 that you are required to supply a semi-formal design, ADV_TDS.4, and interface specification, ADV_FSP.5. At EAL6 you must also supply a formal model of the security policy, ADV_SPM.1, and a complete mapping between design and implementation, ADV_IMP.2. By EAL7 you are required to supply a complete formal design, ADV_TDS.6, and specification, ADV_FSP.6. You then have a formal model of the security policy, the interface enforcing that policy, the design backing the interface, and a mapping from the formal design to the implementation. That is pretty dang exhaustive. I guess you could go further and demand a formal, machine-checkable correspondence between the implementation and the design? Since you bring up that you have worked with the process, what products at EAL5 or higher have you had the displeasure of working on?
  - tptacek 3y ago
    This thread keeps trying to escape down little rabbitholes of abstraction. I'll be clear: if Apple wanted to ship an EAL5 product, the very first product decision they would need to make in service of that would be to stop rendering HTML. No browsers. No rich media. No installable apps that could do any kind of IPC. This is what we mean when we say an EAL5+ product is a different kind of thing. It's why this whole CCTL EAL thing is such a farce. The methods you're blaming big tech companies for not using are all things that preclude most of the products users want to use. Can we stop pretending that any vendor in the industry has the option of serving customers with formally verified "EAL6" products? They don't. This isn't a thing, and hearing "EAL6" and "penetration testing" mentioned in the same sentence is grating. You don't penetration test a formally verified product. You do verification and assurance work for it, but nobody in the field would ever call it pentesting.
    - Veserv 3y ago
      I am blaming them for selling products that are inadequate for the threat environment they are expected to operate in and lying and/or insinuating that they are adequate for that threat environment, especially when they know for certain that they are not and certify as such. If the customers truly want those products and features, security be damned, like you say then they will do that even if the companies are completely truthful. The companies do not because they know that it will hurt their margins or make their products non-viable. In addition, the lies suck all of the air out of the room for actual secure products because why go through the extremely hard work of actually making something secure when you can just lie about it. This has happened time and time again. There were multiple TCSEC Level A1 certified implementations when the DoD demanded real security. But, as soon as they lowered requirements to allow consumer systems that were inadequate for the threat environment, all of the effort and funding behind actual secure systems dried up. What we are left with now is a total wasteland of insecure products controlling systems too big for their britches and endless attacks getting closer and closer to total societal disruption. The only saving grace is that it is taking time for the attackers to scale to exploit this entire greenfield opportunity. As to your other points, the SKPP explicitly included a penetration test done by a NSA team of the formally verified product under certification [1]: "AVA_VLA_EXP.4.3E The NSA evaluator shall perform independent penetration testing.". So, actually, "EAL6" and "penetration testing" do belong in the same sentence. If Apple did want to ship a EAL5 product, the very first product decision would be starting over. HTML support or not does not even figure into the list; they have to tackle much more basic defects before getting to that. And I do not see how HTML and rich media support is even a challenge unless you made your security properties depend on exact rendering. You just use a separation kernel architecture to isolate the untrusted HTML renderer into a ephemeral sandbox that takes a HTML file and outputs an image. You might then say, "Apple already sandboxes the renderer.". Yeah, but their sandbox sucks and is regularly defeated. Almost as if having formally verified separation kernels as an underpinning might allow the obvious solution to work; assuming the same minds that built on sand do not add in more harebrained ideas. Hard to put it past them when they made so many other terrible security decisions. Also, you did not answer what EAL5 or higher products you worked on that informed your disgust with the high assurance Common Criteria process. [1] https://www.niap-ccevs.org/MMO/PP/pp_skpp_hr_v1.03.pdf https://www.niap-ccevs.org/MMO/PP/pp_skpp_hr_v1.03.pdf Page 118

by Veserv 3y ago

tptacek 3y ago

In that paragraph, the words "of various degrees" is load-bearing. I don't think you're making a coherent case for the security or insecurity of an iPhone with any of this stuff. Real security in complex products simply has nothing to do with the CCTL process. Which, I'm sorry to say again, is farcical. If it's not clear: I've had the displeasure of working with this process in my career, most of which I've spent in vulnerability research.

Veserv 3y ago
Yes, various degrees; we are discussing three distinct levels. Why on earth would they all demand the same degree of formal methods? Up to EAL4 all you need is a informal specification; what is basically useless paperwork. It is only at EAL5 that you are required to supply a semi-formal design, ADV_TDS.4, and interface specification, ADV_FSP.5. At EAL6 you must also supply a formal model of the security policy, ADV_SPM.1, and a complete mapping between design and implementation, ADV_IMP.2. By EAL7 you are required to supply a complete formal design, ADV_TDS.6, and specification, ADV_FSP.6. You then have a formal model of the security policy, the interface enforcing that policy, the design backing the interface, and a mapping from the formal design to the implementation. That is pretty dang exhaustive. I guess you could go further and demand a formal, machine-checkable correspondence between the implementation and the design? Since you bring up that you have worked with the process, what products at EAL5 or higher have you had the displeasure of working on?
- tptacek 3y ago
  This thread keeps trying to escape down little rabbitholes of abstraction. I'll be clear: if Apple wanted to ship an EAL5 product, the very first product decision they would need to make in service of that would be to stop rendering HTML. No browsers. No rich media. No installable apps that could do any kind of IPC. This is what we mean when we say an EAL5+ product is a different kind of thing. It's why this whole CCTL EAL thing is such a farce. The methods you're blaming big tech companies for not using are all things that preclude most of the products users want to use. Can we stop pretending that any vendor in the industry has the option of serving customers with formally verified "EAL6" products? They don't. This isn't a thing, and hearing "EAL6" and "penetration testing" mentioned in the same sentence is grating. You don't penetration test a formally verified product. You do verification and assurance work for it, but nobody in the field would ever call it pentesting.
  - Veserv 3y ago
    I am blaming them for selling products that are inadequate for the threat environment they are expected to operate in and lying and/or insinuating that they are adequate for that threat environment, especially when they know for certain that they are not and certify as such. If the customers truly want those products and features, security be damned, like you say then they will do that even if the companies are completely truthful. The companies do not because they know that it will hurt their margins or make their products non-viable. In addition, the lies suck all of the air out of the room for actual secure products because why go through the extremely hard work of actually making something secure when you can just lie about it. This has happened time and time again. There were multiple TCSEC Level A1 certified implementations when the DoD demanded real security. But, as soon as they lowered requirements to allow consumer systems that were inadequate for the threat environment, all of the effort and funding behind actual secure systems dried up. What we are left with now is a total wasteland of insecure products controlling systems too big for their britches and endless attacks getting closer and closer to total societal disruption. The only saving grace is that it is taking time for the attackers to scale to exploit this entire greenfield opportunity. As to your other points, the SKPP explicitly included a penetration test done by a NSA team of the formally verified product under certification [1]: "AVA_VLA_EXP.4.3E The NSA evaluator shall perform independent penetration testing.". So, actually, "EAL6" and "penetration testing" do belong in the same sentence. If Apple did want to ship a EAL5 product, the very first product decision would be starting over. HTML support or not does not even figure into the list; they have to tackle much more basic defects before getting to that. And I do not see how HTML and rich media support is even a challenge unless you made your security properties depend on exact rendering. You just use a separation kernel architecture to isolate the untrusted HTML renderer into a ephemeral sandbox that takes a HTML file and outputs an image. You might then say, "Apple already sandboxes the renderer.". Yeah, but their sandbox sucks and is regularly defeated. Almost as if having formally verified separation kernels as an underpinning might allow the obvious solution to work; assuming the same minds that built on sand do not add in more harebrained ideas. Hard to put it past them when they made so many other terrible security decisions. Also, you did not answer what EAL5 or higher products you worked on that informed your disgust with the high assurance Common Criteria process. [1] https://www.niap-ccevs.org/MMO/PP/pp_skpp_hr_v1.03.pdf https://www.niap-ccevs.org/MMO/PP/pp_skpp_hr_v1.03.pdf Page 118