4 ms·

I went down the rabbit hole one day when discussing how certificates work. To me trust was a vague concept considering what was at stake. I ended up at Auditors

I went down the rabbit hole one day when discussing how certificates work. To me trust was a vague concept considering what was at stake. I ended up at Auditors but even then it feels odd that so much trust comes down to a few people.And even worse those auditors use computers that use certificates of companies they audit. I used to work in a casino and Compliance/Regulator (a type of audit) would check the settings I input. After much employee churn it got to the point where the regulator was just a warm body with a clipboard. I'd enter a setting say jackpot limit $1,200, the auditor asked me to show him, they checked it off. But the regulator used my copy of the data so they were telling me what I had input was right but they were using the exact same data. The regulator should have used a separate source of data (they used to), what if my data was wrong? It's happened before many times. Anyway that's what CA auditors reminded me of.

tialaramex 8y ago
Audit ought correctly to be a last resort, we should only audit things because the superior options were impractical (or outright impossible). Auditors are invariably conflicted (not giving a clean audit is bad for business) and other actors must work to mitigate this problem. Part of the failure of Symantec (causing them to exit the CA business last year as a result of financial pressure) was that rather than having in place their own effective oversight they had increasingly relied upon audits from the Big Four to confirm that things were being done correctly at their "partner" companies, even though the wider PKI community had seen repeatedly that audit is not very effective and should not be a substitute for oversight roles. As a result when problems with CrossCert were identified, Symantec found they didn't have the records that ought to have existed, and ended up asking CrossCert for them, of course CrossCert did not have such records. This situation couldn't have happened if Symantec had effective oversight, since necessarily this oversight role would examine the (non-existent) records. Individual audit companies may prove either incompetent or corrupt too. Mozilla concluded that the Hong Kong EY (Once "Ernst and Young" but short names are cool now apparently) was either incompetent or corrupt and told its parent that it will no longer accept audits from this outfit. This was part of the problem with StartCom, their audits don't mention things that should have been evident at the time to a competent auditor. A central purpose of Certificate Transparency is to reduce the need to rely on auditors. With all the signed certificates made public, independent researchers can interrogate data once only accessible to auditors. For example, the question "Were any certificates issued for Debian Weak Keys?" is something we might once have resolved with audit - let's ask the CA "Do you check for weak Debian keys?" and if they say "Yes" that's a pass. But today researchers can simply download all the public keys, and memcmp() them against Debian Weak Keys. Upon finding one, they raise it as a Problem Report. Sure enough the CA discovers that although of course they believed they were checking for these weak keys, for a brief period the check was disabled and they issued a certificate. Oops. Observatory tools like Censys might have helped, but Certificate Transparency gives us guaranteed coverage. Nevertheless, audit is a necessity for some practical questions. Auditors physically visiting a facility can verify that it is indeed physically secure (are the data centre fire doors left open onto a shopping centre car park?) and that documentation physically exists (are there, in fact, binders full of procedures? It's funny, when they know nobody follows the procedures humans are often too lazy to even bother writing them down...). Likewise auditors must attend certain ceremonies e.g. to create a new private key. It would be very expensive for representatives from the major Public Trust Stores to be sent around the world to do this, and it's clearly not acceptable to just insist that all CAs must be physically located in Northern California.
- dghughes 8y ago
  >Individual audit companies may prove either incompetent or corrupt too... I agree. I've experienced both situations. KPMG did an audit of my former workplace and I was amazed at what they found and so fast. It was a single receipt that a cashier rang in but the supervisor had forgot to log off the POS. The cashier didn't notice and rang in that one item under the supervior's login. The KPMG auditor caught it on paper not electronic they had gone through four year's of paperwork for one department and spotted it (on the first day of the audit too!). After that Deloitte or KPMG seemed to be mailing it in. I knew some departments in the place where I worked had greatly slipped in following protocol and none of it was discovered by the auditors. Or I should say I wasn't aware of any discoveries since I was not directly involved in the process.

tialaramex 8y ago

Audit ought correctly to be a last resort, we should only audit things because the superior options were impractical (or outright impossible). Auditors are invariably conflicted (not giving a clean audit is bad for business) and other actors must work to mitigate this problem. Part of the failure of Symantec (causing them to exit the CA business last year as a result of financial pressure) was that rather than having in place their own effective oversight they had increasingly relied upon audits from the Big Four to confirm that things were being done correctly at their "partner" companies, even though the wider PKI community had seen repeatedly that audit is not very effective and should not be a substitute for oversight roles. As a result when problems with CrossCert were identified, Symantec found they didn't have the records that ought to have existed, and ended up asking CrossCert for them, of course CrossCert did not have such records. This situation couldn't have happened if Symantec had effective oversight, since necessarily this oversight role would examine the (non-existent) records. Individual audit companies may prove either incompetent or corrupt too. Mozilla concluded that the Hong Kong EY (Once "Ernst and Young" but short names are cool now apparently) was either incompetent or corrupt and told its parent that it will no longer accept audits from this outfit. This was part of the problem with StartCom, their audits don't mention things that should have been evident at the time to a competent auditor. A central purpose of Certificate Transparency is to reduce the need to rely on auditors. With all the signed certificates made public, independent researchers can interrogate data once only accessible to auditors. For example, the question "Were any certificates issued for Debian Weak Keys?" is something we might once have resolved with audit - let's ask the CA "Do you check for weak Debian keys?" and if they say "Yes" that's a pass. But today researchers can simply download all the public keys, and memcmp() them against Debian Weak Keys. Upon finding one, they raise it as a Problem Report. Sure enough the CA discovers that although of course they believed they were checking for these weak keys, for a brief period the check was disabled and they issued a certificate. Oops. Observatory tools like Censys might have helped, but Certificate Transparency gives us guaranteed coverage. Nevertheless, audit is a necessity for some practical questions. Auditors physically visiting a facility can verify that it is indeed physically secure (are the data centre fire doors left open onto a shopping centre car park?) and that documentation physically exists (are there, in fact, binders full of procedures? It's funny, when they know nobody follows the procedures humans are often too lazy to even bother writing them down...). Likewise auditors must attend certain ceremonies e.g. to create a new private key. It would be very expensive for representatives from the major Public Trust Stores to be sent around the world to do this, and it's clearly not acceptable to just insist that all CAs must be physically located in Northern California.

dghughes 8y ago
>Individual audit companies may prove either incompetent or corrupt too... I agree. I've experienced both situations. KPMG did an audit of my former workplace and I was amazed at what they found and so fast. It was a single receipt that a cashier rang in but the supervisor had forgot to log off the POS. The cashier didn't notice and rang in that one item under the supervior's login. The KPMG auditor caught it on paper not electronic they had gone through four year's of paperwork for one department and spotted it (on the first day of the audit too!). After that Deloitte or KPMG seemed to be mailing it in. I knew some departments in the place where I worked had greatly slipped in following protocol and none of it was discovered by the auditors. Or I should say I wasn't aware of any discoveries since I was not directly involved in the process.