4 ms·

Often when things like this happen there's a large conspiracy at play in some peoples minds, "Apple deliberately left a security vulnerability". But it falls ap

Often when things like this happen there's a large conspiracy at play in some peoples minds, "Apple deliberately left a security vulnerability". But it falls apart pretty quickly, it's in an open sourced package, so the assumption is someone is eventually going to see it, so it's not going to remain a secret and thus is useless as a stealth backdoor. The likelihood is pretty simple, someone fucked up. On a potentially huge level, but a fuck up none the less. These things do unfortunately happen, and no doubt it'll prompt an internal review of their change management process, and their build chain, and what they can do to isolate issues like this in the future.

corresation 13y ago
While I believe that it was just a screw-up that wasn't caught because of gaps of process (the code essentially repeats blocks with minor changes, so I can envision someone doing some refactoring -- removing the passing of the context -- and trying to save time by copying the first changed block to the second, failing to overwrite that single additional line), considering the possibility that it was intentional in no universe suggests that it was Apple the organization that chose and instituted the vulnerability. A single employee receiving a second paycheck from a three-letter organization could have been responsible. A vulnerability could have been used to plant it. And so on. Remember when everyone was railing about Google giving NSA a backdoor (cue a thousand "don't be evil" cries) when in reality the NSA and friends were simply exploiting a weakness of Google's network. The net effect was the same. It could be intentional and still entirely unintentional as far as Apple the company is concerned. And to the open source thing -- it sat there for 18 months. Indeed, it was caught [EDIT: Actually I don't know how it was caught. I faintly recall someone posting an issue someone submitted to Apple detailing some weird behavior with invalid private keys, but can't find it now]
- ybaumes 13y ago
  "And to the open source thing -- it sat there for 18 months." !!! Just .. wow. If it was the first time we heard such stories. But it happends many time with the linux kernel source code and all. I definitively conclude that source code begin open to everyone is not a mark of higher quality.
- nicholassmith 13y ago
  "It could be intentional and still entirely unintentional as far as Apple the company is concerned." Agreed, as far as Apple know it was a bug, but one of their staff could have been turned. But they'll be able to audit the change logs, and I imagine a company famously known for their OTT internal security would be preparing the car batteries and nipple clamps if they thought they'd found a bad actor.
nly 13y ago
> it's in an open sourced package, so the assumption is someone is eventually going to see it, so it's not going to remain a secret and thus is useless as a stealth backdoor. The fact that people can see the nature of the bug in the source code for themselves just makes the case that it was a deliberate backdoor even more deniable which, as Schneier is pointing out, is a very desirable property. You're also assuming that a backdoor you can use indefinitely is desirable. As an added side-benefit, any other proprietary or FOSS products that adopted this code are now vulnerable.
- nicholassmith 13y ago
  Sure, it makes it even more deniable, but on the balance between 'well thought out conspiracy with I know-you know-I know you know I know' logic, and a fuck up, I'd err towards it being a fuck up.
  - nly 13y ago
    You're just casually brushing this off with Hanlon's Razor. You shouldn't discount the fact that as technical folk and programmers here on HN we're probably biased[0]. Our ability to recognise just how easily this flaw could have been both introduced and detected can both be used both against, and in defence, of the conspiracy theory. [0] https://en.wikipedia.org/wiki/Attribution_%28psychology%29#Defensive_attribution_hypothesis https://en.wikipedia.org/wiki/Attribution_%28psychology%29#D...
    - nicholassmith 13y ago
      I've been a proponent of Hanlon's Razor a number of times, so guilty as charged. As technically minded people we intrinsically understand that it could be the work of someone malicious, but we should also understand from personal experiences that fuck ups happen significantly more often than brown envelopes full of cash.
mzs 13y ago
The thing that bugs me is so many people are saying there is no way that could be a merge/editing error so it had to have been deliberate bad actor, but all they are comparing is a diff between the sources that match two released versions and using that as evidence that it was not very plausible that it was such en error. What I think happened is very plausible though: if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0) goto fail; if ((err = SSLHashSHA1.final(&hashCtx, &hashOut)) != 0) goto fail; Some other check was added: if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0) goto fail; + if ((err = SSLHashSHA1.foobar(&hashCtx, &signedParams)) != 0) + goto fail; if ((err = SSLHashSHA1.final(&hashCtx, &hashOut)) != 0) goto fail; Then that check was merged into another check, otherwise refactored, or decided it was not needed there anymore - but mistakenly only the added if ... line was removed: if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0) goto fail; - if ((err = SSLHashSHA1.foobar(&hashCtx, &signedParams)) != 0) goto fail; if ((err = SSLHashSHA1.final(&hashCtx, &hashOut)) != 0) goto fail; And that's how that ended-up the way that it did. if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0) goto fail; goto fail; if ((err = SSLHashSHA1.final(&hashCtx, &hashOut)) != 0) goto fail; Then of course it was not the only change submitted, it must have been a pretty sizable diff of many files, and it slipped-by in all that.
- cpeterso 13y ago
  The scenario you've described would require multiple changes to the same function, but the file diffs making their around the blogosphere show only cosmetic changes to that file and none of those affect the "goto fail" function.
  - mzs 13y ago
    Except there was stuff going on related to this: Security-55471/libsecurity_ssl/security_ssl/tls_digest.c #ifdef KERNEL static int HashSHA1Update(SSLBuffer *digestCtx, const SSLBuffer *data) { return ccHashUpdate(ccsha1_di(), digestCtx, data); } #else static int HashSHA1Update(SSLBuffer *digestCtx, const SSLBuffer *data) { /* 64 bits cast: safe, SSL records are always smaller than 2^32 bytes */ assert(digestCtx->length >= sizeof(CC_SHA1_CTX)); CC_SHA1_CTX *ctx = (CC_SHA1_CTX *)digestCtx->data; CC_SHA1_Update(ctx, data->data, (CC_LONG)data->length); return 0; } #endif const HashReference SSLHashSHA1 = { SSL_SHA1_DIGEST_LENGTH, 40, SSL_SHA1_CONTEXT_SIZE, HashSHA1Init, HashSHA1Update, HashSHA1Final, HashSHA1Close, HashSHA1Clone }; Security-55179.13/libsecurity_ssl/security_ssl/sslDigests.c static OSStatus HashSHA1Update(SSLBuffer *digestCtx, const SSLBuffer *data) { /* 64 bits cast: safe, SSL records are always smaller than 2^32 bytes */ assert(digestCtx->length >= sizeof(CC_SHA1_CTX)); CC_SHA1_CTX *ctx = (CC_SHA1_CTX *)digestCtx->data; CC_SHA1_Update(ctx, data->data, (CC_LONG)data->length); return noErr; } const HashReference SSLHashSHA1 = { sizeof(CC_SHA1_CTX), CC_SHA1_DIGEST_LENGTH, 40, HashSHA1Init, HashSHA1Update, HashSHA1Final, HashSHA1Close, HashSHA1Clone }; Do you see how there was some simple refactoring going on there in the structure and how the entire in-kernel functionality was added? I could totally see debugging function being created and sprinkled in all the places after all the update()s were called while verifying kernel stuff was working then removed and in this one place one line not enough deleted. edit: Ha, I had my own copy paste error after getting bad gateway error, I fixed the duplicated code, sorry.

corresation 13y ago

While I believe that it was just a screw-up that wasn't caught because of gaps of process (the code essentially repeats blocks with minor changes, so I can envision someone doing some refactoring -- removing the passing of the context -- and trying to save time by copying the first changed block to the second, failing to overwrite that single additional line), considering the possibility that it was intentional in no universe suggests that it was Apple the organization that chose and instituted the vulnerability. A single employee receiving a second paycheck from a three-letter organization could have been responsible. A vulnerability could have been used to plant it. And so on. Remember when everyone was railing about Google giving NSA a backdoor (cue a thousand "don't be evil" cries) when in reality the NSA and friends were simply exploiting a weakness of Google's network. The net effect was the same. It could be intentional and still entirely unintentional as far as Apple the company is concerned. And to the open source thing -- it sat there for 18 months. Indeed, it was caught [EDIT: Actually I don't know how it was caught. I faintly recall someone posting an issue someone submitted to Apple detailing some weird behavior with invalid private keys, but can't find it now]

ybaumes 13y ago
"And to the open source thing -- it sat there for 18 months." !!! Just .. wow. If it was the first time we heard such stories. But it happends many time with the linux kernel source code and all. I definitively conclude that source code begin open to everyone is not a mark of higher quality.
nicholassmith 13y ago
"It could be intentional and still entirely unintentional as far as Apple the company is concerned." Agreed, as far as Apple know it was a bug, but one of their staff could have been turned. But they'll be able to audit the change logs, and I imagine a company famously known for their OTT internal security would be preparing the car batteries and nipple clamps if they thought they'd found a bad actor.

nly 13y ago

> it's in an open sourced package, so the assumption is someone is eventually going to see it, so it's not going to remain a secret and thus is useless as a stealth backdoor. The fact that people can see the nature of the bug in the source code for themselves just makes the case that it was a deliberate backdoor even more deniable which, as Schneier is pointing out, is a very desirable property. You're also assuming that a backdoor you can use indefinitely is desirable. As an added side-benefit, any other proprietary or FOSS products that adopted this code are now vulnerable.

nicholassmith 13y ago
Sure, it makes it even more deniable, but on the balance between 'well thought out conspiracy with I know-you know-I know you know I know' logic, and a fuck up, I'd err towards it being a fuck up.
- nly 13y ago
  You're just casually brushing this off with Hanlon's Razor. You shouldn't discount the fact that as technical folk and programmers here on HN we're probably biased[0]. Our ability to recognise just how easily this flaw could have been both introduced and detected can both be used both against, and in defence, of the conspiracy theory. [0] https://en.wikipedia.org/wiki/Attribution_%28psychology%29#Defensive_attribution_hypothesis https://en.wikipedia.org/wiki/Attribution_%28psychology%29#D...
  - nicholassmith 13y ago
    I've been a proponent of Hanlon's Razor a number of times, so guilty as charged. As technically minded people we intrinsically understand that it could be the work of someone malicious, but we should also understand from personal experiences that fuck ups happen significantly more often than brown envelopes full of cash.

mzs 13y ago

The thing that bugs me is so many people are saying there is no way that could be a merge/editing error so it had to have been deliberate bad actor, but all they are comparing is a diff between the sources that match two released versions and using that as evidence that it was not very plausible that it was such en error. What I think happened is very plausible though: if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0) goto fail; if ((err = SSLHashSHA1.final(&hashCtx, &hashOut)) != 0) goto fail; Some other check was added: if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0) goto fail; + if ((err = SSLHashSHA1.foobar(&hashCtx, &signedParams)) != 0) + goto fail; if ((err = SSLHashSHA1.final(&hashCtx, &hashOut)) != 0) goto fail; Then that check was merged into another check, otherwise refactored, or decided it was not needed there anymore - but mistakenly only the added if ... line was removed: if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0) goto fail; - if ((err = SSLHashSHA1.foobar(&hashCtx, &signedParams)) != 0) goto fail; if ((err = SSLHashSHA1.final(&hashCtx, &hashOut)) != 0) goto fail; And that's how that ended-up the way that it did. if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0) goto fail; goto fail; if ((err = SSLHashSHA1.final(&hashCtx, &hashOut)) != 0) goto fail; Then of course it was not the only change submitted, it must have been a pretty sizable diff of many files, and it slipped-by in all that.

cpeterso 13y ago
The scenario you've described would require multiple changes to the same function, but the file diffs making their around the blogosphere show only cosmetic changes to that file and none of those affect the "goto fail" function.
- mzs 13y ago
  Except there was stuff going on related to this: Security-55471/libsecurity_ssl/security_ssl/tls_digest.c #ifdef KERNEL static int HashSHA1Update(SSLBuffer *digestCtx, const SSLBuffer *data) { return ccHashUpdate(ccsha1_di(), digestCtx, data); } #else static int HashSHA1Update(SSLBuffer *digestCtx, const SSLBuffer *data) { /* 64 bits cast: safe, SSL records are always smaller than 2^32 bytes */ assert(digestCtx->length >= sizeof(CC_SHA1_CTX)); CC_SHA1_CTX *ctx = (CC_SHA1_CTX *)digestCtx->data; CC_SHA1_Update(ctx, data->data, (CC_LONG)data->length); return 0; } #endif const HashReference SSLHashSHA1 = { SSL_SHA1_DIGEST_LENGTH, 40, SSL_SHA1_CONTEXT_SIZE, HashSHA1Init, HashSHA1Update, HashSHA1Final, HashSHA1Close, HashSHA1Clone }; Security-55179.13/libsecurity_ssl/security_ssl/sslDigests.c static OSStatus HashSHA1Update(SSLBuffer *digestCtx, const SSLBuffer *data) { /* 64 bits cast: safe, SSL records are always smaller than 2^32 bytes */ assert(digestCtx->length >= sizeof(CC_SHA1_CTX)); CC_SHA1_CTX *ctx = (CC_SHA1_CTX *)digestCtx->data; CC_SHA1_Update(ctx, data->data, (CC_LONG)data->length); return noErr; } const HashReference SSLHashSHA1 = { sizeof(CC_SHA1_CTX), CC_SHA1_DIGEST_LENGTH, 40, HashSHA1Init, HashSHA1Update, HashSHA1Final, HashSHA1Close, HashSHA1Clone }; Do you see how there was some simple refactoring going on there in the structure and how the entire in-kernel functionality was added? I could totally see debugging function being created and sprinkled in all the places after all the update()s were called while verifying kernel stuff was working then removed and in this one place one line not enough deleted. edit: Ha, I had my own copy paste error after getting bad gateway error, I fixed the duplicated code, sorry.