7 ms·

It's interesting watching all the speculation about "was it a backdoor, or just a bug?" Lots of points in favor or against: 1) It's a huge compromise, and "op

by rdl 13y ago

It's interesting watching all the speculation about "was it a backdoor, or just a bug?" Lots of points in favor or against: 1) It's a huge compromise, and "open" to anyone to exploit, which would ultimately get caught and fixed faster. But it's also not targeting anything specific, so there's less of a signature of the attacker. 2) Incredibly simple, and thus a plausible mistake. 3) Hidden in plain sight I'd generally come down on the side of "accident". The better question is if an systematic testing system for exploitable weaknesses (or a lucky chance finding) could find something like this (either a regression, or a badly implemented new feature) and exploit it -- basically assuming there are going to be errors in implementations. That's understood to be a standard technique in the IC, and a whole industry built around it... and then, the scale of compromise. There are lots of mitigation techniques for vulnerabilities like this (essentially, devices which are too fast-changing and too difficult to fully trust, but which have to touch sensitive data), but it's not as if people can carry around a firewall in their pocket these days, sadly. I'm certainly re-generating any keys which touched iOS or OSX devices (thankfully very few), and reinstalling OSes, in the interim.

brudgers 13y ago
My gut says the probability of a random typo executing without raising an error or exception is rather low. The probability of it doing so in a way that well aligns with the interests of nation states, large corporations, and/or criminal enterprise is even lower. This might explain DROPOUTJEEP particularly considering how much more efficient it is than breaking messages after they are encrypted. http://mobile.eweek.com/security/nsa-spying-on-apple-iphones.html http://mobile.eweek.com/security/nsa-spying-on-apple-iphones...
- pjene 13y ago
  Sure, this aligns with interests, but the bug's existence is predicated on the entire code base being written with substandard style rules and no static analysis or tests, which suggests to me that incompetence got here first.
  - brudgers 13y ago
    Weaknesses are what sophisticated adversaries attempt to exploit. Lack of static analysis in the code base might make an exploit take a particular form and make it easier to craft an exploit. But it's the size of the target base and its quality which makes it worth trying considering the way in which iPhones may be present in even a security savvy an individual's social context.
  - drawkbox 13y ago
    It is mind blowing that they may not run static analysis on something as big as OS code bases, for something as simple as a duplicated switch break. Easily could have been a copy paste fail but that is why on a project this big you need to have that. Or if they do use static analysis on builds and it failed or by-passed this area, there is another possible hole.
  - terminus 13y ago
    > entire code base being written with substandard style rules and no static analysis or tests, which suggests to me that incompetence got here first. That and inadequate, bordering on zero, code review. Even a beginner C programmer looking at this code could see how fishy it looks. No code review while checking in code to libssl. That takes a lot of incompetence.
    - SimHacker 13y ago
      Whether it was incompetence or malice, whoever was responsible for the extra goto, as well as whoever was responsible for ANY of that code not following common practices including ALWAYS using brackets, should be fired. And that fact that this bug and terrible coding style was in the publicly available source code for so long totally disproves ESR's "many eyes make all bugs shallow" myth. Thanks for the false sense of security, Eric. The chickens have come home to roost again. If "many eyes make all bugs shallow" were true, somebody would have raised the flag "Hey everybody, these Bozos are actually omitting brackets in their SSL code! Somebody could carelessly insert a statement and accidentally (or maliciously) introduce a hard to see bug some day!" Hardly anybody actually bothers to read code in the real world. So there aren't "many" eyes, and even if there were, many bugs aren't shallow even to expert eyes, and "all" bugs will never be shallow to most eyes. That's why it's important to pay competent security professionals to actually take the time and effort to audit source code, which is difficult work that requires much time and effort that takes them away from other valuable, high paying, less tedious and mind numbing work.
- lloeki 13y ago
  > My gut says the probability of a random typo executing without raising an error or exception is rather low In many code editors, numerous fat fingered shortcuts could produce such a compilable line duplication/deletion (deletion because maybe that's not a goto line duplicated but a test line deleted). I'm baffled that neither clang nor gcc spits a warning for the unreachable code. Not trying to shift blames or anything, but rather than do some finger pointing, I'd rather see this as a warning for all of us to ramp up our game and better our tools so that everyone benefits, reducing the risk of both honest and intentional or covert[1] malfunctions. [0]: example custom shortcut as ^d to duplicate (could have been cmd(+shift)+D, which sits right along cmd(+shift)+S): http://www.xinsight.ca/blog/xcode-trick-creating-a-shortcut-to-duplicate-a-line/ http://www.xinsight.ca/blog/xcode-trick-creating-a-shortcut-... [1]: http://en.wikipedia.org/wiki/Underhanded_C_Contest http://en.wikipedia.org/wiki/Underhanded_C_Contest
0x0 13y ago
Why would you need to regenerate keys? How does mitm ssl attacks compromise keys?
- rdl 13y ago
  Compromise system (via update mechanism or other things secured with SSL, VPN software clients, ...); use that to steal keys. I don't believe I've been a victim, and most of the ways you'd exploit this would leave enough traces to be high-risk-of-detection, I think. But, it's hard to prove a negative, so it's safer to act as if it had been compromised. (I'm also updating keys anyway, so this is just a matter of waiting a day or two to do so.)
- deleted 13y ago
  [deleted]
higherpurpose 13y ago
Whether it was intentional backdoor or not, NSA must've had a field day with it by now.
- rdl 13y ago
  In general, NSA is not my most serious threat (in terms of actual harm; they are the most powerful by far though) -- I agree they overreach, and for some people are a serious threat, but for me my primary concerns are Chinese/other foreign intel (who are documented as going after industrial/economic material much more than NSA) and independent/criminal/etc. types. The scary thing is the bar is so low; even I could turn this into a nice weaponized thing to go after the long tail of people who don't upgrade, for the next months. The "diode" etc. infrastructure NSA built isn't that different from botnet C&C or the kind of relays people have when hacking.
  - sosborn 13y ago
    > NSA is not my most serious threat I wish more people understood this. Not to let NSA off the hook (what they are doing is awful), but the threat posed by the NSA is a higher-level down the road/slippery slope threat. There are other more immediate and real dangers out there that are actively trying to steal whatever they can find.
    - MatthiasP 13y ago
      It also very much depends on which side of the Atlantic you are living on.
      - peteretep 13y ago
        More likely Pacific
        grogenaut 13y ago
        Asia is technically on the other side of the atlantic as well, just further that way. Just to out pedantic you.
Varcht 13y ago
Regarding hidden in plain sight, would a semicolon at the end of the if statement throw a warning? Would be a little more hidden.
- huhtenberg 13y ago
  Yeah, that'd trigger a warning in most compilers - "Empty statement found. Was it an intent?" or something similar.
  - valleyer 13y ago
    Not true, try for (;;) ; in gcc / clang
    - SimHacker 13y ago
      That's because "for (;;) ;" is and idiom that's commonly used in a programming pattern called the "for(;;);ce field", to work around other C and C preprocessor quirks, by syntactically insulating and protecting statements in C preprocessor macros from outside interference. As a matter of fact, by deploying one of those the beginning of the macro, one in the middle, and one at the end, you can set up what's called a "trifor(;;);ce field", an anti-anti-pattern that is documented on the c2 wiki thusly: The trifor(;;);ce has the power to grant the wish of whomever touches it, and molds the Sacred Realm to reflect that person's heart. The trifor(;;);ce does not discriminate between "good" or "evil". However, if a person without an equal balance of power, wisdom, and courage makes a wish, the trifor(;;);ce will split into its three separate parts: the piece that best personifies the wish-maker will be the only piece to remain in hand, whilst the other two will take residence in whosoever most personifies them. Reassembly is then required for such a person's wish to be granted, but does not exclude another from making the attempt. No, I'm just joking and making shit up. Don't anyone ever do that! As bitchy and pedantic as gcc and clang are about other things, I'm disappointed they don't complain about that.
blazespin 13y ago
It was likely someone subverting the process for reasons of urgency, for good intentions or malice. Simple static analysis or unit tests would have caught this.
- patricjansson 13y ago
  Exactly! So strange apple didnt have a testcase for this.
  - jeffaustin 13y ago
    Yea, and we're supposed to believe that no QA person did any testing on this either.
tlrobinson 13y ago
Has the author of the buggy code been identified?
- deleted 13y ago
  [deleted]
corresation 13y ago
3) Hidden in plain sight I'm baffled that this wasn't caught long, long ago. Most of us have worked with internal systems where certs, for whatever reason, don't match the site, and it's surprising many people haven't noted that such doesn't raise any errors on iOS/OSX. Though thinking back....I actually remember encountering exactly that on the iPad once, surprised that it didn't raise a flag. Like probably most I just shrugged and continued on.
- mehrdada 13y ago
  Because it does not work like that. SecureTransport actually verifies the validity of the TLS certificate. If you hand it an invalid certificate in the first place, it will complain. The issue is one level deeper. When you connect to a server that supports ephemeral key exchange, the parties involved will generate new key pairs on the fly to use for that connection. In order to make sure that the server has the key published in its certificate, the ephemeral public key shown to the client must be signed by the static private key of the certificate and the client MUST verify it. It is the ephemeral key signature that is not validated, not the signature on the certificate itself. What happens here is that the server gives the client a _valid_ certificate for which it does not own the private key, and in the ephemeral step, it simply generates a key pair without a valid signature. At no point in the handshake the server is asked to verify its ownership of the private key associated to the certificate it is presenting To summarize, TLS certificate should be valid, but you are never asked to verify you have its private key when the connection used DHE or ECDHE.
  - corresation 13y ago
    Thank you this is illuminating.
gnyman 13y ago
Does not NSA have access to multiple of the private keys of the preinstalled root certificates anyways? If so they can MITM any SSL connection where you don't specifically say which CA cert it should use? Isn't it the case anyways that SSL is not very efficient against state sponsored attacks as almost all of them can generate certificates for any domain? All this speaks towards it being a unfortunate mistake not something malicious, unless there is something wrong with my understanding how things work :-)
wreegab 13y ago
I believe the faulty function -- at line 574 -- is a copy/paste of another function located at line 328 in the same file. I diff'ed both functions to try to understand the changes which may have been made. The "goto fail;" is somewhat "drowned" into other changes around, so it stands out less using a diff tool because of these surrounding changes. The surrounding changes in question: 1) Moving a "SSLFreeBuffer(&hashCtx)" instruction at a higher position: Very strange since it accomplish absolutely nothing at its new position, as opposed to it's original position. 2) Renaming of two variables: "hash" became "hashOut" and "exchangeParams" became "signedParams". 3) Indentation of "goto"s Without the above changes, the "goto fail;" stand out rather well when using a diff tool.

by rdl 13y ago

brudgers 13y ago

My gut says the probability of a random typo executing without raising an error or exception is rather low. The probability of it doing so in a way that well aligns with the interests of nation states, large corporations, and/or criminal enterprise is even lower. This might explain DROPOUTJEEP particularly considering how much more efficient it is than breaking messages after they are encrypted. http://mobile.eweek.com/security/nsa-spying-on-apple-iphones.html http://mobile.eweek.com/security/nsa-spying-on-apple-iphones...

pjene 13y ago
Sure, this aligns with interests, but the bug's existence is predicated on the entire code base being written with substandard style rules and no static analysis or tests, which suggests to me that incompetence got here first.
- brudgers 13y ago
  Weaknesses are what sophisticated adversaries attempt to exploit. Lack of static analysis in the code base might make an exploit take a particular form and make it easier to craft an exploit. But it's the size of the target base and its quality which makes it worth trying considering the way in which iPhones may be present in even a security savvy an individual's social context.
- drawkbox 13y ago
  It is mind blowing that they may not run static analysis on something as big as OS code bases, for something as simple as a duplicated switch break. Easily could have been a copy paste fail but that is why on a project this big you need to have that. Or if they do use static analysis on builds and it failed or by-passed this area, there is another possible hole.
- terminus 13y ago
  > entire code base being written with substandard style rules and no static analysis or tests, which suggests to me that incompetence got here first. That and inadequate, bordering on zero, code review. Even a beginner C programmer looking at this code could see how fishy it looks. No code review while checking in code to libssl. That takes a lot of incompetence.
  - SimHacker 13y ago
    Whether it was incompetence or malice, whoever was responsible for the extra goto, as well as whoever was responsible for ANY of that code not following common practices including ALWAYS using brackets, should be fired. And that fact that this bug and terrible coding style was in the publicly available source code for so long totally disproves ESR's "many eyes make all bugs shallow" myth. Thanks for the false sense of security, Eric. The chickens have come home to roost again. If "many eyes make all bugs shallow" were true, somebody would have raised the flag "Hey everybody, these Bozos are actually omitting brackets in their SSL code! Somebody could carelessly insert a statement and accidentally (or maliciously) introduce a hard to see bug some day!" Hardly anybody actually bothers to read code in the real world. So there aren't "many" eyes, and even if there were, many bugs aren't shallow even to expert eyes, and "all" bugs will never be shallow to most eyes. That's why it's important to pay competent security professionals to actually take the time and effort to audit source code, which is difficult work that requires much time and effort that takes them away from other valuable, high paying, less tedious and mind numbing work.
lloeki 13y ago
> My gut says the probability of a random typo executing without raising an error or exception is rather low In many code editors, numerous fat fingered shortcuts could produce such a compilable line duplication/deletion (deletion because maybe that's not a goto line duplicated but a test line deleted). I'm baffled that neither clang nor gcc spits a warning for the unreachable code. Not trying to shift blames or anything, but rather than do some finger pointing, I'd rather see this as a warning for all of us to ramp up our game and better our tools so that everyone benefits, reducing the risk of both honest and intentional or covert[1] malfunctions. [0]: example custom shortcut as ^d to duplicate (could have been cmd(+shift)+D, which sits right along cmd(+shift)+S): http://www.xinsight.ca/blog/xcode-trick-creating-a-shortcut-to-duplicate-a-line/ http://www.xinsight.ca/blog/xcode-trick-creating-a-shortcut-... [1]: http://en.wikipedia.org/wiki/Underhanded_C_Contest http://en.wikipedia.org/wiki/Underhanded_C_Contest

0x0 13y ago

Why would you need to regenerate keys? How does mitm ssl attacks compromise keys?

rdl 13y ago
Compromise system (via update mechanism or other things secured with SSL, VPN software clients, ...); use that to steal keys. I don't believe I've been a victim, and most of the ways you'd exploit this would leave enough traces to be high-risk-of-detection, I think. But, it's hard to prove a negative, so it's safer to act as if it had been compromised. (I'm also updating keys anyway, so this is just a matter of waiting a day or two to do so.)
deleted 13y ago
[deleted]

higherpurpose 13y ago

Whether it was intentional backdoor or not, NSA must've had a field day with it by now.

rdl 13y ago
In general, NSA is not my most serious threat (in terms of actual harm; they are the most powerful by far though) -- I agree they overreach, and for some people are a serious threat, but for me my primary concerns are Chinese/other foreign intel (who are documented as going after industrial/economic material much more than NSA) and independent/criminal/etc. types. The scary thing is the bar is so low; even I could turn this into a nice weaponized thing to go after the long tail of people who don't upgrade, for the next months. The "diode" etc. infrastructure NSA built isn't that different from botnet C&C or the kind of relays people have when hacking.
- sosborn 13y ago
  > NSA is not my most serious threat I wish more people understood this. Not to let NSA off the hook (what they are doing is awful), but the threat posed by the NSA is a higher-level down the road/slippery slope threat. There are other more immediate and real dangers out there that are actively trying to steal whatever they can find.
  - MatthiasP 13y ago
    It also very much depends on which side of the Atlantic you are living on.
    - peteretep 13y ago
      More likely Pacific
      - grogenaut 13y ago
        Asia is technically on the other side of the atlantic as well, just further that way. Just to out pedantic you.

Varcht 13y ago

Regarding hidden in plain sight, would a semicolon at the end of the if statement throw a warning? Would be a little more hidden.

huhtenberg 13y ago
Yeah, that'd trigger a warning in most compilers - "Empty statement found. Was it an intent?" or something similar.
- valleyer 13y ago
  Not true, try for (;;) ; in gcc / clang
  - SimHacker 13y ago
    That's because "for (;;) ;" is and idiom that's commonly used in a programming pattern called the "for(;;);ce field", to work around other C and C preprocessor quirks, by syntactically insulating and protecting statements in C preprocessor macros from outside interference. As a matter of fact, by deploying one of those the beginning of the macro, one in the middle, and one at the end, you can set up what's called a "trifor(;;);ce field", an anti-anti-pattern that is documented on the c2 wiki thusly: The trifor(;;);ce has the power to grant the wish of whomever touches it, and molds the Sacred Realm to reflect that person's heart. The trifor(;;);ce does not discriminate between "good" or "evil". However, if a person without an equal balance of power, wisdom, and courage makes a wish, the trifor(;;);ce will split into its three separate parts: the piece that best personifies the wish-maker will be the only piece to remain in hand, whilst the other two will take residence in whosoever most personifies them. Reassembly is then required for such a person's wish to be granted, but does not exclude another from making the attempt. No, I'm just joking and making shit up. Don't anyone ever do that! As bitchy and pedantic as gcc and clang are about other things, I'm disappointed they don't complain about that.

blazespin 13y ago

It was likely someone subverting the process for reasons of urgency, for good intentions or malice. Simple static analysis or unit tests would have caught this.

patricjansson 13y ago
Exactly! So strange apple didnt have a testcase for this.
- jeffaustin 13y ago
  Yea, and we're supposed to believe that no QA person did any testing on this either.

tlrobinson 13y ago

Has the author of the buggy code been identified?

deleted 13y ago
[deleted]

corresation 13y ago

3) Hidden in plain sight I'm baffled that this wasn't caught long, long ago. Most of us have worked with internal systems where certs, for whatever reason, don't match the site, and it's surprising many people haven't noted that such doesn't raise any errors on iOS/OSX. Though thinking back....I actually remember encountering exactly that on the iPad once, surprised that it didn't raise a flag. Like probably most I just shrugged and continued on.

mehrdada 13y ago
Because it does not work like that. SecureTransport actually verifies the validity of the TLS certificate. If you hand it an invalid certificate in the first place, it will complain. The issue is one level deeper. When you connect to a server that supports ephemeral key exchange, the parties involved will generate new key pairs on the fly to use for that connection. In order to make sure that the server has the key published in its certificate, the ephemeral public key shown to the client must be signed by the static private key of the certificate and the client MUST verify it. It is the ephemeral key signature that is not validated, not the signature on the certificate itself. What happens here is that the server gives the client a _valid_ certificate for which it does not own the private key, and in the ephemeral step, it simply generates a key pair without a valid signature. At no point in the handshake the server is asked to verify its ownership of the private key associated to the certificate it is presenting To summarize, TLS certificate should be valid, but you are never asked to verify you have its private key when the connection used DHE or ECDHE.
- corresation 13y ago
  Thank you this is illuminating.

gnyman 13y ago

Does not NSA have access to multiple of the private keys of the preinstalled root certificates anyways? If so they can MITM any SSL connection where you don't specifically say which CA cert it should use? Isn't it the case anyways that SSL is not very efficient against state sponsored attacks as almost all of them can generate certificates for any domain? All this speaks towards it being a unfortunate mistake not something malicious, unless there is something wrong with my understanding how things work :-)

wreegab 13y ago

I believe the faulty function -- at line 574 -- is a copy/paste of another function located at line 328 in the same file. I diff'ed both functions to try to understand the changes which may have been made. The "goto fail;" is somewhat "drowned" into other changes around, so it stands out less using a diff tool because of these surrounding changes. The surrounding changes in question: 1) Moving a "SSLFreeBuffer(&hashCtx)" instruction at a higher position: Very strange since it accomplish absolutely nothing at its new position, as opposed to it's original position. 2) Renaming of two variables: "hash" became "hashOut" and "exchangeParams" became "signedParams". 3) Indentation of "goto"s Without the above changes, the "goto fail;" stand out rather well when using a diff tool.