3 ms·

Once you know that your code has been corrupted, it would take a team of consultants put together by someone like ptacek a relatively short period of time to fi

Once you know that your code has been corrupted, it would take a team of consultants put together by someone like ptacek a relatively short period of time to find the holes (and, at the same time, discover 10x or more code weaknesses that weren't deliberately put there.) If the toolchain wasn't corrupted, and this was in the original source code (C?) - it's not going to be too difficult to find in code review, if you are looking for it - but how many companies are willing to spend the time (and $$$) to review their presumed secure code for holes added by third-parties? Now that Juniper knows that their source-code/tool chain has been breached, they will be prepared to spend the money to clean it up - which, for any reasonable amount of code, will cost millions of $$$ if not tens of millions. I'm wondering if patio11/ptacek didn't show some pretty amazing foresight in tracking down the resumes of the top security intrusion people in the industry - I have to believe that the Ciscos, Junipers, and other security hardware/software companies in the world are going to be in desperate need of Top People, and are going to be ready to pay large sums of $$$ for fast service in this space.

lazaroclapp 11y ago
> If the toolchain wasn't corrupted, and this was in the original source code (C?) - it's not going to be too difficult to find in code review, if you are looking for it I was part of a research project, in which, as part of the evaluation, we were given small Android applications (1-10k lines of code, not obfuscated). We were told that each of the apps contained a single piece of embedded malicious functionality hidden in its otherwise known/innocuous behavior. We were given a few weeks to find the malicious functionality. The main goal was to use our prototype software analysis tools. But in those cases in which we had to perform manual source analysis, the implanted functionality often eluded us for 10-25% of the cases. Now, you might be able to find people who are more experienced at software auditing than us (a bunch of Ph.D. students in security and compiler-related areas). But you might also find more experienced implant writers as well. Also, in a case like this, you don't know how many implants there are, how much of the internal company network is also compromised, how many (if any) of your remaining employees is an attacker, etc.
- ghshephard 11y ago
  Fair point - but I'm thinking that a company like Juniper, which has critical branding and revenue on the lines, can afford to hire the 100 top-of-the-line pen testers for 90 days to scrub through their code (with various members have different areas of speciality - which would be important in a situation like ScreenOS which has so much breadth). Figure a top-end pen tester goes for around $300-$400K/year + 50% consulting Markup - $600K/year works out to $150K/90 days * 100 pen testers = $15mm. The level of performance out of a $400k/year pen tester is pretty good. 100 of them can really scrub a code base pretty well. $15mm might sounds like a lot of money to you and I - but you would be amazed to see how quickly the money flows when there is a security issue at stake. Also - since the Target CEO (http://www.cbc.ca/news/business/tony-fisher-fired-as-target-canada-president-1.2647934 http://www.cbc.ca/news/business/tony-fisher-fired-as-target-...) got axed as a result of a security event, budgeting for security responses has strong CxO level support. In addition to the consulting workforce you can pull in for this type of project, you also presumably have a highly motivated internal workforce that will be pulling long hours, and working hard to support the pen testers as well.
  - dogma1138 11y ago
    You won't find that many pen testers making 600K a year, I don't think you find a single one that's paid that much for actual technical delivery. Pen-testers are also not the people that should be doing code reviews necessarily as they most likely lack experience in that regard. Money doesn't solve everything, you can bring out 200 people and they'll still won't find everything and being able to figure out how everything works. The hardcoded password backdoor was well obfuscated but at least something still "easily" detectable the Dual-EC vuln where some one replaced the values with their own pre-computed pair I don't even want to know how they found that out other than by chance. And when you'll start going deeper It's just gets more complicated It won't be easy to find a 100 people that will be able to do code review on say the kernel of ScreenOS world wide, heck considering the specific skill set that they'll need to have: being able to understand the source code, assembly, being able to figure out how if it can be leveraged in any way which will affect the security of product on a software or hardware level (we are dealing with mother of all registers here after all) and being available for hire that's not something that trivial. And lastly this might not be solvable with money at all identifying backdoors in unknown code is very difficult the technology that can do so is very limited and the manual approach is prone to human errors and oversight. Even if you get a team of a 100 leading code review, malware and security experts in the world I would not bet money that they'll be able to scrub anything within a year. And I would not even attempt to hire a 100 of them, the time it will take to bring them up to speed would be pretty exhaustive, a safer bet would be to have your own developers which worked on the products from year to go over small parts of the code and flag something unusual. If Juniper is smart they'll crowd source that internally and have various code snippets pop up on the screens of randomly selected developers and have them being evaluated, when a snippet gets enough flags forward it to an expert and have them review it. But that again only works if a) the backdoor is localized and not staged across various steps in the logic flaw it self, and b) the backdoor is in the actual source code and not only present in the binaries because the tool chain itself was compromised. Life isn't simple.

lazaroclapp 11y ago

> If the toolchain wasn't corrupted, and this was in the original source code (C?) - it's not going to be too difficult to find in code review, if you are looking for it I was part of a research project, in which, as part of the evaluation, we were given small Android applications (1-10k lines of code, not obfuscated). We were told that each of the apps contained a single piece of embedded malicious functionality hidden in its otherwise known/innocuous behavior. We were given a few weeks to find the malicious functionality. The main goal was to use our prototype software analysis tools. But in those cases in which we had to perform manual source analysis, the implanted functionality often eluded us for 10-25% of the cases. Now, you might be able to find people who are more experienced at software auditing than us (a bunch of Ph.D. students in security and compiler-related areas). But you might also find more experienced implant writers as well. Also, in a case like this, you don't know how many implants there are, how much of the internal company network is also compromised, how many (if any) of your remaining employees is an attacker, etc.

ghshephard 11y ago
Fair point - but I'm thinking that a company like Juniper, which has critical branding and revenue on the lines, can afford to hire the 100 top-of-the-line pen testers for 90 days to scrub through their code (with various members have different areas of speciality - which would be important in a situation like ScreenOS which has so much breadth). Figure a top-end pen tester goes for around $300-$400K/year + 50% consulting Markup - $600K/year works out to $150K/90 days * 100 pen testers = $15mm. The level of performance out of a $400k/year pen tester is pretty good. 100 of them can really scrub a code base pretty well. $15mm might sounds like a lot of money to you and I - but you would be amazed to see how quickly the money flows when there is a security issue at stake. Also - since the Target CEO (http://www.cbc.ca/news/business/tony-fisher-fired-as-target-canada-president-1.2647934 http://www.cbc.ca/news/business/tony-fisher-fired-as-target-...) got axed as a result of a security event, budgeting for security responses has strong CxO level support. In addition to the consulting workforce you can pull in for this type of project, you also presumably have a highly motivated internal workforce that will be pulling long hours, and working hard to support the pen testers as well.
- dogma1138 11y ago
  You won't find that many pen testers making 600K a year, I don't think you find a single one that's paid that much for actual technical delivery. Pen-testers are also not the people that should be doing code reviews necessarily as they most likely lack experience in that regard. Money doesn't solve everything, you can bring out 200 people and they'll still won't find everything and being able to figure out how everything works. The hardcoded password backdoor was well obfuscated but at least something still "easily" detectable the Dual-EC vuln where some one replaced the values with their own pre-computed pair I don't even want to know how they found that out other than by chance. And when you'll start going deeper It's just gets more complicated It won't be easy to find a 100 people that will be able to do code review on say the kernel of ScreenOS world wide, heck considering the specific skill set that they'll need to have: being able to understand the source code, assembly, being able to figure out how if it can be leveraged in any way which will affect the security of product on a software or hardware level (we are dealing with mother of all registers here after all) and being available for hire that's not something that trivial. And lastly this might not be solvable with money at all identifying backdoors in unknown code is very difficult the technology that can do so is very limited and the manual approach is prone to human errors and oversight. Even if you get a team of a 100 leading code review, malware and security experts in the world I would not bet money that they'll be able to scrub anything within a year. And I would not even attempt to hire a 100 of them, the time it will take to bring them up to speed would be pretty exhaustive, a safer bet would be to have your own developers which worked on the products from year to go over small parts of the code and flag something unusual. If Juniper is smart they'll crowd source that internally and have various code snippets pop up on the screens of randomly selected developers and have them being evaluated, when a snippet gets enough flags forward it to an expert and have them review it. But that again only works if a) the backdoor is localized and not staged across various steps in the logic flaw it self, and b) the backdoor is in the actual source code and not only present in the binaries because the tool chain itself was compromised. Life isn't simple.