4 ms·
Maybe the silver lining here is that it puts the final nail in the coffin for "many eyes make all bugs shallow" - which was always total BS from the day it was
by notacoward 13y ago
Maybe the silver lining here is that it puts the final nail in the coffin for "many eyes make all bugs shallow" - which was always total BS from the day it was uttered. There's so much code out there, much of it highly specialized and even project-specific, that there are very few eyes looking at any particular piece of code, and not all eyes are connected to the greatest of brains.
Most static code analyzers could have caught this particular bug, as it requires only a fairly simple kind of reasoning about allocated vs. used length. In fact I believe it probably was found by static analysis before being reported by a human, which is a shame because it misses an opportunity to highlight the value of such tools. Maybe next time people will spend a little less time designing a logo and a little more time doing things that actually help (though that's a wish and what I expect is the exact opposite).
The real lesson here is that we should apply as many different tools and processes as we can at improving code quality for critical infrastructure. Code reviews are nice. Static analysis is nice. Detailed tests are nice. However, none of these alone is sufficient even when pursued with fanatical devotion.
- nikcub 13y ago> Maybe the silver lining here is that it puts the final nail in the coffin for "many eyes make all bugs shallow" The OpenSSL project doesn't seem to have a 'review' step[0] in code commits, unlike FreeBSD, OpenBSD and Chromium (i'm just picking projects i'm familiar with, no bias). So it might not have been 'many eyes', but the eyes of just one. edit: [0] a proper review step, what the openssl project does is have feedback on commits (which is what the OP article is referring to). see below.
- sp332 13y agoThe article says there was a reviewer who looked at this code before it was moved to the release branch.
- nikcub 13y agoIt refers to 'review' as in reviewing the features and how it is implemented. Here is the log: http://rt.openssl.org/Ticket/Display.html?id=2658&user=guest&pass=guest http://rt.openssl.org/Ticket/Display.html?id=2658&user=guest... That is very far from it being a security review, or any real proper review. It is more a request for comments (which defaults to code being committed) You'll notice that the workflow for commits doesn't have a review stage, and it isn't implemented in the workflow software. Compare to a chromium example, where each commit, eg. http://src.chromium.org/viewvc/chrome?view=revision&revision=262978 http://src.chromium.org/viewvc/chrome?view=revision&revision... has a review stage that the original committer doesn't have permissions for. In this case 3 reviewers for that commit: https://codereview.chromium.org/231923002 https://codereview.chromium.org/231923002 Code doesn't get committed without at least 4 people, its like a 2-man rule that is enforced in the workflow. In contrast the openssl process is that the same committer can push the code after getting no objections. There is a big difference in how each of these processes work.
- notacoward 13y agoThe "many eyes" canard isn't just about formal reviews, though. It's also about the idea that when something's open source the reviews aren't limited to maintainers or close colleagues who would be involved regardless of open vs. proprietary. What actually seems to happen in the great majority of cases is that the circle of people who actually understand any particular commit is only very slightly greater than it would be otherwise. The bugs are only a very tiny bit shallower, if at all. The fact that it was just one reviewer in this case, despite the code being widely available, only underscores that point.
- dangayle 13y ago>> Maybe next time people will spend a little less time designing a logo and a little more time doing things that actually help (though that's a wish and what I expect is the exact opposite). So you want the graphic designers contributing to crypto code? As someone who is a mid level programmer but a decent/good designer, the only way I'm able to contribute to some projects is through things like logos. Don't use this straw man argument to make your point, which is otherwise valid.
- tptacek 13y agoIt sure would be nice if some of those designers would take a whack at the Mozilla certificate configuration UI. It hasn't meaningfully changed in a decade, it's one of the Internet's biggest security flaws, and it's something any UX person could safely propose changes to.
- sleevi 13y ago"It's one of the Internet's biggest security flaws" - aww, you're killing me here :) We have looked at it. Repeatedly. First when porting Chrome to Linux (and trying to decide what sort of UI there should be for NSS cert management). Then for ChromeOS. And then again, revisiting it in light of the OpenSSL migration. The reality is, the certificate management/configuration UIs, of all platforms (Windows, OS X, Linux-via-Firefox-and-Chrome-which-just-copies-Firefoxes-PSM), is hard. It's hard to express the decisions. It's hard to explain how the branching and cross-certifications of CAs can affect trust decisions (eg: recall DigiNotar's cross-certifications). And in the end, it's not where users are spending their time/being tripped up. Getting the lock icon into a more meaningful state is a far, far greater investment, with far, far more actionable returns in users security. The work of the Chrome Security Enamel team on this - especially Adrienne Porter Felt - has been focused on trying to improve this.
- tptacek 13y agoI understand why you're writing this. You think I'm saying, "the UI that users use to modify the root certificate store is so bad that it's the Internet's worst problem". I'm not saying that. I'm saying that the fact that there's only a UI to configure your root store, and not a UI to: * set policy for which CAs you trust with which domains * subscribe to policies published by trusted third policies * monitor which sites bind you to which CAs * easily opt in and out of trusting different CAs ... is the big problem. It's a UX problem, not a simple UI problem. There are additional features that need to be added, and that can be added, without litigating the whole CA system.
- deleted 13y ago[deleted]
- orblivion 13y ago> Most static code analyzers could have caught this particular bug Does that mean nobody out there in the crypto world ran a static analyzer on OpenSSL in the last 2 years?
- nuncanada 13y agoOpenSSL's code uses particularly bad MACRO's ifdefs that defeat static analyzer's capabilities to analyze the code...
- notacoward 13y agoI'm not sure how much those macros affect anything here. For one thing, most static analyzers are actually very good at cutting through that kind of crap, running its analyses on code that has already been pre-processed using exactly the same command-line flags as the real compiler saw. Even if that weren't the case, I think this particular bug would still qualify as low-hanging fruit. It doesn't involve a lot of macros. It doesn't involve dynamically assigned function pointers. It's not limited to one execution through multiple iterations of a complex loop. I'm painfully familiar with the constructs that can defeat static analysis, and none of them seem present in this case. The code allocates a buffer of size N, then reads from an offset that's not checked to be less than N. That's kind of Static Analysis 101. My biggest worry here, TBH, is that static analysis was done, and this was flagged, but it was buried among so many other reports - many of them false negatives - that nobody paid any attention. That would be truly sad.
- notacoward 13y agoMeant to say "false positives" but it's too late to edit. IRTE
- DanBC 13y ago... And wasn't the Debian bug caused by someone running a tool against the code and trusting the tool?
- 13y ago
- tfinniga 13y agoA lot of what Linus says only really applies to linux kernel development. And some of that only applies when you're Linus.
- anonbanker 13y agoI don't know; this seems like a clear-cut case of "not enough eyes" to me.