4 ms·

It is neither pointless nor content-less. It gets to the exact heart of the problems with ESR's claim. If Joe Shmoe had said it, people would not say "no shit

by copergi 13y ago

It is neither pointless nor content-less. It gets to the exact heart of the problems with ESR's claim. If Joe Shmoe had said it, people would not say "no shit", they would still say "no that is complete nonsense". Because it is in fact complete nonsense, for exactly the reasons Theo pointed out. All the eyes in the world don't amount to anything if they aren't being used to audit code. And to do that, they need to be attached to people capable of auditing code. ESR is simply being used as an example of his hypothesis being wrong since he is the most obvious example. It isn't attacking him to point out the flaw in his hypothesis.

Crito 13y ago
"Eyes" obviously refers to "eyes of people auditing the code", not "eyes in skulls of people who are currently in the building", not "eyes sitting in jars in the Mutter Museum", not "every eyeball belonging to children currently living in Idaho". Furthermore, different eyeballs of course have different worth. Nobody in the world disputes that. The law itself doesn't say anything about whether or not people will choose to examine the code. It just says that more people examining the code is better than fewer people examining the code. One audit is better than none. Two is better than one. Three is better than two. One shitty auditor is better than none. Two shitty auditors are better than one. One auditor twice as good as a shitty auditor is better than one shitty auditor, if you really want to get into tedious eyeball calculus. Etc. Seems stupidly obvious; so obvious it isn't even worth stating? That's because it is. Or to address your point another way: If I said "You need fuel to make your car go." would you object that this is bullshit because not only do you need fuel, but it needs to be in your car, and the right sort of fuel? I don't think you would say, "But diesel fuel sitting in a puddle on the ground is useless and won't make your petrol car go anywhere!"
- stan_rogers 13y ago
  The point you seem to be deliberately missing is that the opportunity for something to happen and something happened are not equivalent. There is a natural assumption that because there is the opportunity for many people (however you decide to define "many") to review/audit/test code in open source projects, somebody must be doing so. Even when you base that assumption on the number of contributors to the project, non sequitur; feature addition/refinement does not imply bug fixing.
  - SimHacker 13y ago
    The NSA has lots of eyes, and they don't report the bugs they find. (Unless it's via parallel reconstruction...)
- copergi 13y ago
  Using analogies to avoid dealing with the actual topic is not productive. ESR claimed that open source projects are inherently more secure because "many eyes make all bugs shallow". That is what we are talking about. Not cars, not airliners. His hypothesis relies on the (false) assumption that many eyes are looking at the code simply because it is open source. That is not the case, as obvious security holes like these demonstrate time and again. It takes dedicated, qualified people spending significant time auditing code to make it more secure, not being open source. A closed source project with 1 security expert auditing the code is more secure than an open source project with 0 security experts auditing the code and a million users downloading the source and compiling it.
- deleted 13y ago
  [deleted]
- einhverfr 13y ago
  Another thought occurs to me. It isn't clear to me how much reading the source actually helps with finding most security vulnerabilities. It does seem like some of these recent ones in SSL have been found by detailed code review (after years of providing bypasses), but I mean in general. It's remarkably hard to find a security problem reading the code unless that security problem is blatantly obvious. Part of the problem is that code also communicates intent, and security is in part counteracting that intent. Most (more than 90%) of the security problems I found (almost all of which were in open source prjects) started off by observing misbehavior or wondering if I could induce misbehavior. This works because I would start off attacking an API, not the underlying implementation. Only after showing the problem would I look at the code. I think on one case, I was looking at code and thought "maybe that's a weak point." But in every other case, my testing was fully black-box to start (things later lead to a code audit in several cases).
  - clarry 13y ago
    > It's remarkably hard to find a security problem reading the code unless that security problem is blatantly obvious. Then I guess I would consider most (C code) security problems rather obvious. The kind of stuff you see patches linked to in CVEs.. good old buffer overflows, off by ones, arithmetic screwups (mixing types or not checking for overflows), missing return value checks, the occasional swapped parameter, typo, simple logic whoopsie, etc.. These are incredibly common and rather easy to find once you get used to that sort of grunt work. But obviously you have to know what to look for. The remarkably hard ones (for me anyway) are much more subtle, things like race conditions...
    - SimHacker 13y ago
      Using min where I mean max and max where I mean min is my favorite classic mistake that I usually just make most of the time and remember to check for and fix afterwards. The same thing with misusing "this" in JavaScript closures: no matter how hard I try to consciously avoid doing it, it's always the first thing I check for when code doesn't work as intended, because I still make that same mistake all the time. Another source of confusion in code that Ben Shneiderman pointed out, is that the meaning of "and" and "or" are opposite to programmers and normal human beings, so languages like SQL and logical expressions in most programming languages are fundamentally confusing to many people, and the inconsistency can be an un-obvious blind spot. Normal humans mean to union when they say "this AND that AND those AND these", while programmers mean to intersect when they say "this AND that AND those AND these". So that's a terrible source of confusion. In other words, programmers think adding "AND" clauses narrows down the selection like "((user.age >= 18) AND (user.gender = 'F'))", while humans think adding "AND" clauses augments the selection like "adults AND women". I'm not advocating changing the meaning of "AND" and "OR", just pointing out that it's a source of confusion you should look out for, and be aware of when talking with non-programmers. I saw Alvy Ray Smith give a talk about 3D graphics programming, and he confessed that he and his colleagues would just write some code and then flip the signs around until it did what they meant it to do. That made me feel a lot less embarrassed about doing that myself.
- SimHacker 13y ago
  Don't let your mouth write a check your eyeballs can't keep. And I'd rather be an asshole with eyeballs than a mouth full of bullshit.

by copergi 13y ago

Crito 13y ago

"Eyes" obviously refers to "eyes of people auditing the code", not "eyes in skulls of people who are currently in the building", not "eyes sitting in jars in the Mutter Museum", not "every eyeball belonging to children currently living in Idaho". Furthermore, different eyeballs of course have different worth. Nobody in the world disputes that. The law itself doesn't say anything about whether or not people will choose to examine the code. It just says that more people examining the code is better than fewer people examining the code. One audit is better than none. Two is better than one. Three is better than two. One shitty auditor is better than none. Two shitty auditors are better than one. One auditor twice as good as a shitty auditor is better than one shitty auditor, if you really want to get into tedious eyeball calculus. Etc. Seems stupidly obvious; so obvious it isn't even worth stating? That's because it is. Or to address your point another way: If I said "You need fuel to make your car go." would you object that this is bullshit because not only do you need fuel, but it needs to be in your car, and the right sort of fuel? I don't think you would say, "But diesel fuel sitting in a puddle on the ground is useless and won't make your petrol car go anywhere!"

stan_rogers 13y ago
The point you seem to be deliberately missing is that the opportunity for something to happen and something happened are not equivalent. There is a natural assumption that because there is the opportunity for many people (however you decide to define "many") to review/audit/test code in open source projects, somebody must be doing so. Even when you base that assumption on the number of contributors to the project, non sequitur; feature addition/refinement does not imply bug fixing.
- SimHacker 13y ago
  The NSA has lots of eyes, and they don't report the bugs they find. (Unless it's via parallel reconstruction...)
copergi 13y ago
Using analogies to avoid dealing with the actual topic is not productive. ESR claimed that open source projects are inherently more secure because "many eyes make all bugs shallow". That is what we are talking about. Not cars, not airliners. His hypothesis relies on the (false) assumption that many eyes are looking at the code simply because it is open source. That is not the case, as obvious security holes like these demonstrate time and again. It takes dedicated, qualified people spending significant time auditing code to make it more secure, not being open source. A closed source project with 1 security expert auditing the code is more secure than an open source project with 0 security experts auditing the code and a million users downloading the source and compiling it.
deleted 13y ago
[deleted]
einhverfr 13y ago
Another thought occurs to me. It isn't clear to me how much reading the source actually helps with finding most security vulnerabilities. It does seem like some of these recent ones in SSL have been found by detailed code review (after years of providing bypasses), but I mean in general. It's remarkably hard to find a security problem reading the code unless that security problem is blatantly obvious. Part of the problem is that code also communicates intent, and security is in part counteracting that intent. Most (more than 90%) of the security problems I found (almost all of which were in open source prjects) started off by observing misbehavior or wondering if I could induce misbehavior. This works because I would start off attacking an API, not the underlying implementation. Only after showing the problem would I look at the code. I think on one case, I was looking at code and thought "maybe that's a weak point." But in every other case, my testing was fully black-box to start (things later lead to a code audit in several cases).
- clarry 13y ago
  > It's remarkably hard to find a security problem reading the code unless that security problem is blatantly obvious. Then I guess I would consider most (C code) security problems rather obvious. The kind of stuff you see patches linked to in CVEs.. good old buffer overflows, off by ones, arithmetic screwups (mixing types or not checking for overflows), missing return value checks, the occasional swapped parameter, typo, simple logic whoopsie, etc.. These are incredibly common and rather easy to find once you get used to that sort of grunt work. But obviously you have to know what to look for. The remarkably hard ones (for me anyway) are much more subtle, things like race conditions...
  - SimHacker 13y ago
    Using min where I mean max and max where I mean min is my favorite classic mistake that I usually just make most of the time and remember to check for and fix afterwards. The same thing with misusing "this" in JavaScript closures: no matter how hard I try to consciously avoid doing it, it's always the first thing I check for when code doesn't work as intended, because I still make that same mistake all the time. Another source of confusion in code that Ben Shneiderman pointed out, is that the meaning of "and" and "or" are opposite to programmers and normal human beings, so languages like SQL and logical expressions in most programming languages are fundamentally confusing to many people, and the inconsistency can be an un-obvious blind spot. Normal humans mean to union when they say "this AND that AND those AND these", while programmers mean to intersect when they say "this AND that AND those AND these". So that's a terrible source of confusion. In other words, programmers think adding "AND" clauses narrows down the selection like "((user.age >= 18) AND (user.gender = 'F'))", while humans think adding "AND" clauses augments the selection like "adults AND women". I'm not advocating changing the meaning of "AND" and "OR", just pointing out that it's a source of confusion you should look out for, and be aware of when talking with non-programmers. I saw Alvy Ray Smith give a talk about 3D graphics programming, and he confessed that he and his colleagues would just write some code and then flip the signs around until it did what they meant it to do. That made me feel a lot less embarrassed about doing that myself.
SimHacker 13y ago
Don't let your mouth write a check your eyeballs can't keep. And I'd rather be an asshole with eyeballs than a mouth full of bullshit.