5 ms·

What a pointless and content-less rebuttal. I mean shit, I don't need to be an aeronautical engineer or work in the FAA to say that several people investigatin

by Crito 13y ago

What a pointless and content-less rebuttal. I mean shit, I don't need to be an aeronautical engineer or work in the FAA to say that several people investigating an airliner crash will be more effective than one single dude sifting through a literal field of debris. ESR's quote isn't wrong (perhaps hyperbolic, but in spirit it isn't wrong). It's just an uninteresting observation. What do you do when something has you stumped at work? You ask the guy across the hall to look over your shoulder for a second... It's just common sense. People simultaneously revere and hate the quote only because ESR said it. If Joe Shmoe had said it all those years ago, it would have just been met with "no shit" and never given second thought.

smsm42 13y ago
I think there's a point here. The point being much more people talk about many eyes than actually audit existing code. The fact is, there are quite a number of bugs that stayed in the code for a long time despite being not that deep. Maybe that says we don't have enough eyeballs, but most of people quoting that sentence quietly assume we do, equating number of participants in FOSS communities with number of people actually auditing the code. Unfortunately, the former is much bigger than the latter. We pretty much have the same situation as we have in science publications where a lot of results are not reproducible and a lot nobody even bothers to check for reproducibility, despite reproducibility being the basic tenet of open publication and peer review and being necessary to validate the result. Unless there's a reason to - e.g. an identifiable bug - not many bother to go back to existing code and check it for correctness.
- Crito 13y ago
  The quote is hyperbolic for the "all", and should probably use the word "shallower", but otherwise I don't see anything wrong with it. "much more people talk about many eyes than actually audit existing code" is true of free software, but is also true of just about everything. More people think "I feel comfortable crossing this bridge because plenty of engineers have looked at it" than there are engineers actually looking at bridges. I haven't read the quote in it's original context for several years, but I don't remember it conflating software users with software developers.
  - Perseids 13y ago
    The difference is that engineers actually know how to build safe bridges. I feel (and am actually) safe living in a house designed by a single competent architect, but I am not secure using a TLS implementation written by a single competent programmer. The only way we have found to end up with reasonable secure code is to have it reviewed by a lot of competent people.
- TheCondor 13y ago
  There are a dozen points here... The shallow bugs statement is true when there is a community that is large and with incredibly deep understanding; linux and posix is a great example, there is staggering depth of knowledge on those interfaces and behavior. Unfortunately there isn't wide and deep knowledge of tls architecture and implementation, there isn't even a lot of commonality across interfaces. There is a social axiom that you and I don't know crypto and we should leave it to the experts, yet they need help too. There is conventional c style and this function, while documented, does the opposite. I had to look at the code a couple times to see the bug, a lot of reviews could have missed it. There is conventional c style and the whole failure chain cleanup mess, look at that code again, they've got their own data representation and free function that uses it to detect if memory is allocated, the free is private but the initialize is done inline. That stuff happens everywhere by many projects, I'm not saying its wrong but it leaves it open for easy bugs, one gaffed variable declaration potentially screws everything and you need to know the data structure even though you don't directly use it. And I don't want to pile on gnutls, I think their intentions are good and this is a bug, but this looks fucking prime for unit testing... There are a lot of variables to determine and measure a projects health, the whole community needs to step up in the quality department, there are lots of ways to contribute and despite the adage, more people should look at crypto.
- TheCondor 13y ago
  There are a dozen points here... The shallow bugs statement is true when there is a community that is large and with incredibly deep understanding; linux and posix is a great example, there is staggering depth of knowledge on those interfaces and behavior. Unfortunately there isn't wide and deep knowledge of tls architecture and implementation, there isn't even a lot of commonality across interfaces. There is a social axiom that you and I don't know crypto and we should leave it to the experts, yet they need help too. There is conventional c style and this function, while documented, does the opposite. I had to look at the code a couple times to see the bug, a lot of reviews could have missed it. There is conventional c style and the whole failure chain cleanup mess, look at that code again, they've got their own data representation and free function that uses it to detect if memory is allocated, the free is private but the initialize is done inline. That stuff happens everywhere by many projects, I'm not saying its wrong but it leaves it open for easy bugs, one gaffed variable declaration potentially screws everything and you need to know the data structure even though you don't directly use it. And I don't want to pile on gnutls, I think their intentions are good and this is a bug, but this looks fucking prime for unit testing... There are a lot of variables to determine and measure a projects health, the whole community needs to step up in the quality department, there are lots of ways to contribute and despite the adage, more people should look at crypto.
- TheCondor 13y ago
  There are a dozen points here... The shallow bugs statement is true when there is a community that is large and with incredibly deep understanding; linux and posix is a great example, there is staggering depth of knowledge on those interfaces and behavior. Unfortunately there isn't wide and deep knowledge of tls architecture and implementation, there isn't even a lot of commonality across interfaces. There is a social axiom that you and I don't know crypto and we should leave it to the experts, yet they need help too. There is conventional c style and this function, while documented, does the opposite. I had to look at the code a couple times to see the bug, a lot of reviews could have missed it. There is conventional c style and the whole failure chain cleanup mess, look at that code again, they've got their own data representation and free function that uses it to detect if memory is allocated, the free is private but the initialize is done inline. That stuff happens everywhere by many projects, I'm not saying its wrong but it leaves it open for easy bugs, one gaffed variable declaration potentially screws everything and you need to know the data structure even though you don't directly use it. And I don't want to pile on gnutls, I think their intentions are good and this is a bug, but this looks fucking prime for unit testing... There are a lot of variables to determine and measure a projects health, the whole community needs to step up in the quality department, there are lots of ways to contribute and despite the adage, more people should look at crypto.
- TheCondor 13y ago
  There are a dozen points here... The shallow bugs statement is true when there is a community that is large and with incredibly deep understanding; linux and posix is a great example, there is staggering depth of knowledge on those interfaces and behavior. Unfortunately there isn't wide and deep knowledge of tls architecture and implementation, there isn't even a lot of commonality across interfaces. There is a social axiom that you and I don't know crypto and we should leave it to the experts, yet they need help too. There is conventional c style and this function, while documented, does the opposite. I had to look at the code a couple times to see the bug, a lot of reviews could have missed it. There is conventional c style and the whole failure chain cleanup mess, look at that code again, they've got their own data representation and free function that uses it to detect if memory is allocated, the free is private but the initialize is done inline. That stuff happens everywhere by many projects, I'm not saying its wrong but it leaves it open for easy bugs, one gaffed variable declaration potentially screws everything and you need to know the data structure even though you don't directly use it. And I don't want to pile on gnutls, I think their intentions are good and this is a bug, but this looks fucking prime for unit testing... There are a lot of variables to determine and measure a projects health, the whole community needs to step up in the quality department, there are lots of ways to contribute and despite the adage, more people should look at crypto.
ori_b 13y ago
> I mean shit, I don't need to be an aeronautical engineer or work in the FAA to say that several people investigating an airliner crash will be more effective than one single dude sifting through a literal field of debris. With zero people looking at airliner failures, nothing will be discovered, no matter how many inspectors the FAA has on its payroll. In the software world, there are almost no people doing code audits. Bugs and security holes are left in live code for years. Basically, Theo is saying "Shut up and audit." How much open source code have you audited for bugs recently? How many subtle correctness issues have you found in projects you've looked at? For the sake of code quality, I certainly hope it's higher than the amount I've audited.
- Crito 13y ago
  "Linus's Law" itself says little about whether or not people will actually look at code. It just says that more people looking is better than fewer people looking. How much code I have audited, you have audited, ESR has audited, or hell, how much code Paul McCartney has audited, really has little to do with the obvious correctness and banality of the 'law'.
  - ori_b 13y ago
    In other words, your interpretation of "Linus's Law" has no impact on code quality in the real world. Edit: However, since it's meant to imply that in open source, there are people looking at code, I think that the criticisms stand.
    - Crito 13y ago
      Yes, in the sense that Crito's Law (coined by me, now) "The bigger the ship, the more freight you can fit in it." has no impact on the real-world ship transport industry. It is obviously true to the point of being banal. It is a pointless statement of uncontroversial fact that provides next to no utility to anybody. It's not even interesting for being a tautology. If I were a particularly objectionable and self-promoting person, then perhaps people might object to Crito's Law whenever it were quoted on shipping forums, but that wouldn't make it incorrect. Nor would my shameless self-promotion make it profound. (Is Crito's Law _precisely_ true? Well no, some large ships are not designed for freight after all... but the general principle is true.)
copergi 13y ago
It is neither pointless nor content-less. It gets to the exact heart of the problems with ESR's claim. If Joe Shmoe had said it, people would not say "no shit", they would still say "no that is complete nonsense". Because it is in fact complete nonsense, for exactly the reasons Theo pointed out. All the eyes in the world don't amount to anything if they aren't being used to audit code. And to do that, they need to be attached to people capable of auditing code. ESR is simply being used as an example of his hypothesis being wrong since he is the most obvious example. It isn't attacking him to point out the flaw in his hypothesis.
- Crito 13y ago
  "Eyes" obviously refers to "eyes of people auditing the code", not "eyes in skulls of people who are currently in the building", not "eyes sitting in jars in the Mutter Museum", not "every eyeball belonging to children currently living in Idaho". Furthermore, different eyeballs of course have different worth. Nobody in the world disputes that. The law itself doesn't say anything about whether or not people will choose to examine the code. It just says that more people examining the code is better than fewer people examining the code. One audit is better than none. Two is better than one. Three is better than two. One shitty auditor is better than none. Two shitty auditors are better than one. One auditor twice as good as a shitty auditor is better than one shitty auditor, if you really want to get into tedious eyeball calculus. Etc. Seems stupidly obvious; so obvious it isn't even worth stating? That's because it is. Or to address your point another way: If I said "You need fuel to make your car go." would you object that this is bullshit because not only do you need fuel, but it needs to be in your car, and the right sort of fuel? I don't think you would say, "But diesel fuel sitting in a puddle on the ground is useless and won't make your petrol car go anywhere!"
  - stan_rogers 13y ago
    The point you seem to be deliberately missing is that the opportunity for something to happen and something happened are not equivalent. There is a natural assumption that because there is the opportunity for many people (however you decide to define "many") to review/audit/test code in open source projects, somebody must be doing so. Even when you base that assumption on the number of contributors to the project, non sequitur; feature addition/refinement does not imply bug fixing.

by Crito 13y ago

smsm42 13y ago

I think there's a point here. The point being much more people talk about many eyes than actually audit existing code. The fact is, there are quite a number of bugs that stayed in the code for a long time despite being not that deep. Maybe that says we don't have enough eyeballs, but most of people quoting that sentence quietly assume we do, equating number of participants in FOSS communities with number of people actually auditing the code. Unfortunately, the former is much bigger than the latter. We pretty much have the same situation as we have in science publications where a lot of results are not reproducible and a lot nobody even bothers to check for reproducibility, despite reproducibility being the basic tenet of open publication and peer review and being necessary to validate the result. Unless there's a reason to - e.g. an identifiable bug - not many bother to go back to existing code and check it for correctness.

Crito 13y ago
The quote is hyperbolic for the "all", and should probably use the word "shallower", but otherwise I don't see anything wrong with it. "much more people talk about many eyes than actually audit existing code" is true of free software, but is also true of just about everything. More people think "I feel comfortable crossing this bridge because plenty of engineers have looked at it" than there are engineers actually looking at bridges. I haven't read the quote in it's original context for several years, but I don't remember it conflating software users with software developers.
- Perseids 13y ago
  The difference is that engineers actually know how to build safe bridges. I feel (and am actually) safe living in a house designed by a single competent architect, but I am not secure using a TLS implementation written by a single competent programmer. The only way we have found to end up with reasonable secure code is to have it reviewed by a lot of competent people.
TheCondor 13y ago
There are a dozen points here... The shallow bugs statement is true when there is a community that is large and with incredibly deep understanding; linux and posix is a great example, there is staggering depth of knowledge on those interfaces and behavior. Unfortunately there isn't wide and deep knowledge of tls architecture and implementation, there isn't even a lot of commonality across interfaces. There is a social axiom that you and I don't know crypto and we should leave it to the experts, yet they need help too. There is conventional c style and this function, while documented, does the opposite. I had to look at the code a couple times to see the bug, a lot of reviews could have missed it. There is conventional c style and the whole failure chain cleanup mess, look at that code again, they've got their own data representation and free function that uses it to detect if memory is allocated, the free is private but the initialize is done inline. That stuff happens everywhere by many projects, I'm not saying its wrong but it leaves it open for easy bugs, one gaffed variable declaration potentially screws everything and you need to know the data structure even though you don't directly use it. And I don't want to pile on gnutls, I think their intentions are good and this is a bug, but this looks fucking prime for unit testing... There are a lot of variables to determine and measure a projects health, the whole community needs to step up in the quality department, there are lots of ways to contribute and despite the adage, more people should look at crypto.
TheCondor 13y ago
There are a dozen points here... The shallow bugs statement is true when there is a community that is large and with incredibly deep understanding; linux and posix is a great example, there is staggering depth of knowledge on those interfaces and behavior. Unfortunately there isn't wide and deep knowledge of tls architecture and implementation, there isn't even a lot of commonality across interfaces. There is a social axiom that you and I don't know crypto and we should leave it to the experts, yet they need help too. There is conventional c style and this function, while documented, does the opposite. I had to look at the code a couple times to see the bug, a lot of reviews could have missed it. There is conventional c style and the whole failure chain cleanup mess, look at that code again, they've got their own data representation and free function that uses it to detect if memory is allocated, the free is private but the initialize is done inline. That stuff happens everywhere by many projects, I'm not saying its wrong but it leaves it open for easy bugs, one gaffed variable declaration potentially screws everything and you need to know the data structure even though you don't directly use it. And I don't want to pile on gnutls, I think their intentions are good and this is a bug, but this looks fucking prime for unit testing... There are a lot of variables to determine and measure a projects health, the whole community needs to step up in the quality department, there are lots of ways to contribute and despite the adage, more people should look at crypto.
TheCondor 13y ago
There are a dozen points here... The shallow bugs statement is true when there is a community that is large and with incredibly deep understanding; linux and posix is a great example, there is staggering depth of knowledge on those interfaces and behavior. Unfortunately there isn't wide and deep knowledge of tls architecture and implementation, there isn't even a lot of commonality across interfaces. There is a social axiom that you and I don't know crypto and we should leave it to the experts, yet they need help too. There is conventional c style and this function, while documented, does the opposite. I had to look at the code a couple times to see the bug, a lot of reviews could have missed it. There is conventional c style and the whole failure chain cleanup mess, look at that code again, they've got their own data representation and free function that uses it to detect if memory is allocated, the free is private but the initialize is done inline. That stuff happens everywhere by many projects, I'm not saying its wrong but it leaves it open for easy bugs, one gaffed variable declaration potentially screws everything and you need to know the data structure even though you don't directly use it. And I don't want to pile on gnutls, I think their intentions are good and this is a bug, but this looks fucking prime for unit testing... There are a lot of variables to determine and measure a projects health, the whole community needs to step up in the quality department, there are lots of ways to contribute and despite the adage, more people should look at crypto.
TheCondor 13y ago
There are a dozen points here... The shallow bugs statement is true when there is a community that is large and with incredibly deep understanding; linux and posix is a great example, there is staggering depth of knowledge on those interfaces and behavior. Unfortunately there isn't wide and deep knowledge of tls architecture and implementation, there isn't even a lot of commonality across interfaces. There is a social axiom that you and I don't know crypto and we should leave it to the experts, yet they need help too. There is conventional c style and this function, while documented, does the opposite. I had to look at the code a couple times to see the bug, a lot of reviews could have missed it. There is conventional c style and the whole failure chain cleanup mess, look at that code again, they've got their own data representation and free function that uses it to detect if memory is allocated, the free is private but the initialize is done inline. That stuff happens everywhere by many projects, I'm not saying its wrong but it leaves it open for easy bugs, one gaffed variable declaration potentially screws everything and you need to know the data structure even though you don't directly use it. And I don't want to pile on gnutls, I think their intentions are good and this is a bug, but this looks fucking prime for unit testing... There are a lot of variables to determine and measure a projects health, the whole community needs to step up in the quality department, there are lots of ways to contribute and despite the adage, more people should look at crypto.

ori_b 13y ago

> I mean shit, I don't need to be an aeronautical engineer or work in the FAA to say that several people investigating an airliner crash will be more effective than one single dude sifting through a literal field of debris. With zero people looking at airliner failures, nothing will be discovered, no matter how many inspectors the FAA has on its payroll. In the software world, there are almost no people doing code audits. Bugs and security holes are left in live code for years. Basically, Theo is saying "Shut up and audit." How much open source code have you audited for bugs recently? How many subtle correctness issues have you found in projects you've looked at? For the sake of code quality, I certainly hope it's higher than the amount I've audited.

Crito 13y ago
"Linus's Law" itself says little about whether or not people will actually look at code. It just says that more people looking is better than fewer people looking. How much code I have audited, you have audited, ESR has audited, or hell, how much code Paul McCartney has audited, really has little to do with the obvious correctness and banality of the 'law'.
- ori_b 13y ago
  In other words, your interpretation of "Linus's Law" has no impact on code quality in the real world. Edit: However, since it's meant to imply that in open source, there are people looking at code, I think that the criticisms stand.
  - Crito 13y ago
    Yes, in the sense that Crito's Law (coined by me, now) "The bigger the ship, the more freight you can fit in it." has no impact on the real-world ship transport industry. It is obviously true to the point of being banal. It is a pointless statement of uncontroversial fact that provides next to no utility to anybody. It's not even interesting for being a tautology. If I were a particularly objectionable and self-promoting person, then perhaps people might object to Crito's Law whenever it were quoted on shipping forums, but that wouldn't make it incorrect. Nor would my shameless self-promotion make it profound. (Is Crito's Law _precisely_ true? Well no, some large ships are not designed for freight after all... but the general principle is true.)

copergi 13y ago

It is neither pointless nor content-less. It gets to the exact heart of the problems with ESR's claim. If Joe Shmoe had said it, people would not say "no shit", they would still say "no that is complete nonsense". Because it is in fact complete nonsense, for exactly the reasons Theo pointed out. All the eyes in the world don't amount to anything if they aren't being used to audit code. And to do that, they need to be attached to people capable of auditing code. ESR is simply being used as an example of his hypothesis being wrong since he is the most obvious example. It isn't attacking him to point out the flaw in his hypothesis.

Crito 13y ago
"Eyes" obviously refers to "eyes of people auditing the code", not "eyes in skulls of people who are currently in the building", not "eyes sitting in jars in the Mutter Museum", not "every eyeball belonging to children currently living in Idaho". Furthermore, different eyeballs of course have different worth. Nobody in the world disputes that. The law itself doesn't say anything about whether or not people will choose to examine the code. It just says that more people examining the code is better than fewer people examining the code. One audit is better than none. Two is better than one. Three is better than two. One shitty auditor is better than none. Two shitty auditors are better than one. One auditor twice as good as a shitty auditor is better than one shitty auditor, if you really want to get into tedious eyeball calculus. Etc. Seems stupidly obvious; so obvious it isn't even worth stating? That's because it is. Or to address your point another way: If I said "You need fuel to make your car go." would you object that this is bullshit because not only do you need fuel, but it needs to be in your car, and the right sort of fuel? I don't think you would say, "But diesel fuel sitting in a puddle on the ground is useless and won't make your petrol car go anywhere!"
- stan_rogers 13y ago
  The point you seem to be deliberately missing is that the opportunity for something to happen and something happened are not equivalent. There is a natural assumption that because there is the opportunity for many people (however you decide to define "many") to review/audit/test code in open source projects, somebody must be doing so. Even when you base that assumption on the number of contributors to the project, non sequitur; feature addition/refinement does not imply bug fixing.