4 ms·

Linus' Law is not some kind of catch all that applies to auditing code for security weaknesses. It specifically refers to the rapid quality control that happens

Linus' Law is not some kind of catch all that applies to auditing code for security weaknesses. It specifically refers to the rapid quality control that happens when you release early and release often - the bazaar method of software development, as outlined here: http://www.catb.org/esr/writings/homesteading/cathedral-bazaar/ar01s04.html http://www.catb.org/esr/writings/homesteading/cathedral-baza...

Animats 12y ago
Yes, that's the party line. If it was true, the number of open bugs would decrease over time. For most open source programs, it increases. Mozilla has passed the 1 million bug mark.
- reubenmorais 12y ago
  > For most open source programs, it increases. I know of at least one study that contradicts this claim: "We found that with shorter release cycles, users do not experience significantly more post-release bugs and bugs are fixed faster, yet users experience these bugs earlier during software execution (the program crashes earlier)." http://swat.polymtl.ca/~foutsekh/docs/Khomh-MSR-2012.pdf http://swat.polymtl.ca/~foutsekh/docs/Khomh-MSR-2012.pdf
- rndgermandude 12y ago
  To be fair, mozilla uses bugzilla to track everything, not just "bug" bugs. Each new feature in development has a bug, or multiple bugs. When code is refactored there is a bug for that, when somebody wants commit access there is a bug for that, when an employee needs a new laptop there is a bug for that, when a community organizer wants some money or gear for an event there is a bug for that, and so on... Other organizations use their bug/issue trackers in a similar manner.
- bzbarsky 12y ago
  Mozilla has passed the 1 million bugs _ever filed_ mark. Across all Mozilla products, for that matter. Some possibly more useful statistics: For the "Core" product (read: Gecko), there are as of right now 52k open bugs and 230k closed bugs. For the "Firefox" product (desktop, not Firefox android), there are 21k open bugs and 128k closed bugs. Note that these "bugs" include feature requests, so unless you think the number of requested features will decline over time....
danieldk 12y ago
Given enough eyeballs, all bugs are shallow. Given the universal quantifier, one would assume that ESR meant security bugs here as well. Also, it is stated as a law, but ESR did not back it up with much statistical evidence to support it. I would be surprised if it holds in reality for all bugs. Extra questions: - Does releasing early and often lead to more (non-shallow) eyeballs? Not really: see Heartbleed or the OpenSSL RNG bug in Debian. - Do you need more eyeballs, or just the right eyeballs? (Given the right eyeballs, all bugs are shallow) - Isn't the effect of moving to a safer language larger?
- Taek 12y ago
  This law mostly applies to usability bugs. If a driver isn't working, somebody is going to notice. Security bugs on the other hand are often invisible during all forms of standard use. Heartbleed for example will never disrupt the user experience. The only way to find that problem is to be actively looking for it. Generally speaking, 'eyeballs' aren't actively looking for problems. They're passively using software and noticing when something seems out of place. Many security bugs will never seem out of place, and are only discoverable when someone is intentionally probing for security bugs.
- icebraining 12y ago
  - Does releasing early and often lead to more (non-shallow) eyeballs? Not really: see Heartbleed or the OpenSSL RNG bug in Debian. The existence of those bugs doesn't disprove the claim. You'd need to show that software that is not released early and often doesn't have (even) fewer eyeballs.
wiml 12y ago
I always understood this saying to be referring to the difficulty of fixing bugs once known. Some bugs are really hard to understand, reason about, and figure out how to fix. Given a large number of contributors, someone is likely to have just the right mindset and familiarity that the bug is easy for them to understand and fix (it is "shallow" for them).
- schoen 12y ago
  This article and your observation remind me that there's a huge gap between ways in which bugs might be "shallow". If Linus's Law and so on really did rely on people encountering the bugs by chance in everyday use of the software, it's no wonder (in hindsight) this doesn't help with a lot of the security bugs we face, many of which would never be triggered randomly in normal use, but rather require constructing elaborate attack scenarios. Even those that might occur by chance are not likely to be repeatable, and so not likely to get reported or analyzed. Maybe this points to a change in our prototypical concept of a "bug". When ESR first wrote "The Cathedral and the Bazaar", I would have associated "bug" with something like "the TIFFs produced by program A can't be read by program B", or "program C seems to crash if you have a non-ASCII filename", or "program D drops its network connections if more than 65536 packets are received". Today, I would associate it with something like "an attacker who sends a certificate containing an extension with invalid ASN.1 encoding that follows one or more syntactically valid extensions that are marked Critical and that are unknown to the user-agent can get remote code execution" or "an attacker who sends an XML payload that is parsed correctly by library A and incorrectly by library B due to discrepant handling of Unicode escaping can request operations that should be forbidden". Well, a literal hot-off-the-presses example would be a bug in handling multibyte characters in the regular expression library in Flash Player, which was exploitable: http://googleprojectzero.blogspot.com/2015/02/exploitingscve-2015-0318sinsflash.html http://googleprojectzero.blogspot.com/2015/02/exploitingscve... In other words, these bugs are often complicated artifacts that require research to find and malice to use -- not annoyances and breakages that are frustrating end users every day.

Animats 12y ago

Yes, that's the party line. If it was true, the number of open bugs would decrease over time. For most open source programs, it increases. Mozilla has passed the 1 million bug mark.

reubenmorais 12y ago
> For most open source programs, it increases. I know of at least one study that contradicts this claim: "We found that with shorter release cycles, users do not experience significantly more post-release bugs and bugs are fixed faster, yet users experience these bugs earlier during software execution (the program crashes earlier)." http://swat.polymtl.ca/~foutsekh/docs/Khomh-MSR-2012.pdf http://swat.polymtl.ca/~foutsekh/docs/Khomh-MSR-2012.pdf
rndgermandude 12y ago
To be fair, mozilla uses bugzilla to track everything, not just "bug" bugs. Each new feature in development has a bug, or multiple bugs. When code is refactored there is a bug for that, when somebody wants commit access there is a bug for that, when an employee needs a new laptop there is a bug for that, when a community organizer wants some money or gear for an event there is a bug for that, and so on... Other organizations use their bug/issue trackers in a similar manner.
bzbarsky 12y ago
Mozilla has passed the 1 million bugs _ever filed_ mark. Across all Mozilla products, for that matter. Some possibly more useful statistics: For the "Core" product (read: Gecko), there are as of right now 52k open bugs and 230k closed bugs. For the "Firefox" product (desktop, not Firefox android), there are 21k open bugs and 128k closed bugs. Note that these "bugs" include feature requests, so unless you think the number of requested features will decline over time....

danieldk 12y ago

Given enough eyeballs, all bugs are shallow. Given the universal quantifier, one would assume that ESR meant security bugs here as well. Also, it is stated as a law, but ESR did not back it up with much statistical evidence to support it. I would be surprised if it holds in reality for all bugs. Extra questions: - Does releasing early and often lead to more (non-shallow) eyeballs? Not really: see Heartbleed or the OpenSSL RNG bug in Debian. - Do you need more eyeballs, or just the right eyeballs? (Given the right eyeballs, all bugs are shallow) - Isn't the effect of moving to a safer language larger?

Taek 12y ago
This law mostly applies to usability bugs. If a driver isn't working, somebody is going to notice. Security bugs on the other hand are often invisible during all forms of standard use. Heartbleed for example will never disrupt the user experience. The only way to find that problem is to be actively looking for it. Generally speaking, 'eyeballs' aren't actively looking for problems. They're passively using software and noticing when something seems out of place. Many security bugs will never seem out of place, and are only discoverable when someone is intentionally probing for security bugs.
icebraining 12y ago
- Does releasing early and often lead to more (non-shallow) eyeballs? Not really: see Heartbleed or the OpenSSL RNG bug in Debian. The existence of those bugs doesn't disprove the claim. You'd need to show that software that is not released early and often doesn't have (even) fewer eyeballs.

wiml 12y ago

I always understood this saying to be referring to the difficulty of fixing bugs once known. Some bugs are really hard to understand, reason about, and figure out how to fix. Given a large number of contributors, someone is likely to have just the right mindset and familiarity that the bug is easy for them to understand and fix (it is "shallow" for them).

schoen 12y ago
This article and your observation remind me that there's a huge gap between ways in which bugs might be "shallow". If Linus's Law and so on really did rely on people encountering the bugs by chance in everyday use of the software, it's no wonder (in hindsight) this doesn't help with a lot of the security bugs we face, many of which would never be triggered randomly in normal use, but rather require constructing elaborate attack scenarios. Even those that might occur by chance are not likely to be repeatable, and so not likely to get reported or analyzed. Maybe this points to a change in our prototypical concept of a "bug". When ESR first wrote "The Cathedral and the Bazaar", I would have associated "bug" with something like "the TIFFs produced by program A can't be read by program B", or "program C seems to crash if you have a non-ASCII filename", or "program D drops its network connections if more than 65536 packets are received". Today, I would associate it with something like "an attacker who sends a certificate containing an extension with invalid ASN.1 encoding that follows one or more syntactically valid extensions that are marked Critical and that are unknown to the user-agent can get remote code execution" or "an attacker who sends an XML payload that is parsed correctly by library A and incorrectly by library B due to discrepant handling of Unicode escaping can request operations that should be forbidden". Well, a literal hot-off-the-presses example would be a bug in handling multibyte characters in the regular expression library in Flash Player, which was exploitable: http://googleprojectzero.blogspot.com/2015/02/exploitingscve-2015-0318sinsflash.html http://googleprojectzero.blogspot.com/2015/02/exploitingscve... In other words, these bugs are often complicated artifacts that require research to find and malice to use -- not annoyances and breakages that are frustrating end users every day.