5 ms·

> I feel like we, as an industry, got a bit lucky that someone with the skills, time, and energy was on hand at the right time to make a huge difference. T

> I feel like we, as an industry, got a bit lucky that someone with the skills, time, and energy was on hand at the right time to make a huge difference. This is where statistics of "many eyes" and "sunlight as disinfectant" hits home. However bizarrely improbable that anyone would just wander along and find a a bug, people do, because they can. With proprietary/closed code, the probability is zero.

candiddevmike 2y ago
> With proprietary/closed code, the probability is zero. People discover bugs all the time in closed source software...
- nonrandomstring 2y ago
  Which people?
  - thfuran 2y ago
    Customers, though they usually aren't finding security vulnerabilities.
    - nonrandomstring 2y ago
      > Customers Ah I see what you mean. Problem of language then. "Finding bugs", to me as a coder, means something different from observing the effects of bugs. Sure, I can see incorrect behaviours in lots of proprietary software. And I can guess what might cause it. But without the source code that's not the same as "finding bugs".
      - xyzzy_plugh 2y ago
        "finding a bug" in source code implies there's a mistake there. A bug can be that the implementation doesn't match the spec, or that the spec changed, with respect to the user (customer). Reporting bugs vs finding them is perhaps what you're looking for, but I'd argue finding is overloaded. It's perfectly valid to kick the tires on some software and find some bugs. I've found thousands of bugs in closed source software before, and subsequently reported those bugs.
        nonrandomstring 2y ago
        > "finding a bug" in source code implies there's a mistake there. This is a good point of course. Implementation, protocol and runtime bugs are indeed invisible from the source POV. As are lower level Ken Thompson "trusting trust" [0,1] bugs in your tool chain. Most of all though, many "bugs" are just malicious functions the coders meant to put in there. [0] https://cybershow.uk/episodes.php?id=2 https://cybershow.uk/episodes.php?id=2 [1] www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf
    - prmoustache 2y ago
      In this very blogpost, we are very much in a situation where users (github + the author) found the bug. They however didn't find the cause of the bug. And in many software, open source or proprietary, there are a lot of bugs sitting out in issues tracker for days, weeks, months or even years.
  - hobs 2y ago
    https://googleprojectzero.blogspot.com/ https://googleprojectzero.blogspot.com/ immediately comes to mind (though its not a good example of an anyone set of people)
  - myself248 2y ago
    Three letter agencies, and blackhats, who have the code even though it's not "open". They then use those vulnerabilities without disclosing them or fixing them, for as long as they're able.
    - nonrandomstring 2y ago
      Amen to that. Damn right proprietary code is a double danger in that regard. Officially illegal to inspect, except to a few criminals. Isn't the entirety of Microsoft's Windows code now "out there"? It has been for years, right?
- smsm42 2y ago
  True. What is missing is the next step - looking at the code and figuring out why the weird thing is going on. In closed source, what'd happen is a call to support, who may or may not properly file a bug, which may or may not get to the eyes of an engineer that knows what's going on, who may or may not be interested in actually fixing it... But more likely it'd end up in the huge pile of "you're just holding it wrong" tickets and nobody ever would pay attention to it, because it's impossible for one company to pay attention to millions of weird complaints, and nobody else can. It's a numbers game, and with closed source, the numbers are much less favorable, because it's economically impossible for it to be otherwise.
- noodlesUK 2y ago
  I'm sure that lots of HNers including myself have encountered security vulns in different proprietary pieces of software that we encounter. I can think of at least two or three occasions where I discovered an exploitable vuln, didn't have a contact to report it to, decided I didn't want to have the back-and-forth of some non tech company threatening me with a lawsuit, and took no further action (typically for lower severity vulns in less important services). With open source software you can hop on GitHub and dig deeper into a vuln or other bug and actually report it somewhere.
pixl97 2y ago
Well, in closed they find 'bugs' all the time, a large portion of the time you don't need the code at all. That said, you're not solving the issue and probably not doing anything about it. On top of that, the vast majority of people won't know what to do if they find a bug. In the distant past I was one of those people, and only realized some of the bugs I saw in hindsight. It's been nearly 30 years so the details are mostly gone now, but I remember messing around in Windows and I could make the Microsoft Netmeeting application crash with a buffer overrun error. Of course I was really new to computers then and understanding that buffer overflows in networked applications were really bad things (and seemingly beyond a lot of people that had been in the industry). Even then attempting to report security issues back in those days would have been far more difficult (hell and even risky in many cases). So really it is many things that are required. Running into the issue. Having a deep enough understanding of computing to realize the issue is bad. Having a means of reporting the bug to a place where people will look at it. And a security culture that knows when and how to act on the bug report.
- nonrandomstring 2y ago
  I think this is a good taxonomy/sequence to note 1) Observing 2) Understanding as bad 3) Reporting (or fixing) 4) Closing the loop on remedy Both proprietary and open code runs into problems at (3) because people don't want to hear it, for commercial or ego reasons. With FOSS at least you get the direct intervention route of simply fixing and publishing the patch, which done responsibly may or not force a maintainer's hand. With proprietary you can piss into the wind and be ghosted, or sued, hence more irresponsible/anonymous disclosure. Anything that maximises the likelihood of getting from (1) to (4) safely must be a good thing, so I think FOSS yields the better security model.
rmetzler 2y ago
I also think that open source is better than closed source. Nothing to argue about. What I was wondering when I read the same sentence you quoted: how many really serious security bugs like Heartbleed, CVE-2008-0166, or the zx drama are happening without people finding out about it and publishing their findings?
- rstuart4133 2y ago
  In open source there are only two likely outcomes when someone notices a security issue: either they plan to hoard it for their on gain, or the tell the world about it and earn the kudo's. The ability to earn kudo's is a right proper pain in the arse because a lot of things that have little to no impact on security are loudly touted as security bugs and so a lot of time is wasted on triage of non-security issues in open source. The problem with closed source is there is a third possibility: ignore the problem and save on the cost of fixing it. The responsible disclosure regime we have now is because companies almost always chose this option, ie denied it was a problem and refused to invest to fix it. When the discoverer then released the bug anyway they we so enamoured this this approach tried solving the disclosure problem by suing the researcher. If you think companies still don't ignore security issues when they are given a choice, you are kidding yourself. The problem compounds because when you do find a bug open source makes it easy to see if you can use it to create a security issue. In proprietary code that's much harder, so I'm 100% certain a fair number of potential security issues don't get patched because it isn't obvious how to exploit them. Nonetheless they are chinks in the armour, so they give the bad guys a excellent set of starting places to start looking.
Twirrim 2y ago
I was reflecting on xz being an absolute triumph of open source software: 1) Someone noticed something that was odd. They were able to go look at what was happening, along with source code and see that suspicious things had happened. 2) They were able to reach out to security experts across almost all the main distributions and get additional eyes on it who also confirmed there was a security thing and were immediately able to take actions to deal with the insanity. 3) Disclosure went public, and lots of eyes with expertise across all kinds of aspects of software and security have been able to tease apart what was done, and how, figuring out risks etc. 4) Other suspicious commits across other bits of software, from the same developer, have been tracked down and identified and the work continues in figuring out consequences. 5) Every distribution has become alert to the minutia of the ways that things were compromised around the build archives etc and have been able to start figuring out how to catch other cases and stop this in future. Stuff will start to spread to various open source projects as distributions work on their packaging tooling/processes, helping ensure other projects don't become vulnerable to compromise through the same mechanisms. If you contrast this with closed source, where unless there is an exploit, it's reports of e.g. "Your software is running slightly slow" like in the xz/openssh case, is unlikely to get much attention, if any. Then once the closed source company finally finds out about it, at best you'll get a very carefully phrased explanation of what happened, revealing just the bare minimum amount of data they think they can possibly get away with. That badly harms the whole industry's ability to avoid repeats.
- nonrandomstring 2y ago
  Absolutely agree that xz is a paradigmatic model of how well things can go with free open source. However, that also led me and others to a lot of dark reflecting on how vulnerable and exposed open source devs and maintainers are [0] and about complex social engineering attacks on the supply chain. It also raises troubling questions about how proprietary vendors can take advantage of that and spin it to their own ends. [0] https://cybershow.uk/blog/posts/poison-code https://cybershow.uk/blog/posts/poison-code
  - gunapologist99 2y ago
    And SolarWinds is a prime example of the opposite approach.

candiddevmike 2y ago

> With proprietary/closed code, the probability is zero. People discover bugs all the time in closed source software...

nonrandomstring 2y ago
Which people?
- thfuran 2y ago
  Customers, though they usually aren't finding security vulnerabilities.
  - nonrandomstring 2y ago
    > Customers Ah I see what you mean. Problem of language then. "Finding bugs", to me as a coder, means something different from observing the effects of bugs. Sure, I can see incorrect behaviours in lots of proprietary software. And I can guess what might cause it. But without the source code that's not the same as "finding bugs".
    - xyzzy_plugh 2y ago
      "finding a bug" in source code implies there's a mistake there. A bug can be that the implementation doesn't match the spec, or that the spec changed, with respect to the user (customer). Reporting bugs vs finding them is perhaps what you're looking for, but I'd argue finding is overloaded. It's perfectly valid to kick the tires on some software and find some bugs. I've found thousands of bugs in closed source software before, and subsequently reported those bugs.
      - nonrandomstring 2y ago
        > "finding a bug" in source code implies there's a mistake there. This is a good point of course. Implementation, protocol and runtime bugs are indeed invisible from the source POV. As are lower level Ken Thompson "trusting trust" [0,1] bugs in your tool chain. Most of all though, many "bugs" are just malicious functions the coders meant to put in there. [0] https://cybershow.uk/episodes.php?id=2 https://cybershow.uk/episodes.php?id=2 [1] www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf
  - prmoustache 2y ago
    In this very blogpost, we are very much in a situation where users (github + the author) found the bug. They however didn't find the cause of the bug. And in many software, open source or proprietary, there are a lot of bugs sitting out in issues tracker for days, weeks, months or even years.
- hobs 2y ago
  https://googleprojectzero.blogspot.com/ https://googleprojectzero.blogspot.com/ immediately comes to mind (though its not a good example of an anyone set of people)
- myself248 2y ago
  Three letter agencies, and blackhats, who have the code even though it's not "open". They then use those vulnerabilities without disclosing them or fixing them, for as long as they're able.
  - nonrandomstring 2y ago
    Amen to that. Damn right proprietary code is a double danger in that regard. Officially illegal to inspect, except to a few criminals. Isn't the entirety of Microsoft's Windows code now "out there"? It has been for years, right?
smsm42 2y ago
True. What is missing is the next step - looking at the code and figuring out why the weird thing is going on. In closed source, what'd happen is a call to support, who may or may not properly file a bug, which may or may not get to the eyes of an engineer that knows what's going on, who may or may not be interested in actually fixing it... But more likely it'd end up in the huge pile of "you're just holding it wrong" tickets and nobody ever would pay attention to it, because it's impossible for one company to pay attention to millions of weird complaints, and nobody else can. It's a numbers game, and with closed source, the numbers are much less favorable, because it's economically impossible for it to be otherwise.
noodlesUK 2y ago
I'm sure that lots of HNers including myself have encountered security vulns in different proprietary pieces of software that we encounter. I can think of at least two or three occasions where I discovered an exploitable vuln, didn't have a contact to report it to, decided I didn't want to have the back-and-forth of some non tech company threatening me with a lawsuit, and took no further action (typically for lower severity vulns in less important services). With open source software you can hop on GitHub and dig deeper into a vuln or other bug and actually report it somewhere.

pixl97 2y ago

Well, in closed they find 'bugs' all the time, a large portion of the time you don't need the code at all. That said, you're not solving the issue and probably not doing anything about it. On top of that, the vast majority of people won't know what to do if they find a bug. In the distant past I was one of those people, and only realized some of the bugs I saw in hindsight. It's been nearly 30 years so the details are mostly gone now, but I remember messing around in Windows and I could make the Microsoft Netmeeting application crash with a buffer overrun error. Of course I was really new to computers then and understanding that buffer overflows in networked applications were really bad things (and seemingly beyond a lot of people that had been in the industry). Even then attempting to report security issues back in those days would have been far more difficult (hell and even risky in many cases). So really it is many things that are required. Running into the issue. Having a deep enough understanding of computing to realize the issue is bad. Having a means of reporting the bug to a place where people will look at it. And a security culture that knows when and how to act on the bug report.

nonrandomstring 2y ago
I think this is a good taxonomy/sequence to note 1) Observing 2) Understanding as bad 3) Reporting (or fixing) 4) Closing the loop on remedy Both proprietary and open code runs into problems at (3) because people don't want to hear it, for commercial or ego reasons. With FOSS at least you get the direct intervention route of simply fixing and publishing the patch, which done responsibly may or not force a maintainer's hand. With proprietary you can piss into the wind and be ghosted, or sued, hence more irresponsible/anonymous disclosure. Anything that maximises the likelihood of getting from (1) to (4) safely must be a good thing, so I think FOSS yields the better security model.

rmetzler 2y ago

I also think that open source is better than closed source. Nothing to argue about. What I was wondering when I read the same sentence you quoted: how many really serious security bugs like Heartbleed, CVE-2008-0166, or the zx drama are happening without people finding out about it and publishing their findings?

rstuart4133 2y ago
In open source there are only two likely outcomes when someone notices a security issue: either they plan to hoard it for their on gain, or the tell the world about it and earn the kudo's. The ability to earn kudo's is a right proper pain in the arse because a lot of things that have little to no impact on security are loudly touted as security bugs and so a lot of time is wasted on triage of non-security issues in open source. The problem with closed source is there is a third possibility: ignore the problem and save on the cost of fixing it. The responsible disclosure regime we have now is because companies almost always chose this option, ie denied it was a problem and refused to invest to fix it. When the discoverer then released the bug anyway they we so enamoured this this approach tried solving the disclosure problem by suing the researcher. If you think companies still don't ignore security issues when they are given a choice, you are kidding yourself. The problem compounds because when you do find a bug open source makes it easy to see if you can use it to create a security issue. In proprietary code that's much harder, so I'm 100% certain a fair number of potential security issues don't get patched because it isn't obvious how to exploit them. Nonetheless they are chinks in the armour, so they give the bad guys a excellent set of starting places to start looking.

Twirrim 2y ago

I was reflecting on xz being an absolute triumph of open source software: 1) Someone noticed something that was odd. They were able to go look at what was happening, along with source code and see that suspicious things had happened. 2) They were able to reach out to security experts across almost all the main distributions and get additional eyes on it who also confirmed there was a security thing and were immediately able to take actions to deal with the insanity. 3) Disclosure went public, and lots of eyes with expertise across all kinds of aspects of software and security have been able to tease apart what was done, and how, figuring out risks etc. 4) Other suspicious commits across other bits of software, from the same developer, have been tracked down and identified and the work continues in figuring out consequences. 5) Every distribution has become alert to the minutia of the ways that things were compromised around the build archives etc and have been able to start figuring out how to catch other cases and stop this in future. Stuff will start to spread to various open source projects as distributions work on their packaging tooling/processes, helping ensure other projects don't become vulnerable to compromise through the same mechanisms. If you contrast this with closed source, where unless there is an exploit, it's reports of e.g. "Your software is running slightly slow" like in the xz/openssh case, is unlikely to get much attention, if any. Then once the closed source company finally finds out about it, at best you'll get a very carefully phrased explanation of what happened, revealing just the bare minimum amount of data they think they can possibly get away with. That badly harms the whole industry's ability to avoid repeats.

nonrandomstring 2y ago
Absolutely agree that xz is a paradigmatic model of how well things can go with free open source. However, that also led me and others to a lot of dark reflecting on how vulnerable and exposed open source devs and maintainers are [0] and about complex social engineering attacks on the supply chain. It also raises troubling questions about how proprietary vendors can take advantage of that and spin it to their own ends. [0] https://cybershow.uk/blog/posts/poison-code https://cybershow.uk/blog/posts/poison-code
- gunapologist99 2y ago
  And SolarWinds is a prime example of the opposite approach.