8 ms·
The author asks for evidence of GPLv3 software hurting people. This includes the Linux kernel, for which there are many examples of incidents. The below was the
by Godel_unicode 10y ago
The author asks for evidence of GPLv3 software hurting people. This includes the Linux kernel, for which there are many examples of incidents. The below was the first I found[1] and it has the benefit of involving multiple open-source projects under different licenses.
The idea that software is inherently safer because it's released under an open-source license is overly simplistic in the extreme. Just the most obvious reason for this is that opportunity for independent review doesn't guarantee that review will happen. The wisdom of the crowd doesn't guarantee a third party review will be better or more thorough than a solid first party system. Open-source provides the possibility of review, and that's all. Hypothetical eyes make no bugs shallow.
Edit - I'm not making a claim one way or the other about the relative safety of open vs proprietary software, I'm merely saying that the comparison is deeper and more nuanced than the way it is typically portrayed.
1 - http://www.bbc.com/news/technology-28867113 http://www.bbc.com/news/technology-28867113
- ktRolster 10y agoWith open source software, you have the possibility of knowing how safe it is. With closed source, you don't (ok, it's a lot harder to read through binaries, but still vaguely possible).
- tigershark 10y agoNo way that you can understand how safe is the neural network that is driving your car. And this applies equally to open source autopilot systems and closed ones. The article is just a rant that misses this fundamental point.
- chmars 10y agoI cannot, others can - I can even pay them to do if necessary.
- tigershark 10y agoNo one can. If anyone could certify the neural network entire response mapping then you could simply write an exact algorithm with the same responses. Good luck for doing it to automatically drive a car. Strangely every company in the world is focusing on NN for autopilot, not on hand written algorithms.
- ktRolster 10y agoIf anyone could certify the neural network entire response mapping then you could simply write an exact algorithm with the same responses. You can, actually, the result would be extremely complex, though.
- EGreg 10y agoCan you pay someone to solve the halting problem?
- halomru 10y agoNo, but neither can anyone else. I can't proof that the neutral network always behaves correctly, but with testing and statistics you can verify the network up to a chosen certainty (below 100% obviously). Or you can actively search for problems with fuzzing techniques. All without solving the halting problem.
- deleted 10y ago[deleted]
- chris_wot 10y agoSays who? You may not be able to (I sure can't) but that doesn't mean there aren't those who can and do understand this sort of thing. And there are those who will literally sit down for months and years till they do understand it. Or they might rip it out and replace it.
- tigershark 10y agoGuys, are you really saying that you think that you can certify every response of a NN for every input? Especially for NN as complicated as autopilot systems? Are you aware that if that was true then you would be the richest men in the world given that everyone would come to you to write a perfect algorithm to drive a car?
- ptero 10y agoCertifiable reliability is not the only goal. But even in safety critical tasks I have seen NNs and MDPs used to derive a good enough solution, then training turned off and the system certified (e.g. by running a large gold standard collection of test cases)
- jobigoud 10y agoThey will understand how it works in principle, not why it took this or that specific decision and they might not be able to exactly predict what it will do in an unforeseen situation. The decision itself might be taken using an amount of data completely overwhelming to human being analysis, processed at inhuman speed.
- tigershark 10y agoExactly, so having an open source autopilot systems would not have changed absolutely nothing, and the guy on the tesla watching Harry Potter would have been dead the same.
- nickpsecurity 10y agoNice strawman given most systems use deterministic methods for that reason. Plus, simplest is often best in safety-critical. That said, you could've gotten more answers on Google with terms formal verification neural nets. NASA paper with good methods was first result for me.
- tigershark 10y agoGood methods for what? Not certainly to certify that each combination of inputs gives a certain output, as you can read from that paper. And that is the only way that would have ensured that the guy on the tesla didn't die. That methodology furthermore has nothing to do with open source.
- nickpsecurity 10y ago"Good methods for what? Not certainly to certify that each combination of inputs gives a certain output, as you can read from that paper." You're ignoring the biggest issue in your whole tangent: real-world engineers with sense won't use NN's in systems unless the problem itself is non-deterministic & other methods don't work. Personally, I push control theory, state machines, ladder logic, whatever for applications like this. You can deterministically analyze them. Whereas, problems with too much uncertainty or requiring approximate solutions will always have an error margin. You personally have been pushing this thread to focus as much as possible on the NN case while ignoring 99% of real-world, use cases representing how the OP topic will usually play out. You also act like the inherent error in the tooling is bad when the problem domain itself has inherent error. So, I decided to address your tangent. The first solution, safety monitors, goes way back in safety-critical use. That's to have a simple solution checking the ranges or other behavior of the NN that's able to override faulty NN with simpler logic. This can be done for GPS-guided drones that usually move in straight lines or smooth curves. The NN part is reset, diagnosed, or deactivated while the fail-safe system takes plane on simpler path or out of the area. Second part checks the nets themselves. Again, given a non-deterministic problem domain, the nets will be non-deterministic with the best one can do getting the error margin way down or within fail-safe bounds. This other paper I found summarizes a lot of V&V methods for NN's: https://people.cs.umass.edu/~btaylor/publications/PSI000008.pdf https://people.cs.umass.edu/~btaylor/publications/PSI000008.... Safety monitors. Training data or updates with constraints built in. Automated, test-case generation tweaking expected data. Rule extraction. System visualization. Simulation. All should contribute to safer NN's. They can't be perfectly correct, though, because vision, complex flying, thinking... none of this is 100% accurate no matter what method you use.
- amalcon 10y agoSure you can. It's just not by verifying the algorithm, it's by analyzing real-world performance. This is the same way that we understand how safe the human driving the car is, and the insurance industry is very good at this.
- eridius 10y agoAnalyzing real-world performance doesn't care if the source is open or proprietary, so you're kind of missing the point of this whole thread.
- nickpsecurity 10y agoThis isn't true at all. What's necessary is code written in a verifiable way, qualified reviewer assessing it, trust in that reviewer/assessment, and way to ensure you're using what was reviewed in evaluated configuration. It's how main, security certifications work if you do it at high-assurance level. Lower levels are paper BS or simply inadequate given short-term, code review only catches so much. I broke it down here with a quick revision just now: http://pastebin.com/pjc0cueA http://pastebin.com/pjc0cueA
- dandelion_lover 10y ago>Hypothetical eyes make no bugs shallow. https://www.fsf.org/blogs/community/who-actually-reads-the-code https://www.fsf.org/blogs/community/who-actually-reads-the-c...
- __david__ 10y agoAnecdotally, I read through things occasionally. I once was curious about something in the xnu kernel and noticed a bug in the code, just from reading it. I sent Apple a bug report and eventually they fixed it (though I think it took a couple years).
- DHMO 10y agoBeyond the "email me" example, as an author of free source projects for the past several years, I can say that if the project is used and if users have access to the source, they will find and fix bugs- not all, but more than would have been otherwise. On the flipside, if few use your project, bugs can last for ages. I've had this experience also- distributing projects with free source that don't work, and not finding out until much later when I noticed it myself. Making source free is not bad for security, because it's not about high visibility. Security problems are more frequent when the vector of attack is larger, e.g. when your car can be stolen because they didn't anticipate misuse of the entertainment system that was accessible via the carrier. Related to that topic, Chrysler has recently been paying hackers to try to find security flaws: https://www.wired.com/2016/07/chrysler-launches-detroits-first-bug-bounty-hackers/ https://www.wired.com/2016/07/chrysler-launches-detroits-fir... If the source is free or open, then people can find flaws more easily, so that they can be fixed. You want to be able to find bugs fast and fix them quickly. When you don't, that's when you get into serious trouble. As a warning though, source being free and open and well-used doesn't mean that bugs can't go a very long time without being seen, after which point they are everywhere: https://en.wikipedia.org/wiki/Shellshock_(software_bug) https://en.wikipedia.org/wiki/Shellshock_(software_bug)
- microcolonel 10y agoThe linux kernel is GPLv2
- chris_wot 10y agoThat's a bug in OpenSSL, not the Linux kernel. OpenSSL doesn't use the GPL, it is dual licensed under its own license and the SSLeay license. And good luck fixing heartbleed in a proprietary product in the way that occurred for OpenSSL - the entire codebase got looked at and we got BoringSSL and LibreSSL. What would you prefer - code with obscure bug from spaghetti code so bad it kills people [1], or code looked at by enthusiastic amateurs and professionals alike? Car people tend to enjoy pushing the envelope and they are doing this with software already! 1. http://www.safetyresearch.net/blog/articles/toyota-unintended-acceleration-and-big-bowl-“spaghetti”-code http://www.safetyresearch.net/blog/articles/toyota-unintende...
- rando832 10y ago> The below was the first I found[1] Try again, that is bsd openssl code, probably proprietarized, not gpl.
- deleted 10y ago[deleted]
- threatofrain 10y agoYou're arguing that the (probability) x (impact) of a claimed benefit is in question. But your argument merely mitigates the good that is provided by software transparency. What are the bad effects to society for software transparency? Because if open software offers noisy positives with minor negatives, then we should prefer open as the default heuristic until circumstantial information says otherwise. Some projects, whether open or closed, are more valued by institutions or businesses, and consequently some projects will justify more money and eyeballs. Also, what businesses and institutions value can be unintuitive. When there's a bug in Linux, that might immediately degrade a business product, whereas a security problem with SSH might not ever impact a business -- even if important customer data is stolen. I doubt Target experienced a real business problem after its large data theft incident with customer financial data. Therefore, 3rd parties contributing to software will probably be far more prompt for bugs in projects that directly impact business success vs. OpenSSH, but at least the door for extra benefits is open, whereas for closed software the door is shut.
- lqdc13 10y agoI am also for open source, however here are some arguments for closed source: What are the bad effects to society for software transparency? One is companies that are less willing to write code or go to market because their code IP wouldn't be competitive given existing players. Smaller new competitors' code would be quickly copied by larger existing players with their larger teams into larger projects that are already used by everyone. Then there is the case of artistic one-off projects. Imagine a $50 video game releasing its source code immediately or as it is being made. There would be another company that just copies the code or the textures and distributes quickly before the last level is done. at least the door for extra benefits is open, whereas for closed software the door is shut This is true, but some doors that you don't want to be open are also shut. It takes a lot of effort to reverse engineer something. I bet all the openssl bugs would be harder to find if it wasn't open source. Especially, if binaries weren't available either - something like only an API layer existing. Maybe heartbleed wouldn't have been found until openssl was replaced by something else. Finally, something like GNU parallel is very commonly used. On the other hand, some random utility that is on Github and isn't very popular probably isn't looked at by nearly as many people. Out of 100 people who use some software, about 1% looks at source code. So then if you have 1000 people use your tool, only 10 people look at the tool's source code and 1 of them is a malicious actor looking for exploits. I have personally found several bugs in open source software that could have been security exploits, but haven't reported them because I was lazy at the time. Having said all this, most non-application-specific software could be open source and it would probably help it more than hurt it.
- JumpCrisscross 10y ago> The idea that software is inherently safer because it's released under an open-source license is overly simplistic in the extreme Reminds me of the common confounding of elections and democracy.
- zeven7 10y agoIndeed, the DAO was GPL v3[1]. It was taken to the tune of $50M. Slapping a particular license on it doesn't make it more secure. [1] https://github.com/slockit/DAO/blob/develop/LICENSE https://github.com/slockit/DAO/blob/develop/LICENSE
- jasonkostempski 10y ago"opportunity for independent review doesn't guarantee that review will happen." Maybe not with the software most of the world doesn't care about but I'd bet that there will always be someone willing to look real hard at software being put in to popular automobiles.
- qwertyuiop924 10y ago>The idea that software is inherently safer because it's released under an open-source license is overly simplistic in the extreme. Well, no. No it isn't. A higher possibility of more people seeing the code translates into a higher probability of bugs getting fixed. Of course, there's no guarantees, but more eyes means more chances of bugs getting caught. And that means safer.
- nickpsecurity 10y ago"Of course, there's no guarantees, but more eyes means more chances of bugs getting caught. And that means safer." That doesn't. It means potential to be safer if all kinds of things line up. Vast majority of FOSS code is crap like any other code. More eyes didn't change that. What's needed is design/implementation work, qualified review, time, and trustworthiness of reviewers & distribution. http://pastebin.com/pjc0cueA http://pastebin.com/pjc0cueA
- qwertyuiop924 10y agoFair enough. But I would, and indeed, do, argue that potentially safer is better than not potentially safer.
- nickpsecurity 10y agoIn this order: provably safer is better than potentially safer; potentially safer is better than no potential. And still, as I showed, open or closed source at consumer level has little to do with the potential. It's amount of trusted development and/or review of source then distribution of that. Illustrated where greater than means more trustworthy, reliable, and/or effective: 1. Rigorously designed & reviewed open-source is greater than... 2. Rigorously designed & reviewed closed-source is greater than... 3. Typical, open-source is greater than... 4. Typical, closed-source is greater than... 5. Cloud infrastructure based on Concurrent DOS written in a MOL-360 language.
- qwertyuiop924 10y ago