14 ms·

We found over 10000 bugs in various open source projects

raverbashing 10y ago
What I take from this: computer languages (C++ especially, but also systems built in C) are too complex for the average mind No, I'm not significantly smarter than the average programmer That's what needs to be addressed. While C++ facilitates things, it is complex. While C is very simple it makes mistakes easier by requiring things to be done manually IDEs don't help. Code completion is helpful but it is of limited help And we keep repeating ourselves every time we write a for loop
- informatimago 10y ago
  Switch to Common Lisp. And with lisp, you don't have to repeat yourself: any pattern, or any boilerplate can be abstracted away with a macro.
  - bluGill 10y ago
    > can be abstracted away Except for too many levels of abstraction.
    - spacemanmatt 10y ago
      Is that the fault of the programmer or the language, though?
      - steveklabnik 10y ago
        It's a reference to https://en.wikipedia.org/wiki/Fundamental_theorem_of_software_engineering https://en.wikipedia.org/wiki/Fundamental_theorem_of_softwar... (and the "humorous clause" mentioned there as well)
    - raverbashing 10y ago
      Yes, I believe one of the problems is the way things are abstracted, and I don't see a language solving this Adding more levels can only go so far But I don't know how to do it better. Module/Class/Method won't cut it anymore
    - maxerickson 10y ago
      That's kind of what some compiler optimizations do isn't it? Look through the abstraction and discard it?
  - golergka 10y ago
    The fact that something can be done doesn't mean that it should be done.
  - vbezhenar 10y ago
    How can CL prevent something like "i1 == i1" instead of "i1 == i2" error?
    - jjnoakes 10y ago
      First, since the AST is so simple to walk programmatically, one could write a simple check for that kind of bug in, I'm guessing, a few lines of Lisp. Second, just because you can come up with a bug that can be written in a language doesn't mean the language doesn't prevent all sorts of other bugs. All languages can have bugs written in them. That's not the point.
- n00b101 10y ago
  >computer languages (C++ especially, but also systems built in C) are too complex for the average mind I'm not necessarily disagreeing, but have you looked at the source code [1] of the Apollo Guidance computer [2]? This software was shipped to production 50 years ago. It literally implements a "self-driving space rocket." It is staggering that this was completed in such a short time and actually worked. The code is written entirely in the "Apollo Guidance Computer (AGC) Assembly" language. The hardware that it was written for had to be invented in at the same time, and was the world's first computer to use Integrated Circuits ("chips"). They also implemented a Real-Time Operating System with a multi-task priority scheduler, as well as a virtual machine and assembly interpreter within the software. This was mission and life safety critical code, the risk involved boggles the mind. And yet the software was incredibly reliable. It was so reliable that it even saved the Apollo 11 crew and mission during the lunar descent by famously recovering from a peripheral hardware bug. I honestly do not know how this feat was even humanly possible, but I do know that these programmers did not rely on "code completion." [1] https://github.com/chrislgarry/Apollo-11 https://github.com/chrislgarry/Apollo-11 [2] https://en.wikipedia.org/wiki/Apollo_Guidance_Computer https://en.wikipedia.org/wiki/Apollo_Guidance_Computer
  - emodendroket 10y ago
    Yeah, sure. But if they'd had code completion it would have been a help, not a hindrance.
  - drzaiusapelord 10y ago
    >I honestly do not know how this feat was even humanly possible The Apollo program, fixed for inflation, was something like $130 billion dollars. That's buys you a lot of talent, testing, etc. Its also worth noting that the program wasn't without its screw ups. The Apollo 1 deaths. The radar control on Apollo 11 eating up all the CPU during lunar landing. Apollo 13's famous O2 tank explosion. I think you may be glamorizing something that was well done, but falls within the realm of 'you get what you pay for' and the occasional human error, which in the case of Apollo 1, turned out to be fatal. I wouldn't call a program with three deaths and an aborted lunar insert to be infallible. That said, NASA/JPL publish code standards, although I imagine most tech companies would balk at giving up their toys and hot languages/frameworks to do fairly tedious work like this. http://lars-lab.jpl.nasa.gov/ http://lars-lab.jpl.nasa.gov/
- emodendroket 10y ago
  This also includes analysis of C#, which is quite different from C.
  - raverbashing 10y ago
    Yes, while C#/Java will prevent some kind of errors from happening, others are very common (Example: copy/paste errors, parameter order problems, etc)
- spriggan3 10y ago
  > While C is very simple it makes mistakes easier by requiring things to be done manually I wouldn't describe C as simple but basic. Of course, it's start being less basic with all the macros and all the tools needed to compile and distribute a C program , like autoconf ... with all these layers , C becomes as complicated as any other language because it relies on so many 3rd party tools (which are often dynamic languages) to fix its shortcomings. But as long as linux kernel is written in C the language isn't going anywhere.
- Pasick 10y ago
  I wouldn't say that the number of errors in code directly depends on complexity of the language. From my practice I learned that the code quality depends on a software development process implemented in your team. Now we have whole bunch of practices and tools that allow us to minimize risks of appearing stupid mistakes in code: covering the code with unit and integration tests, static and dynamic code analysis, continuous integration, code reviews, refactoring and so on, and, I should say, these things work. So, I think, this article demonstrates that still there are a lot of teams whose development process can be improved.
- nickpsecurity 10y ago
  That's actually the problem Ada's designers set out to solve. They add to what they solve every iteration with safe defaults and contraints on use. Whether you do Ada or not, you will likely find skimming this book to be helpful to see how one can systematically analyze all forms of programmer error and attempt to eliminate them: http://www.adacore.com/uploads/technical-papers/SafeSecureAdav2015-covered.pdf http://www.adacore.com/uploads/technical-papers/SafeSecureAd... That's at language level. At process level, Harlan Mills developed Cleanroom back in 80's to try to get software defects as low as cleanrooms CPU's were built in. It combines a formal approach of describing/connecting things, constrained implementation for easy verification, iterative development, and usage-based testing. Defect rate was extremely low even for first use by teams with cost ranging from slightly more to cheaper due to less debugging. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.88.2667&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.88.... On productivity side, the DSL-based approach on top of safe, functional languages seems to be winning out. Safe, functional combo already knocks out a lot of problems. DSL's for common categories of programming let you read and write less code to get same work done possibly safer with sane auto-generation (eg parsers, protocols, GUI's). iMatix even did it for C-based, enterprise software. So, there's certainly work on methods for mainstream software development of average coders with still high quality. Cleanroom was most successful in terms of defect rate. Eiffel pretty much made a market out of it by combining safer language, safe concurrency, OOP, and Design-by-Contract. They're still going strong. Haskell community probably most advanced given last item on my list with some financial companies using it. Jane Street's and Esterel's Ocaml work shows it has similar benefits. These have steeper learning curve but benefits add up quickly afterward. Cleanroom w/ Design-by-Contract and safe, high-level language seems to be best combo to deal with your list.
JoeAltmaier 10y ago
Many of the errors are just coding irregularities. Half are really good catches; but half are of the 'lint' sort. Still worth reviewing in case an actual mistake was made.
- pm215 10y ago
  Right, and so I've developed the view that the most interesting quality of a static analysis tool is not "how many bugs does it find" but "how many of the things it points out are false positives" and (if you want to be really tough) "how many of the things it points out are likely to cause problems for a real user or be actually exploitable by an attacker". Most software projects already have a huge list of bugs they haven't fixed yet, so a tool that identifies 10 easy to fix real bugs is great, but a tool that mixes those 10 real bugs in with a hundred false positives is much less useful, simply because of the time it takes somebody to sift through the list...
  - eremeevp 10y ago
    I would say, that in this case, with the 50\50 false positive ratio, it is quite manageable, i.e. for every 2 analyzer messages, there is 1 interesting issue in the source code.
  - nickpsecurity 10y ago
    Yes. I initially promoted all kinds of static analysis. These days, I'm firmly convinced the friction aspect means the tool needs to be plug-n-play with almost no false positives. Ideally no false positives at all. That maximizes uptake and knocks out many bugs even if more remain. Another thing I think advocates need to do with lists of such tools is always mention that each one might see errors another won't. This needs to be a refrain that instantly is in people's minds when they wonder about using such tools. Reason is that many make the mistakes of thinking one tool not finding anything means (a) tools don't find stuff in general or (b) there's nothing to find. That implication is even on SQLlite testing page where they applied Clang's analyzer then implied a weaker tool finding nothing in a good codebase meant static analysis itself had nothing to offer (?!).
deleted 10y ago
[deleted]
freddref 10y ago
People shouldn't be writing code at all. A lot of testing and small code corrections should be automated at the very least. We have the iron, lets put it to work.
- jschwartzi 10y ago
  So who writes the automation, then?
  - SEJeff 10y ago
    Skynet of course.
  - jjnoakes 10y ago
    Isn't that like asking "So who builds the assembly lines, then?" way back when?
- cocotino 10y ago
  People shouldn't be writing comments at all…
- eremeevp 10y ago
  The automation code will still contain much of the same errors. Besides, all modern IDE's provide multiple options for testing and refactoring (code corrections), but apparently this is not enough. At least this proves static analysis has plenty bugs to find after all of these other tools are applied.
- a-priori 10y ago
  As a side project, I'm actually working on something along those lines. https://github.com/michaelmelanson/autocoder https://github.com/michaelmelanson/autocoder My theory is that by reducing the search space enough, it should be possible for a computer to generate source code to pass unit tests. The key is from a blog post by Robert "Uncle Bob" Martin where he describes his 'Transformation priority premise' as a way of doing code kata exercises. I think this gives the basis for an automated code generation by presenting unit tests one at a time, and searching for transformations to make the tests pass (while not breaking any earlier ones of course). I have no idea if it'll be a practical way to write software, but I think it's an interesting problem to solve. I'm currently working through some kata exercises, producing the list of transformations manually, to make sure it has the vocabulary of transformations it needs. https://github.com/michaelmelanson/autocoder/blob/master/src/main/java/net/michaelmelanson/autocoder/AutocoderMain.java#L29-L50 https://github.com/michaelmelanson/autocoder/blob/master/src... After that I'll add a search algorithm to find the sequence of transformations automatically. Then we'll see if my theory is right! :) My vision is that this would be incorporated into an IDE such that you as the programmer write the unit tests, and the IDE works in the background to write the corresponding code to pass them.
AdmiralAsshat 10y ago
I don't know what tone the article title is going for. Is it meant to imply that open source software is somehow buggier than closed source? Or simply that they made open source software better by finding these bugs. If the former, it's worth noting that these are 10,000 bugs that are visible because the software is FOSS. Closed source software uses code analyzers/linters too. I worked on a piece of proprietary software where we had a giant spreadsheet for the several hundred bugs and other minor issues that flawfinder found on our software. Those will never get disclosed to the public.
- cocotino 10y ago
  The tone is the same as always: "buy our product". That's what all viva64 submissions are about.
  - Pasick 10y ago
    Do you expect them to say "Don't buy our product"?:)
    - cocotino 10y ago
      No but I expect them not to make it look like they just want to help OSS
      - EvgeniyZh 10y ago
        But they are not even trying. They declare it loud and clear: that's our marketing, but we're happy to help OSS
- wolfgke 10y ago
  > I don't know what tone the article title is going for. Did you really not? Then you have not read enough articles in the viva64 blog. :-) The tone is always "Static analysis with PVS-Studio is a great way to find bugs in projects and it would be a good way to use PVS-Studio for your projects, too".
spacemanmatt 10y ago
Sad but not surprising.
meira 10y ago
Now I challenge you to find 10000 software contributors.
- evgryz 10y ago
  You can find tons of tickets from PVS-Studio in many bug-trackers :-).
kazinator 10y ago
I just love this one: http://www.viva64.com/en/examples/V504/ http://www.viva64.com/en/examples/V504/ void GlyphComboBox::update(SPFont* spfont) { if (!spfont) return // <= //TODO: figure out why do we need to append("") // before clearing items properly... //Gtk is refusing to clear the combobox //when I comment out this line this->append(""); this->remove_all(); } Gee, why do you need the dummy append? It's because you put in so many TODO comments that you concealed the missing semicolon that you otherwise might have found ... This is arguably not a bug; the developer tested the code, massaged it into working as expected, using what look like well-defined constructs, and documented their surprise at certain aspects of the code, leaving this issue easy to find with a grep for TODO. On another note, what is surprising here is that they didn't catch it with their compiler. Look, the function returns void. GCC has a diagnostic for this, something like "return with expression in a function returning void". I think that still kicks in even if that expression has void type.
- mrSugar 10y ago
  Perhaps the append() also returns void. I've seen this kind of bug uncaught by gcc exactly because of that.
- BruiseLee 10y ago
  Gcc will not issue diagnostic if you return a "void expression" because doing so is valid in C++.
  - kazinator 10y ago
    It could still be a warning, since doing so is dubious in C++ even if it is allowed.
- mgrennan 10y ago
  Explains why stile matters. ... if (!spfont) return; ... Just add the CR and all is reveled.
woah 10y ago
Here are the classes of bugs: http://www.viva64.com/en/examples/ http://www.viva64.com/en/examples/ I'm no expert, but it looks like most of these would be caught by the Rust compiler at the time they were typed in.
- stebalien 10y ago
  Yes. That's why they don't like rust: http://www.viva64.com/en/b/0324/ http://www.viva64.com/en/b/0324/
  - jcranmer 10y ago
    That ... is pretty jarring. Their arguments boil down to: a) No one wants to use a "minor" language, so it will never get traction b) It's slow (citation: the Benchmarks Game) c) Static and dynamic analysis can find anything anyways, so what's the point? First off, anyone who is seriously citing the Benchmarks Game as to making a definitive declaration of whether a language is slow or fast is intellectually dishonest. It's a game (as its name illustrates), so it reflects more the effort that people put in to try to optimize code rather than a comparison of "idiomatic" code. A case in point: the fastest program in C uses SSE intrinsics for its implementation. Given that they're selling a static analyzer, I would expect them to make the case for analysis. However, it's worth addressing the limitations. Almost no project has 100% test coverage, and even having test coverage isn't necessarily a cure for having no problems (IOC still found undefined overflows in SQLite, which is the only open source project I know of to have 100% line and branch coverage in their test suite). To my knowledge, there still exists no tools that can identify strict-aliasing violations or ODR violations. It's also worth minding Dijkstra's aphorism: testing cannot prove the absence of bugs, only their presence. Static analysis in C/C++ has to face the additional problem of the inability to write a precise pointer analysis--something that Rust's type-safety would fix.
    - EvgeniyZh 10y ago
      > First off, anyone who is seriously citing the Benchmarks Game as to making a definitive declaration of whether a language is slow or fast is intellectually dishonest. But Rust is actually significantly slower than C++, that's for sure. You're welcome to test it by yourself.
tgarma1234 10y ago
In other news: people who do things in their spare time often make mistakes and do slipshod work.
- throw0219830 10y ago
  You clearly haven't worked in a commercial closed source setting.
- topspin 10y ago
  Projects like Chromium and ChakraCore are not being written in anyone's "spare time." It would be nice if people would stop conflating Open Source with unpaid hobby work. The projects analyzed in this story represent billions in paid labor.
  - tgarma1234 10y ago
    In other news: people who have to focus on detail oriented tasks often make mistakes. Happy now? Sheesh.
    - topspin 10y ago
      people who have to focus on detail oriented tasks often make mistakes Whether they're paid well or not.

What I take from this: computer languages (C++ especially, but also systems built in C) are too complex for the average mind No, I'm not significantly smarter than the average programmer That's what needs to be addressed. While C++ facilitates things, it is complex. While C is very simple it makes mistakes easier by requiring things to be done manually IDEs don't help. Code completion is helpful but it is of limited help And we keep repeating ourselves every time we write a for loop

informatimago 10y ago
Switch to Common Lisp. And with lisp, you don't have to repeat yourself: any pattern, or any boilerplate can be abstracted away with a macro.
- bluGill 10y ago
  > can be abstracted away Except for too many levels of abstraction.
  - spacemanmatt 10y ago
    Is that the fault of the programmer or the language, though?
    - steveklabnik 10y ago
      It's a reference to https://en.wikipedia.org/wiki/Fundamental_theorem_of_software_engineering https://en.wikipedia.org/wiki/Fundamental_theorem_of_softwar... (and the "humorous clause" mentioned there as well)
  - raverbashing 10y ago
    Yes, I believe one of the problems is the way things are abstracted, and I don't see a language solving this Adding more levels can only go so far But I don't know how to do it better. Module/Class/Method won't cut it anymore
  - maxerickson 10y ago
    That's kind of what some compiler optimizations do isn't it? Look through the abstraction and discard it?
- golergka 10y ago
  The fact that something can be done doesn't mean that it should be done.
- vbezhenar 10y ago
  How can CL prevent something like "i1 == i1" instead of "i1 == i2" error?
  - jjnoakes 10y ago
    First, since the AST is so simple to walk programmatically, one could write a simple check for that kind of bug in, I'm guessing, a few lines of Lisp. Second, just because you can come up with a bug that can be written in a language doesn't mean the language doesn't prevent all sorts of other bugs. All languages can have bugs written in them. That's not the point.
n00b101 10y ago
>computer languages (C++ especially, but also systems built in C) are too complex for the average mind I'm not necessarily disagreeing, but have you looked at the source code [1] of the Apollo Guidance computer [2]? This software was shipped to production 50 years ago. It literally implements a "self-driving space rocket." It is staggering that this was completed in such a short time and actually worked. The code is written entirely in the "Apollo Guidance Computer (AGC) Assembly" language. The hardware that it was written for had to be invented in at the same time, and was the world's first computer to use Integrated Circuits ("chips"). They also implemented a Real-Time Operating System with a multi-task priority scheduler, as well as a virtual machine and assembly interpreter within the software. This was mission and life safety critical code, the risk involved boggles the mind. And yet the software was incredibly reliable. It was so reliable that it even saved the Apollo 11 crew and mission during the lunar descent by famously recovering from a peripheral hardware bug. I honestly do not know how this feat was even humanly possible, but I do know that these programmers did not rely on "code completion." [1] https://github.com/chrislgarry/Apollo-11 https://github.com/chrislgarry/Apollo-11 [2] https://en.wikipedia.org/wiki/Apollo_Guidance_Computer https://en.wikipedia.org/wiki/Apollo_Guidance_Computer
- emodendroket 10y ago
  Yeah, sure. But if they'd had code completion it would have been a help, not a hindrance.
- drzaiusapelord 10y ago
  >I honestly do not know how this feat was even humanly possible The Apollo program, fixed for inflation, was something like $130 billion dollars. That's buys you a lot of talent, testing, etc. Its also worth noting that the program wasn't without its screw ups. The Apollo 1 deaths. The radar control on Apollo 11 eating up all the CPU during lunar landing. Apollo 13's famous O2 tank explosion. I think you may be glamorizing something that was well done, but falls within the realm of 'you get what you pay for' and the occasional human error, which in the case of Apollo 1, turned out to be fatal. I wouldn't call a program with three deaths and an aborted lunar insert to be infallible. That said, NASA/JPL publish code standards, although I imagine most tech companies would balk at giving up their toys and hot languages/frameworks to do fairly tedious work like this. http://lars-lab.jpl.nasa.gov/ http://lars-lab.jpl.nasa.gov/
emodendroket 10y ago
This also includes analysis of C#, which is quite different from C.
- raverbashing 10y ago
  Yes, while C#/Java will prevent some kind of errors from happening, others are very common (Example: copy/paste errors, parameter order problems, etc)
spriggan3 10y ago
> While C is very simple it makes mistakes easier by requiring things to be done manually I wouldn't describe C as simple but basic. Of course, it's start being less basic with all the macros and all the tools needed to compile and distribute a C program , like autoconf ... with all these layers , C becomes as complicated as any other language because it relies on so many 3rd party tools (which are often dynamic languages) to fix its shortcomings. But as long as linux kernel is written in C the language isn't going anywhere.
Pasick 10y ago
I wouldn't say that the number of errors in code directly depends on complexity of the language. From my practice I learned that the code quality depends on a software development process implemented in your team. Now we have whole bunch of practices and tools that allow us to minimize risks of appearing stupid mistakes in code: covering the code with unit and integration tests, static and dynamic code analysis, continuous integration, code reviews, refactoring and so on, and, I should say, these things work. So, I think, this article demonstrates that still there are a lot of teams whose development process can be improved.
nickpsecurity 10y ago
That's actually the problem Ada's designers set out to solve. They add to what they solve every iteration with safe defaults and contraints on use. Whether you do Ada or not, you will likely find skimming this book to be helpful to see how one can systematically analyze all forms of programmer error and attempt to eliminate them: http://www.adacore.com/uploads/technical-papers/SafeSecureAdav2015-covered.pdf http://www.adacore.com/uploads/technical-papers/SafeSecureAd... That's at language level. At process level, Harlan Mills developed Cleanroom back in 80's to try to get software defects as low as cleanrooms CPU's were built in. It combines a formal approach of describing/connecting things, constrained implementation for easy verification, iterative development, and usage-based testing. Defect rate was extremely low even for first use by teams with cost ranging from slightly more to cheaper due to less debugging. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.88.2667&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.88.... On productivity side, the DSL-based approach on top of safe, functional languages seems to be winning out. Safe, functional combo already knocks out a lot of problems. DSL's for common categories of programming let you read and write less code to get same work done possibly safer with sane auto-generation (eg parsers, protocols, GUI's). iMatix even did it for C-based, enterprise software. So, there's certainly work on methods for mainstream software development of average coders with still high quality. Cleanroom was most successful in terms of defect rate. Eiffel pretty much made a market out of it by combining safer language, safe concurrency, OOP, and Design-by-Contract. They're still going strong. Haskell community probably most advanced given last item on my list with some financial companies using it. Jane Street's and Esterel's Ocaml work shows it has similar benefits. These have steeper learning curve but benefits add up quickly afterward. Cleanroom w/ Design-by-Contract and safe, high-level language seems to be best combo to deal with your list.

JoeAltmaier 10y ago

Many of the errors are just coding irregularities. Half are really good catches; but half are of the 'lint' sort. Still worth reviewing in case an actual mistake was made.

pm215 10y ago
Right, and so I've developed the view that the most interesting quality of a static analysis tool is not "how many bugs does it find" but "how many of the things it points out are false positives" and (if you want to be really tough) "how many of the things it points out are likely to cause problems for a real user or be actually exploitable by an attacker". Most software projects already have a huge list of bugs they haven't fixed yet, so a tool that identifies 10 easy to fix real bugs is great, but a tool that mixes those 10 real bugs in with a hundred false positives is much less useful, simply because of the time it takes somebody to sift through the list...
- eremeevp 10y ago
  I would say, that in this case, with the 50\50 false positive ratio, it is quite manageable, i.e. for every 2 analyzer messages, there is 1 interesting issue in the source code.
- nickpsecurity 10y ago
  Yes. I initially promoted all kinds of static analysis. These days, I'm firmly convinced the friction aspect means the tool needs to be plug-n-play with almost no false positives. Ideally no false positives at all. That maximizes uptake and knocks out many bugs even if more remain. Another thing I think advocates need to do with lists of such tools is always mention that each one might see errors another won't. This needs to be a refrain that instantly is in people's minds when they wonder about using such tools. Reason is that many make the mistakes of thinking one tool not finding anything means (a) tools don't find stuff in general or (b) there's nothing to find. That implication is even on SQLlite testing page where they applied Clang's analyzer then implied a weaker tool finding nothing in a good codebase meant static analysis itself had nothing to offer (?!).

deleted 10y ago

[deleted]

freddref 10y ago

People shouldn't be writing code at all. A lot of testing and small code corrections should be automated at the very least. We have the iron, lets put it to work.

jschwartzi 10y ago
So who writes the automation, then?
- SEJeff 10y ago
  Skynet of course.
- jjnoakes 10y ago
  Isn't that like asking "So who builds the assembly lines, then?" way back when?
cocotino 10y ago
People shouldn't be writing comments at all…
eremeevp 10y ago
The automation code will still contain much of the same errors. Besides, all modern IDE's provide multiple options for testing and refactoring (code corrections), but apparently this is not enough. At least this proves static analysis has plenty bugs to find after all of these other tools are applied.
a-priori 10y ago
As a side project, I'm actually working on something along those lines. https://github.com/michaelmelanson/autocoder https://github.com/michaelmelanson/autocoder My theory is that by reducing the search space enough, it should be possible for a computer to generate source code to pass unit tests. The key is from a blog post by Robert "Uncle Bob" Martin where he describes his 'Transformation priority premise' as a way of doing code kata exercises. I think this gives the basis for an automated code generation by presenting unit tests one at a time, and searching for transformations to make the tests pass (while not breaking any earlier ones of course). I have no idea if it'll be a practical way to write software, but I think it's an interesting problem to solve. I'm currently working through some kata exercises, producing the list of transformations manually, to make sure it has the vocabulary of transformations it needs. https://github.com/michaelmelanson/autocoder/blob/master/src/main/java/net/michaelmelanson/autocoder/AutocoderMain.java#L29-L50 https://github.com/michaelmelanson/autocoder/blob/master/src... After that I'll add a search algorithm to find the sequence of transformations automatically. Then we'll see if my theory is right! :) My vision is that this would be incorporated into an IDE such that you as the programmer write the unit tests, and the IDE works in the background to write the corresponding code to pass them.

AdmiralAsshat 10y ago

I don't know what tone the article title is going for. Is it meant to imply that open source software is somehow buggier than closed source? Or simply that they made open source software better by finding these bugs. If the former, it's worth noting that these are 10,000 bugs that are visible because the software is FOSS. Closed source software uses code analyzers/linters too. I worked on a piece of proprietary software where we had a giant spreadsheet for the several hundred bugs and other minor issues that flawfinder found on our software. Those will never get disclosed to the public.

cocotino 10y ago
The tone is the same as always: "buy our product". That's what all viva64 submissions are about.
- Pasick 10y ago
  Do you expect them to say "Don't buy our product"?:)
  - cocotino 10y ago
    No but I expect them not to make it look like they just want to help OSS
    - EvgeniyZh 10y ago
      But they are not even trying. They declare it loud and clear: that's our marketing, but we're happy to help OSS
wolfgke 10y ago
> I don't know what tone the article title is going for. Did you really not? Then you have not read enough articles in the viva64 blog. :-) The tone is always "Static analysis with PVS-Studio is a great way to find bugs in projects and it would be a good way to use PVS-Studio for your projects, too".

spacemanmatt 10y ago

Sad but not surprising.

meira 10y ago

Now I challenge you to find 10000 software contributors.

evgryz 10y ago
You can find tons of tickets from PVS-Studio in many bug-trackers :-).

kazinator 10y ago

I just love this one: http://www.viva64.com/en/examples/V504/ http://www.viva64.com/en/examples/V504/ void GlyphComboBox::update(SPFont* spfont) { if (!spfont) return // <= //TODO: figure out why do we need to append("") // before clearing items properly... //Gtk is refusing to clear the combobox //when I comment out this line this->append(""); this->remove_all(); } Gee, why do you need the dummy append? It's because you put in so many TODO comments that you concealed the missing semicolon that you otherwise might have found ... This is arguably not a bug; the developer tested the code, massaged it into working as expected, using what look like well-defined constructs, and documented their surprise at certain aspects of the code, leaving this issue easy to find with a grep for TODO. On another note, what is surprising here is that they didn't catch it with their compiler. Look, the function returns void. GCC has a diagnostic for this, something like "return with expression in a function returning void". I think that still kicks in even if that expression has void type.

mrSugar 10y ago
Perhaps the append() also returns void. I've seen this kind of bug uncaught by gcc exactly because of that.
BruiseLee 10y ago
Gcc will not issue diagnostic if you return a "void expression" because doing so is valid in C++.
- kazinator 10y ago
  It could still be a warning, since doing so is dubious in C++ even if it is allowed.
mgrennan 10y ago
Explains why stile matters. ... if (!spfont) return; ... Just add the CR and all is reveled.

woah 10y ago

Here are the classes of bugs: http://www.viva64.com/en/examples/ http://www.viva64.com/en/examples/ I'm no expert, but it looks like most of these would be caught by the Rust compiler at the time they were typed in.

stebalien 10y ago
Yes. That's why they don't like rust: http://www.viva64.com/en/b/0324/ http://www.viva64.com/en/b/0324/
- jcranmer 10y ago
  That ... is pretty jarring. Their arguments boil down to: a) No one wants to use a "minor" language, so it will never get traction b) It's slow (citation: the Benchmarks Game) c) Static and dynamic analysis can find anything anyways, so what's the point? First off, anyone who is seriously citing the Benchmarks Game as to making a definitive declaration of whether a language is slow or fast is intellectually dishonest. It's a game (as its name illustrates), so it reflects more the effort that people put in to try to optimize code rather than a comparison of "idiomatic" code. A case in point: the fastest program in C uses SSE intrinsics for its implementation. Given that they're selling a static analyzer, I would expect them to make the case for analysis. However, it's worth addressing the limitations. Almost no project has 100% test coverage, and even having test coverage isn't necessarily a cure for having no problems (IOC still found undefined overflows in SQLite, which is the only open source project I know of to have 100% line and branch coverage in their test suite). To my knowledge, there still exists no tools that can identify strict-aliasing violations or ODR violations. It's also worth minding Dijkstra's aphorism: testing cannot prove the absence of bugs, only their presence. Static analysis in C/C++ has to face the additional problem of the inability to write a precise pointer analysis--something that Rust's type-safety would fix.
  - EvgeniyZh 10y ago
    > First off, anyone who is seriously citing the Benchmarks Game as to making a definitive declaration of whether a language is slow or fast is intellectually dishonest. But Rust is actually significantly slower than C++, that's for sure. You're welcome to test it by yourself.

tgarma1234 10y ago

In other news: people who do things in their spare time often make mistakes and do slipshod work.

throw0219830 10y ago
You clearly haven't worked in a commercial closed source setting.
topspin 10y ago
Projects like Chromium and ChakraCore are not being written in anyone's "spare time." It would be nice if people would stop conflating Open Source with unpaid hobby work. The projects analyzed in this story represent billions in paid labor.
- tgarma1234 10y ago
  In other news: people who have to focus on detail oriented tasks often make mistakes. Happy now? Sheesh.
  - topspin 10y ago
    people who have to focus on detail oriented tasks often make mistakes Whether they're paid well or not.