3 ms·

This is an interesting article! I think more of this kind of bug-type/origin-breakdown is a good thing to have. This and the Curl article make me wonder what f

This is an interesting article! I think more of this kind of bug-type/origin-breakdown is a good thing to have. This and the Curl article make me wonder what factors impact the proportion of security bugs which are memory safety bugs. The most obvious variable is codebase size - Microsoft/Chromium both are huge codebases, with multiple millions (billions?) of lines of code each. Curl appears to clock in closer to 100k, and Daniel states "51% (22 of 43) of the issues rated high or critical was due to C mistakes." bc appears to have ~20k lines of code, and this article gives C bug rate closer to 10%. Feature set/attack surface area feels like another important variable, though I'm sure there are arguments for others as well. I'd be nice if we had more directly comparable numbers as well. The denominator for Microsoft's 70% number is "number of CVEs". The denominator for Chromium's number is "high severity security bugs". Curl's 51% number uses a similar metric, and I think the denominator for bc's ~10% number is just "bugs" (though to be fair, OP does discuss this in the post). All that being said, I still think the data point introduced by this article is better to have than not.

gavinhoward 3y ago
Author here. Thank you. :) One thing I am going to do in my new project is catalog all bugs fixed after first release with the following data points: * Is it a C bug? * Is it in a stable module of the monorepo? (I will mark things stable after they have been tested.) * Is the bug in a release, even if it's in an unstable module? * Is it a build bug? * Is it a test bug? * Etc. I'll keep the catalog in a human- and machine-readable format. This should provide a good set of data. I would be thrilled if people downloaded my code for human study, analysis, and code archeology.
- pushfoo 3y ago
  Do you have an interface in mind to reduce friction for building the catalog on a daily basis? GitHub and Gitea don't really seem to provide clean ways to do this out of the box.
  - gavinhoward 3y ago
    I am going to do it by hand for the first while. Until I build my VCS, which will be like Fossil and have an integrated issue tracker that gets cloned with the repo. Then I'll put the catalog into my VCS by hand and track issues normally; the catalog will build itself as people report bugs.

gavinhoward 3y ago

Author here. Thank you. :) One thing I am going to do in my new project is catalog all bugs fixed after first release with the following data points: * Is it a C bug? * Is it in a stable module of the monorepo? (I will mark things stable after they have been tested.) * Is the bug in a release, even if it's in an unstable module? * Is it a build bug? * Is it a test bug? * Etc. I'll keep the catalog in a human- and machine-readable format. This should provide a good set of data. I would be thrilled if people downloaded my code for human study, analysis, and code archeology.

pushfoo 3y ago
Do you have an interface in mind to reduce friction for building the catalog on a daily basis? GitHub and Gitea don't really seem to provide clean ways to do this out of the box.
- gavinhoward 3y ago
  I am going to do it by hand for the first while. Until I build my VCS, which will be like Fossil and have an integrated issue tracker that gets cloned with the repo. Then I'll put the catalog into my VCS by hand and track issues normally; the catalog will build itself as people report bugs.