6 ms·

Am I a Good C Programmer?

33 points by gavinhoward 3y ago

sweetjuly 3y ago
It doesn't look like bc uses fuzzing? It'd be interesting to see the outcome there since even in mature projects it consistently leads to novel memory safety bugs every few days [1]. Fuzzing find a lot of really weird and unintuitive bugs in my experience, especially as it relates to use-after-frees and races. [1] Linux syzkaller bug tracker https://groups.google.com/g/syzkaller-bugs https://groups.google.com/g/syzkaller-bugs
- gavinhoward 3y ago
  Author here. My bc does use fuzzing. [1] I love fuzzing. In my new project, fuzzing generates my test suite that I then go through by hand and verify. In bc, if I change the source code before a release, I run a fuzzer for two weeks on both bc and dc. [2] and [3] are directories of error tests, almost all of which were found by AFL(++). If you run the following on my bc repo: commits1=$(git log --oneline --grep=afl) commits2=$(git log --oneline --grep=AFL) printf '%s\n%s\n' "$commits1" "$commits2" | uniq | wc -l You will get 148 as of 53832b884b5c. Not all of those are bugs found by AFL(++) because some are tending to my fuzzing infrastructure, but at least 2/3 probably are. [1]: https://git.gavinhoward.com/gavin/bc/src/branch/master/tests/fuzzing https://git.gavinhoward.com/gavin/bc/src/branch/master/tests... [2]: https://git.gavinhoward.com/gavin/bc/src/branch/master/tests/bc/errors https://git.gavinhoward.com/gavin/bc/src/branch/master/tests... [3]: https://git.gavinhoward.com/gavin/bc/src/branch/master/tests/dc/errors https://git.gavinhoward.com/gavin/bc/src/branch/master/tests...
aeonik 3y ago
Those links say 70% of all security bugs, but since my bc/dc does not have the ! command (to spawn processes) and only reads files, not writes them (except stdout), it’s hard to actually turn a memory safety issue into a security bug, even if it’s possible. Is this quote assuming ASLR, non-executable stack, stack canaries? I don't really understand what writing files and forking have to do with the security of system? Unless you mean it limits the attack surface from a ROP gadget?
- gavinhoward 3y ago
  Author here. It means that you have to find input that triggers a memory bug to do something malicious. If my dc had the `!` command, then you, as an attacker with low privileges, might be able to modify a dc script to spawn another command that you know can be exploited to get root. Without it, you have to exploit my dc for ROP or some other attack. Of course, then you would just target a shell, right? And that's the point: I've put in enough work, and my bc/dc are locked down enough, that attackers would probably find other ways in.
commandlinefan 3y ago
I was part of a team at a job about 20 years ago that ported an existing C codebase over to (then new) Java. Once we got everything working, a co-worker who was very familiar with the old C codebase remarked that with C it took a lot longer to get things working the first time, but once they were working, they stayed working whereas with Java you could get things working pretty quick, but you'd start to see problems when you actually put load on it.
- gavinhoward 3y ago
  Author here. This is my experience, though I started in Java and transitioned to C. My C code also does not need to be updated for language changes and such.
  - canadianfella 3y ago
    [dead]
  - conor- 3y ago
    How do you feel about Go if you've ever worked with it? It seems like it was intended to solve exactly those types of pain points associated with maintaining Java programs
    - gavinhoward 3y ago
      Go is fine, but I'm not really a fan.
    - jakogut 3y ago
      I'm not the person you responded to, but my biggest complaint with Go is binary size. It makes it rather unusable for low level programs on resource constrained systems.
pjmlp 3y ago
This is the usual argument, "I am a good careful driver, only the other ones do mistakes, so we don't need seat belts".
- PH95VuimJjqBqy 3y ago
  I think comparing an activity that can result in loss of life to one that doesn't is probably on the dishonest side.
  - baydarr 3y ago
    What language is used to control critical devices like pacemakers?
    - r9295 3y ago
      Or fly airplanes for that matter
    - PH95VuimJjqBqy 3y ago
      then make different decisions for those devices. That was actually the point being made.
  - pjmlp 3y ago
    Our lives and livelyhood depend on software on any modern society. Additionally people have died due to software bugs, Therac-25 being quite famous, with its Assembly optimization tricks.
    - PH95VuimJjqBqy 3y ago
      a plumbers livelihood depends on his tools but no one in their right mind would compare a hammer to a seatbelt like this. The reason people like you are not listened to is that you're so unreasonable. It's not an accident that ada was designed years ago for software that IS critical in this manner, because different decisions get made. But the vast majority of software is not going to end up with people dead, hence why arguments comparing life-or-death situations with your run-of-the-mill C code is assinine. Yes, it's true that GCC code will never live under the same level of scrutiny as NASA code, nor should it.
HackerThemAll 3y ago
Come on, are you really comparing a calculator to any web API server that is exposed 24/7/365 to malicious people trying to exploit it using various ways, which has many network endpoints with various HTTP methods, authentication, authorization, RBAC, using databases, connection encryption, randomness, hashing, maybe compression, graphics/video/audio conversion, caching and so on? I'm in love with "bc", I don't know if I use your version or not, but well... it's a calculator... It's well written, it does the job, but come on...
- gavinhoward 3y ago
  Author here. I'm currently making a build system and a VCS with a server that will be exposed 24/7/365 with HTTP, authentication, authorization RBAC, a database, connection encryption, randomness, hashing, compression, caching, and streaming of large files. bc is my only current data point since this project is not finished, but once it is, we'll start to get real data about what I can do.
aw1621107 3y ago
This is an interesting article! I think more of this kind of bug-type/origin-breakdown is a good thing to have. This and the Curl article make me wonder what factors impact the proportion of security bugs which are memory safety bugs. The most obvious variable is codebase size - Microsoft/Chromium both are huge codebases, with multiple millions (billions?) of lines of code each. Curl appears to clock in closer to 100k, and Daniel states "51% (22 of 43) of the issues rated high or critical was due to C mistakes." bc appears to have ~20k lines of code, and this article gives C bug rate closer to 10%. Feature set/attack surface area feels like another important variable, though I'm sure there are arguments for others as well. I'd be nice if we had more directly comparable numbers as well. The denominator for Microsoft's 70% number is "number of CVEs". The denominator for Chromium's number is "high severity security bugs". Curl's 51% number uses a similar metric, and I think the denominator for bc's ~10% number is just "bugs" (though to be fair, OP does discuss this in the post). All that being said, I still think the data point introduced by this article is better to have than not.
- gavinhoward 3y ago
  Author here. Thank you. :) One thing I am going to do in my new project is catalog all bugs fixed after first release with the following data points: * Is it a C bug? * Is it in a stable module of the monorepo? (I will mark things stable after they have been tested.) * Is the bug in a release, even if it's in an unstable module? * Is it a build bug? * Is it a test bug? * Etc. I'll keep the catalog in a human- and machine-readable format. This should provide a good set of data. I would be thrilled if people downloaded my code for human study, analysis, and code archeology.
  - pushfoo 3y ago
    Do you have an interface in mind to reduce friction for building the catalog on a daily basis? GitHub and Gitea don't really seem to provide clean ways to do this out of the box.
    - gavinhoward 3y ago
      I am going to do it by hand for the first while. Until I build my VCS, which will be like Fossil and have an integrated issue tracker that gets cloned with the repo. Then I'll put the catalog into my VCS by hand and track issues normally; the catalog will build itself as people report bugs.
pushfoo 3y ago
There's an interesting question raised by the linked "rant": should updating copyright year serve as a sign of life for a project? I'm currently inclined to think that: 1. Yes, it should 2. The OP of the linked issue may be mistaking a tools / social conventions issue for an error by the author
xedrac 3y ago
I used C for a many years and generally liked it. I then spent the following 10 years using C++ and had mixed feelings about it. I now use Rust and I'm not sure I could ever in good faith write a new project in C or C++ again. Anyway, we're both members of the same church! :)
reify 3y ago
simple answer? Nah! Just pathetically good at seeking out some ego stroking I hate posts like this one. They always start off with a banal post heading Then becomes a way of arguing the posters hiden agenda
- gavinhoward 3y ago
  I mean, to be fair, I put a warning about the ego stroking upfront. :)

33 points by gavinhoward 3y ago

It doesn't look like bc uses fuzzing? It'd be interesting to see the outcome there since even in mature projects it consistently leads to novel memory safety bugs every few days [1]. Fuzzing find a lot of really weird and unintuitive bugs in my experience, especially as it relates to use-after-frees and races. [1] Linux syzkaller bug tracker https://groups.google.com/g/syzkaller-bugs https://groups.google.com/g/syzkaller-bugs

gavinhoward 3y ago
Author here. My bc does use fuzzing. [1] I love fuzzing. In my new project, fuzzing generates my test suite that I then go through by hand and verify. In bc, if I change the source code before a release, I run a fuzzer for two weeks on both bc and dc. [2] and [3] are directories of error tests, almost all of which were found by AFL(++). If you run the following on my bc repo: commits1=$(git log --oneline --grep=afl) commits2=$(git log --oneline --grep=AFL) printf '%s\n%s\n' "$commits1" "$commits2" | uniq | wc -l You will get 148 as of 53832b884b5c. Not all of those are bugs found by AFL(++) because some are tending to my fuzzing infrastructure, but at least 2/3 probably are. [1]: https://git.gavinhoward.com/gavin/bc/src/branch/master/tests/fuzzing https://git.gavinhoward.com/gavin/bc/src/branch/master/tests... [2]: https://git.gavinhoward.com/gavin/bc/src/branch/master/tests/bc/errors https://git.gavinhoward.com/gavin/bc/src/branch/master/tests... [3]: https://git.gavinhoward.com/gavin/bc/src/branch/master/tests/dc/errors https://git.gavinhoward.com/gavin/bc/src/branch/master/tests...

aeonik 3y ago

Those links say 70% of all security bugs, but since my bc/dc does not have the ! command (to spawn processes) and only reads files, not writes them (except stdout), it’s hard to actually turn a memory safety issue into a security bug, even if it’s possible. Is this quote assuming ASLR, non-executable stack, stack canaries? I don't really understand what writing files and forking have to do with the security of system? Unless you mean it limits the attack surface from a ROP gadget?

gavinhoward 3y ago
Author here. It means that you have to find input that triggers a memory bug to do something malicious. If my dc had the `!` command, then you, as an attacker with low privileges, might be able to modify a dc script to spawn another command that you know can be exploited to get root. Without it, you have to exploit my dc for ROP or some other attack. Of course, then you would just target a shell, right? And that's the point: I've put in enough work, and my bc/dc are locked down enough, that attackers would probably find other ways in.

commandlinefan 3y ago

I was part of a team at a job about 20 years ago that ported an existing C codebase over to (then new) Java. Once we got everything working, a co-worker who was very familiar with the old C codebase remarked that with C it took a lot longer to get things working the first time, but once they were working, they stayed working whereas with Java you could get things working pretty quick, but you'd start to see problems when you actually put load on it.

gavinhoward 3y ago
Author here. This is my experience, though I started in Java and transitioned to C. My C code also does not need to be updated for language changes and such.
- canadianfella 3y ago
  [dead]
- conor- 3y ago
  How do you feel about Go if you've ever worked with it? It seems like it was intended to solve exactly those types of pain points associated with maintaining Java programs
  - gavinhoward 3y ago
    Go is fine, but I'm not really a fan.
  - jakogut 3y ago
    I'm not the person you responded to, but my biggest complaint with Go is binary size. It makes it rather unusable for low level programs on resource constrained systems.

pjmlp 3y ago

This is the usual argument, "I am a good careful driver, only the other ones do mistakes, so we don't need seat belts".

PH95VuimJjqBqy 3y ago
I think comparing an activity that can result in loss of life to one that doesn't is probably on the dishonest side.
- baydarr 3y ago
  What language is used to control critical devices like pacemakers?
  - r9295 3y ago
    Or fly airplanes for that matter
  - PH95VuimJjqBqy 3y ago
    then make different decisions for those devices. That was actually the point being made.
- pjmlp 3y ago
  Our lives and livelyhood depend on software on any modern society. Additionally people have died due to software bugs, Therac-25 being quite famous, with its Assembly optimization tricks.
  - PH95VuimJjqBqy 3y ago
    a plumbers livelihood depends on his tools but no one in their right mind would compare a hammer to a seatbelt like this. The reason people like you are not listened to is that you're so unreasonable. It's not an accident that ada was designed years ago for software that IS critical in this manner, because different decisions get made. But the vast majority of software is not going to end up with people dead, hence why arguments comparing life-or-death situations with your run-of-the-mill C code is assinine. Yes, it's true that GCC code will never live under the same level of scrutiny as NASA code, nor should it.

HackerThemAll 3y ago

Come on, are you really comparing a calculator to any web API server that is exposed 24/7/365 to malicious people trying to exploit it using various ways, which has many network endpoints with various HTTP methods, authentication, authorization, RBAC, using databases, connection encryption, randomness, hashing, maybe compression, graphics/video/audio conversion, caching and so on? I'm in love with "bc", I don't know if I use your version or not, but well... it's a calculator... It's well written, it does the job, but come on...

gavinhoward 3y ago
Author here. I'm currently making a build system and a VCS with a server that will be exposed 24/7/365 with HTTP, authentication, authorization RBAC, a database, connection encryption, randomness, hashing, compression, caching, and streaming of large files. bc is my only current data point since this project is not finished, but once it is, we'll start to get real data about what I can do.

aw1621107 3y ago

This is an interesting article! I think more of this kind of bug-type/origin-breakdown is a good thing to have. This and the Curl article make me wonder what factors impact the proportion of security bugs which are memory safety bugs. The most obvious variable is codebase size - Microsoft/Chromium both are huge codebases, with multiple millions (billions?) of lines of code each. Curl appears to clock in closer to 100k, and Daniel states "51% (22 of 43) of the issues rated high or critical was due to C mistakes." bc appears to have ~20k lines of code, and this article gives C bug rate closer to 10%. Feature set/attack surface area feels like another important variable, though I'm sure there are arguments for others as well. I'd be nice if we had more directly comparable numbers as well. The denominator for Microsoft's 70% number is "number of CVEs". The denominator for Chromium's number is "high severity security bugs". Curl's 51% number uses a similar metric, and I think the denominator for bc's ~10% number is just "bugs" (though to be fair, OP does discuss this in the post). All that being said, I still think the data point introduced by this article is better to have than not.

gavinhoward 3y ago
Author here. Thank you. :) One thing I am going to do in my new project is catalog all bugs fixed after first release with the following data points: * Is it a C bug? * Is it in a stable module of the monorepo? (I will mark things stable after they have been tested.) * Is the bug in a release, even if it's in an unstable module? * Is it a build bug? * Is it a test bug? * Etc. I'll keep the catalog in a human- and machine-readable format. This should provide a good set of data. I would be thrilled if people downloaded my code for human study, analysis, and code archeology.
- pushfoo 3y ago
  Do you have an interface in mind to reduce friction for building the catalog on a daily basis? GitHub and Gitea don't really seem to provide clean ways to do this out of the box.
  - gavinhoward 3y ago
    I am going to do it by hand for the first while. Until I build my VCS, which will be like Fossil and have an integrated issue tracker that gets cloned with the repo. Then I'll put the catalog into my VCS by hand and track issues normally; the catalog will build itself as people report bugs.

pushfoo 3y ago

There's an interesting question raised by the linked "rant": should updating copyright year serve as a sign of life for a project? I'm currently inclined to think that: 1. Yes, it should 2. The OP of the linked issue may be mistaking a tools / social conventions issue for an error by the author

xedrac 3y ago

I used C for a many years and generally liked it. I then spent the following 10 years using C++ and had mixed feelings about it. I now use Rust and I'm not sure I could ever in good faith write a new project in C or C++ again. Anyway, we're both members of the same church! :)

reify 3y ago

simple answer? Nah! Just pathetically good at seeking out some ego stroking I hate posts like this one. They always start off with a banal post heading Then becomes a way of arguing the posters hiden agenda

gavinhoward 3y ago
I mean, to be fair, I put a warning about the ego stroking upfront. :)