3 ms·

> Achieving abstraction is super hard when you can just reach down into some bytes and noodle around with them instead. Disagree. In my experience all was wond

> Achieving abstraction is super hard when you can just reach down into some bytes and noodle around with them instead. Disagree. In my experience all was wonderful once I was past the point where I decided to just not worry. Much the opposite, so much pointless and confusing OOP boilerplate (as well as blood, sweat, and all that) goes into "securing" code against misuse. Invest some time into data structures with obvious meaning, or a procedural API that is easily understood. "Nobody" will "ever" misuse, and misuse will be easy to detect. Much better tradeoff IMO. And lastly, of course, abstraction has nothing to do with "security systems". It's against popular opinion, but you can in fact have good abstraction with nothing but plain functions and void pointers. I like to view abstraction as mostly a conceptual thing that doesn't even happen in code.

ekidd 8y ago
> "Nobody" will "ever" misuse, and misuse will be easy to detect... And lastly, of course, abstraction has nothing to do with "security systems". I've spent a lot of time digging around in other people's C code over the years, and with very few exceptions, the kind of low-level byte munging in the original post tends to be absolutely riddled with bugs. There are off-by-one errors, buffer overflows, integer overflows, and memory leaks. The error-handling code paths tend to be broken, too. And if you run a fuzzer like libfuzz or American Fuzzy Lop, it will almost always find even more vulnerabilities. (A handful of C programs do better, including djb's tools, SQLite, dovecot, and significant portions of Apache. Apache seems to mostly succeed because it has good abstractions for strings and buffers.) Back in 2002, I published a ~7,000 line XML-RPC library in C, with extensive unit tests. I ran it through multiple code quality tools, including Electric Fence. I spent a long time carefully examining each function for correctness. I chose Expat as my XML parser, because it was one of the best-written at the time. Overall, I thought I was an unusually paranoid and careful C programmer. But here's the list of CVEs against my library and—most importantly—its dependencies: https://people.canonical.com/~ubuntu-security/cve/pkg/xmlrpc-c.html https://people.canonical.com/~ubuntu-security/cve/pkg/xmlrpc... If anybody here writes C code, and if your code runs on potentially hostile data, I strongly recommend experimenting with a fuzzer. It can be a brutally humbling experience, even if you think you're an exceptionally careful programmer. And if you rely on anybody else's code, like I did, you inherit all their bugs.
- todd8 8y ago
  We need all the help from our programming languages (and tools) that we can get. Donald Knuth is a remarkable programmer and computer scientist. Interestingly, he kept a list of all the errors in his program TeX[1], which starts in March of 1978. TeX is an exceptionally stable program, the design has been frozen since version 3.0 in 1990 reflecting Knuth's desire to have a format suitable for reproducing the typesetting precisely even for archived texts that were developed on hardware that is no longer running. Since 1990 only minor changes have been made to TeX and this is reflected in the current version number, 3.14159265, which is converging to pi. A comment by Knuth in the error log in March of 1990 was "We’re now up to Version 3.0; I sincerely hope all bugs have been found." At that point there had been 908 errors logged. The last entry is currently an entry made in 2014 for error 947. I've always been fascinated by this list (see [1]). [1] http://texdoc.net/texmf-dist/doc/generic/knuth/errata/errorlog.pdf http://texdoc.net/texmf-dist/doc/generic/knuth/errata/errorl...
  - svat 8y ago
    That is a great list, and the paper that accompanies it is a must-read. Note that to make sense of the list (to put it in context, and what all the abbreviations stand for, etc), it is best to read the paper by Knuth. That's "The Errors of TeX", also reprinted in Literate Programming. (The DOI of the paper is 10.1002/spe.4380190702 and that may help one find and read it.) Just the abbreviations are also given in https://www.tug.org/TUGboat/tb10-4/tb26knut.pdf https://www.tug.org/TUGboat/tb10-4/tb26knut.pdf but the paper is much more.
- jstimpfle 8y ago
  I don't disagree, and thank you for the pointers. What I was talking about is API design, and I don't think you are contradicting with what I was saying. As to C being more bug prone, for my own code I believe the biggest improvement to correctness would come from fat pointers with array size information, enabling bounds checking. That would help to spot errors in little-used code. On the other hand, fat pointers and other sophisticated methods can get in the way of a clean normalized data organization. A lot more flexibility with respect to data and code organization is possible in plain C, which is a great enabler of software in the first place. In the end, if the code is simpler and shorter and has more deterministic performance, but each line of code is more bug prone, that can still be a worthwhile tradeoff. By the way, I don't view myself as a very careful C programmer and I don't have the patience to check my code against all kinds of UB and such. I've never done fuzzing (I might try it). But for example valgrind has revealed only few problems once my code was working, in the past. The main problem in my eyes is when coding C in an object-oriented style with endless pointers and lifetime issues. What I do is I try to avoid tree structures (including XML) since I feel it's a lot of peeking and poking and traversing with little reward. Instead, I typically use global data, flat arrays, and integer indices, and as a result the code usually becomes very straightforward. By the way, for a project the size of an XML parser I wouldn't feel shame for such a tiny list of CVEs :-).

ekidd 8y ago

> "Nobody" will "ever" misuse, and misuse will be easy to detect... And lastly, of course, abstraction has nothing to do with "security systems". I've spent a lot of time digging around in other people's C code over the years, and with very few exceptions, the kind of low-level byte munging in the original post tends to be absolutely riddled with bugs. There are off-by-one errors, buffer overflows, integer overflows, and memory leaks. The error-handling code paths tend to be broken, too. And if you run a fuzzer like libfuzz or American Fuzzy Lop, it will almost always find even more vulnerabilities. (A handful of C programs do better, including djb's tools, SQLite, dovecot, and significant portions of Apache. Apache seems to mostly succeed because it has good abstractions for strings and buffers.) Back in 2002, I published a ~7,000 line XML-RPC library in C, with extensive unit tests. I ran it through multiple code quality tools, including Electric Fence. I spent a long time carefully examining each function for correctness. I chose Expat as my XML parser, because it was one of the best-written at the time. Overall, I thought I was an unusually paranoid and careful C programmer. But here's the list of CVEs against my library and—most importantly—its dependencies: https://people.canonical.com/~ubuntu-security/cve/pkg/xmlrpc-c.html https://people.canonical.com/~ubuntu-security/cve/pkg/xmlrpc... If anybody here writes C code, and if your code runs on potentially hostile data, I strongly recommend experimenting with a fuzzer. It can be a brutally humbling experience, even if you think you're an exceptionally careful programmer. And if you rely on anybody else's code, like I did, you inherit all their bugs.

todd8 8y ago
We need all the help from our programming languages (and tools) that we can get. Donald Knuth is a remarkable programmer and computer scientist. Interestingly, he kept a list of all the errors in his program TeX[1], which starts in March of 1978. TeX is an exceptionally stable program, the design has been frozen since version 3.0 in 1990 reflecting Knuth's desire to have a format suitable for reproducing the typesetting precisely even for archived texts that were developed on hardware that is no longer running. Since 1990 only minor changes have been made to TeX and this is reflected in the current version number, 3.14159265, which is converging to pi. A comment by Knuth in the error log in March of 1990 was "We’re now up to Version 3.0; I sincerely hope all bugs have been found." At that point there had been 908 errors logged. The last entry is currently an entry made in 2014 for error 947. I've always been fascinated by this list (see [1]). [1] http://texdoc.net/texmf-dist/doc/generic/knuth/errata/errorlog.pdf http://texdoc.net/texmf-dist/doc/generic/knuth/errata/errorl...
- svat 8y ago
  That is a great list, and the paper that accompanies it is a must-read. Note that to make sense of the list (to put it in context, and what all the abbreviations stand for, etc), it is best to read the paper by Knuth. That's "The Errors of TeX", also reprinted in Literate Programming. (The DOI of the paper is 10.1002/spe.4380190702 and that may help one find and read it.) Just the abbreviations are also given in https://www.tug.org/TUGboat/tb10-4/tb26knut.pdf https://www.tug.org/TUGboat/tb10-4/tb26knut.pdf but the paper is much more.
jstimpfle 8y ago
I don't disagree, and thank you for the pointers. What I was talking about is API design, and I don't think you are contradicting with what I was saying. As to C being more bug prone, for my own code I believe the biggest improvement to correctness would come from fat pointers with array size information, enabling bounds checking. That would help to spot errors in little-used code. On the other hand, fat pointers and other sophisticated methods can get in the way of a clean normalized data organization. A lot more flexibility with respect to data and code organization is possible in plain C, which is a great enabler of software in the first place. In the end, if the code is simpler and shorter and has more deterministic performance, but each line of code is more bug prone, that can still be a worthwhile tradeoff. By the way, I don't view myself as a very careful C programmer and I don't have the patience to check my code against all kinds of UB and such. I've never done fuzzing (I might try it). But for example valgrind has revealed only few problems once my code was working, in the past. The main problem in my eyes is when coding C in an object-oriented style with endless pointers and lifetime issues. What I do is I try to avoid tree structures (including XML) since I feel it's a lot of peeking and poking and traversing with little reward. Instead, I typically use global data, flat arrays, and integer indices, and as a result the code usually becomes very straightforward. By the way, for a project the size of an XML parser I wouldn't feel shame for such a tiny list of CVEs :-).