4 ms·

I only have cursory experience with C, mostly by programming some Arduino stuff, so I can only add this: good god this is a minefield.

rwmj 4y ago
I've been a professional C programmer for nearly 40 years and didn't do especially well (I got about 2/3rds right). In my view it doesn't really matter. Enable maximum warnings and fix everything the compiler warns about. I also use Coverity once in a while, and every test is run with and without valgrind. If you do that you'll be fine.
- adastra22 4y ago
  Yeah, in practice these sort of gotchas ought to trigger warnings. And for a C/C++ developer, warnings should never be allowed to persist in a code base.
- RedShift1 4y ago
  I thought some deductive reasoning and thinking about how things work at the byte level would save me. Computer said no.
  - nayuki 4y ago
    When you write C programs, you are coding on the C abstract machine as specified by the language standard. If your code triggers signed integer overflow, then the C abstract machine says that it's undefined behavior, and the compiler and runtime can make your code behave however it wishes. It doesn't matter that the underlying concrete machine (POSIX, x86 ISA, etc.) has wraparound signed overflow, because your code has to interact with both the abstract machine and the concrete machine at the same time. See: https://www.youtube.com/watch?v=ZAji7PkXaKY https://www.youtube.com/watch?v=ZAji7PkXaKY
    - petergeoghegan 4y ago
      > When you write C programs, you are coding on the C abstract machine as specified by the language standard. Are you really, though? I would argue that it's a matter of perspective and/or semantics. The Linux kernel is built with -fwrapv and with -fno-strict-aliasing, and uses idioms that depend on it directly. We can surmise from that that the kernel must be: 1. Exhibiting undefined behavior (according to a literal interpretation of the standard) OR: 2. Not written in C. Either way, it's quite reasonable to wonder just how much practical applicability your statement really has in any given situation -- since you didn't have any caveats. It's not as if the kernel is some esoteric, obscure case; it's arguably the single most important C codebase in the world. Plus there are plenty of other big C codebases that take the same approach besides Linux. Lots of compiler people seem to take the same hard line on the issue -- "the C abstract machine" and whatnot. It always surprises me, because it seems to presuppose that the only thing that matters is what the ISO standard says. The actual experience of people working on large C codebases doesn't seem to even get acknowledged. Nor does the fact that the committee and the people that work on compilers have significant overlap. I'm not claiming that "low-level C hackers are right and the compiler people are wrong". I'm merely pointing out that there is a vast cultural chasm that just doesn't seem to be acknowledged.
- nayuki 4y ago
  > In my view it doesn't really matter. One of these days, the compiler will do something surprising to one of your expressions involving signed integer overflow, like converting x < x + 1 to true. Or it'll delete a whole loop because it noticed that your code is guaranteed to trigger an out-of-bounds array read, e.g.: https://devblogs.microsoft.com/oldnewthing/?p=633 https://devblogs.microsoft.com/oldnewthing/?p=633 > If you do that you'll be fine. I would not trust code written based on the methodology you described. However, if you also add -fsanitize=undefined,address (UBSan and ASan) and pass those tests, then I would trust your code.
  - rwmj 4y ago
    Yes we do use the sanitization options too.
bluetomcat 4y ago
Almost all of these examples would be sloppy programming in real code. The “int”, “short”, “long”, “long long” and “char” types are generic labels which shouldn’t be used where size and signedness matters. For a guarantee of size and signedness one should use the “[u]intXX_t” types. For sizes and pointer arithmetic - size_t. For subtracting pointers - ptrdiff_t. You simply wouldn’t have most of these issues if you stick to the correct types.
- rwmj 4y ago
  Absolutely. int is a "code smell" for us, except in some well-defined cases (storing file descriptors for example). If someone is using int to iterate a loop, then the code is more usually wrong than right, they should be using size_t.
  - fizzynut 4y ago
    I don't think a for loop using an int is bad or even "more wrong than right". If anything int is much better than using size_t. Using an integer, in the 1000s of for loops I've written, none get even remotely close to the billions - it is optimizing for a 1 in a million case, and if I know something can run into the billions of iterations I'm going to pay more attention to anyway. I've seen 0 occurrences of bugs relating to this kind of overflow. Using a size_t, it is effectively an unsigned integer that risks underflowing which can easily cause bugs like infinite loops if decrementing or other bugs if doing any index arithmetic. I've seen many occurrences of these kind of bugs.
    - rwmj 4y ago
      On Linux/x86-64 int is 31 bits, so you've probably introduced "1000s" of security bugs where the attacker only needs the persistence to add 2 billion items to a network input, local file or similar, to generate a negative pointer of their choosing. Any such code submitted to one of our projects would be rejected or fixed to use the proper type.
      - fizzynut 4y ago

rwmj 4y ago

I've been a professional C programmer for nearly 40 years and didn't do especially well (I got about 2/3rds right). In my view it doesn't really matter. Enable maximum warnings and fix everything the compiler warns about. I also use Coverity once in a while, and every test is run with and without valgrind. If you do that you'll be fine.

adastra22 4y ago
Yeah, in practice these sort of gotchas ought to trigger warnings. And for a C/C++ developer, warnings should never be allowed to persist in a code base.
RedShift1 4y ago
I thought some deductive reasoning and thinking about how things work at the byte level would save me. Computer said no.
- nayuki 4y ago
  When you write C programs, you are coding on the C abstract machine as specified by the language standard. If your code triggers signed integer overflow, then the C abstract machine says that it's undefined behavior, and the compiler and runtime can make your code behave however it wishes. It doesn't matter that the underlying concrete machine (POSIX, x86 ISA, etc.) has wraparound signed overflow, because your code has to interact with both the abstract machine and the concrete machine at the same time. See: https://www.youtube.com/watch?v=ZAji7PkXaKY https://www.youtube.com/watch?v=ZAji7PkXaKY
  - petergeoghegan 4y ago
    > When you write C programs, you are coding on the C abstract machine as specified by the language standard. Are you really, though? I would argue that it's a matter of perspective and/or semantics. The Linux kernel is built with -fwrapv and with -fno-strict-aliasing, and uses idioms that depend on it directly. We can surmise from that that the kernel must be: 1. Exhibiting undefined behavior (according to a literal interpretation of the standard) OR: 2. Not written in C. Either way, it's quite reasonable to wonder just how much practical applicability your statement really has in any given situation -- since you didn't have any caveats. It's not as if the kernel is some esoteric, obscure case; it's arguably the single most important C codebase in the world. Plus there are plenty of other big C codebases that take the same approach besides Linux. Lots of compiler people seem to take the same hard line on the issue -- "the C abstract machine" and whatnot. It always surprises me, because it seems to presuppose that the only thing that matters is what the ISO standard says. The actual experience of people working on large C codebases doesn't seem to even get acknowledged. Nor does the fact that the committee and the people that work on compilers have significant overlap. I'm not claiming that "low-level C hackers are right and the compiler people are wrong". I'm merely pointing out that there is a vast cultural chasm that just doesn't seem to be acknowledged.
nayuki 4y ago
> In my view it doesn't really matter. One of these days, the compiler will do something surprising to one of your expressions involving signed integer overflow, like converting x < x + 1 to true. Or it'll delete a whole loop because it noticed that your code is guaranteed to trigger an out-of-bounds array read, e.g.: https://devblogs.microsoft.com/oldnewthing/?p=633 https://devblogs.microsoft.com/oldnewthing/?p=633 > If you do that you'll be fine. I would not trust code written based on the methodology you described. However, if you also add -fsanitize=undefined,address (UBSan and ASan) and pass those tests, then I would trust your code.
- rwmj 4y ago
  Yes we do use the sanitization options too.

bluetomcat 4y ago

Almost all of these examples would be sloppy programming in real code. The “int”, “short”, “long”, “long long” and “char” types are generic labels which shouldn’t be used where size and signedness matters. For a guarantee of size and signedness one should use the “[u]intXX_t” types. For sizes and pointer arithmetic - size_t. For subtracting pointers - ptrdiff_t. You simply wouldn’t have most of these issues if you stick to the correct types.

rwmj 4y ago
Absolutely. int is a "code smell" for us, except in some well-defined cases (storing file descriptors for example). If someone is using int to iterate a loop, then the code is more usually wrong than right, they should be using size_t.
- fizzynut 4y ago
  I don't think a for loop using an int is bad or even "more wrong than right". If anything int is much better than using size_t. Using an integer, in the 1000s of for loops I've written, none get even remotely close to the billions - it is optimizing for a 1 in a million case, and if I know something can run into the billions of iterations I'm going to pay more attention to anyway. I've seen 0 occurrences of bugs relating to this kind of overflow. Using a size_t, it is effectively an unsigned integer that risks underflowing which can easily cause bugs like infinite loops if decrementing or other bugs if doing any index arithmetic. I've seen many occurrences of these kind of bugs.
  - rwmj 4y ago
    On Linux/x86-64 int is 31 bits, so you've probably introduced "1000s" of security bugs where the attacker only needs the persistence to add 2 billion items to a network input, local file or similar, to generate a negative pointer of their choosing. Any such code submitted to one of our projects would be rejected or fixed to use the proper type.
    - fizzynut 4y ago