6 ms·

Whenever a story about UB comes up on hacker news, the comments usually fill with both the predictable "Don't use C" comments and also a lot of ill-informed ran

Whenever a story about UB comes up on hacker news, the comments usually fill with both the predictable "Don't use C" comments and also a lot of ill-informed ranting about how stupid all this UB is. For the first group of commenters, please don't bother posting again, just search HN and you can find a lifetime's supply of C love/hate arguments. For the second group of people, I'd recommend reading two fantastic guides to UB: http://blog.regehr.org/archives/213 http://blog.regehr.org/archives/213 http://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html http://blog.llvm.org/2011/05/what-every-c-programmer-should-... These should be enough to explain why UB is trickier than you think...

lmm 11y ago
Plenty of topics are discussed repeatedly here. I won't stop telling people to quit C any more than I'd stop telling them to quit smoking.
- kleiba 11y ago
  ...or telling them to stop posting on HN.
- derefr 11y ago
  But why type it yourself? Someone's probably argued it better than you ever can, and you can just link to that. It's not like you'll come up with any novel arguments at this point, or even novel presentations of existing arguments. It's all been said, and said well.
- pjmlp 11y ago
  I have been doing it since 1992, surely won't stop now. :)
- dmm 11y ago
  I'm writing some tools to capture video streams, doing demuxing/muxing, decoding, etc. All of the libraries that do these things are written in C. Using another language, aside from c++, would involve using some kind of ffi mapping, made by me or someone else. So now I not only have to understand the semantics of c and the libraries I'm using I have to understand the semantics of the ffi mapping. Using the ffmpeg libraries I've had to dig into the source to figure out what it's doing[0]. If I'm using python(or whatever) now I have to do that and figure out what the mapping is doing. And I'm still not safe from segfaults and memory leaks, they're just harder to find and understand. Soft realtime is trivial in C. If I'm capturing a video stream at 10fps I have about 100ms to do everything to handle a frame. This is very easy to do in c. If I'm using a garbage collected language I have 100ms-X, where X is the longest garbage collection pause. I tried doing a similar project in java and found it very difficult. Using any library, for example to output a jpeg, created tons of garbage, resulting in long gc pauses. I tried a bunch of different -XXasdf options that helped but to really solving the problem would involve reimplementing a bunch of stuff to reuse buffers and avoid garbage. I freely admit this may be because of my ignorance of how to do it correctly but again it's trivial to avoid these pauses in c. So I ask sincerely what should I do? I'll gladly throw out the years of painfully acquired c knowledge if you have a better solution for me today. [0] I wonder why avformat_find_stream_info is detecting the from framerate? Good luck :) http://ffmpeg.org/doxygen/trunk/libavformat_2utils_8c_source.html#l03123 http://ffmpeg.org/doxygen/trunk/libavformat_2utils_8c_source...
  - lmm 11y ago
    It's not a great situation yeah - I'd heard there was some effort on Ocaml multimedia libraries but I don't know what's happened to that. Mozilla appear to have got some of their video stack implemented in rust but nowhere near the range of what ffmpeg supports. Hopefully in a few years we'll get there, but in the meantime there's no way around running some C code. Even so, I would (and have) still use an FFI and put as much as possible in a safer language. You will get segfaults and memory leaks because you're still calling C code, but you won't be adding more, and IME that's well worth the overhead of an FFI. (Also ffmpeg has had quite a bit of attention from people using valgrind, afl-fuzz etc, so its defect rate is lower than your C code would likely have, unless you're also using those tools). Alternatively, depending on your use case you might be able to separate the decoding/encoding stages from the processing, in which case you can write the code without needing much in the way of libraries, and pipe to/from ffmpeg. Write a quick prototype and see if you can get the performance you need within GC or a small amount of -XXasdf (you might be surprised, particularly with the state of Haskell and OCaml today. I've done literally this from Java, JNI-ing the ffmpeg libraries to do realtime streaming, and had it work fine, and that was five years ago). In the rare case something like that doesn't cut it Rust is what I'd use (there are other options but I know less about them).
tomp 11y ago
The second guide explains some cases for undefined behavior, but it mixes up some things (or explains them poorly)... For example, I understand why signed integer overflow and violating type rules are UB; the first allows the compiler to optimize many inequalities, while the second enables various optimizations that rely on alias analysis. But I can't understand why e.g. use of an uninitialized variable or oversized shift amounts are UB. Unless these also enable some powerful optimizations (the article doesn't mention any, and I can't think of any either), it would be much better to leave them implementation defined, not undefined. In that case, reading an uninitialized variable could result in any value, but it would still make the whole program valid. Edit: should have said "platform-defined" instead of "implementation-defined".
- bluetomcat 11y ago
  On some architectures, an uninitialized value may contain either a trap representation or a valid value, so enforcing any consistent implementation defined behavior seems impossible.
  - tomp 11y ago
    Really? I didn't know that. Which architectures are that (that contain a trap)? Still, crashing would be a valid "implementation-defined" behaviour in my book.
    - bluetomcat 11y ago
      x86 has signalling floating-point NaNs, for example. This means that any attempt to access an uninitialized floating-point value could potentially crash the program (or at least affect the control flow).
      - tomp 11y ago
        Hm... according to the example at http://en.cppreference.com/w/cpp/types/numeric_limits/signaling_NaN http://en.cppreference.com/w/cpp/types/numeric_limits/signal... sNaN doesn't actually trap - it just records a special bit in a FPU exception register. So yes, it can affect the control flow - which is something you should expect reading aribtrary values!
oselhn 11y ago
Problem is that most of the C/C++ programmers do not known about most UB cases. Using C/C++ only if is really needed helps a lot.
- pjmlp 11y ago
  The worse ones are the ones that think "my compiler == C language".
legulere 11y ago
Instead we should maybe argue how undefined variable and undefined behaviour handling in LLVM is making things worse than they actually could be: Undef variables taking on different values at different use points can lead to e.g. a == a being false. Code paths being possible with undefined values, that wouldn't be possible with any value sounds like a terrible idea to me, possibly exploitable, at least a source for heisenbugs When llvm finds undefined behaviour in a code path (not possible undefined behaviour, but there actually is undefined behaviour), it does strange things instead of just trapping. When the execution reaches that code path, the program already is in an erroneous state, no need to optimize stuff there. For instance: int main(int argc, char argv) { int a = 1; int b = 0; printf("%d", a/b); } compiles to: define i32 @main(i32 %argc, i8 nocapture readnone %argv) #0 { %1 = tail call i32 (i8, ...) @printf(i8* getelementptr inbounds ([3 x i8]* @.str, i64 0, i64 0), i32 undef) #2 ret i32 0 } Why let the program behave buggy when llvm can see that undefined behaviour will occur? Why not just trap instead? That would have zero cost.
- lomnakkus 11y ago
  > Why let the program behave buggy when llvm can see that undefined behaviour will occur? It's not quite that easy. Because compilers these days have a large number of phases/optimization passes it can be prohibitively difficult to trace back from the location of a potential UB to what might/does trigger it (in the source). So what compilers do is ASSUME that no UB can occur, hence there's no need to generate a check for "b=0". You can run clang with -fsanitize=undefined, though, and get runtime-checks for many UBs. It'll cost you runtime performance, though.
vardump 11y ago
Those who complain about UB should first master asm programming with microcontroller and some modern architecture. Maybe then they'd understand the cost better. Compilers should generate warnings on all compile time UB and maybe even do data flow analysis for possible runtime UB. Although, IMHO, code with a reference to block scope uninitialized variable shouldn't even compile.
- Someone 11y ago
  It can be hard or even impossible to prove that a variable gets read before it gets written to. People who want to get every last bit of performance out of their systems do want to write code like: int x; ... if(condition1) x = expression1; ... while(expression2) { x = DoSomethingWith(x); } where the intelligent reader knows things are fine, but a compiler has a hard time proving it. For example, a contract implicit in an API may enforce the invariant, but that cannot be deduced from the code.
- cyphar 11y ago
  > Although, IMHO, code with a reference to block scope uninitialized variable shouldn't even compile. Funny story, that is actually what caused Debian's weak keys fiasco. The code was using uninitialised memory to seed the PRNG and someone removed it because of valgrind warnings. So sometimes UB is exactly what you want. :P

lmm 11y ago

Plenty of topics are discussed repeatedly here. I won't stop telling people to quit C any more than I'd stop telling them to quit smoking.

kleiba 11y ago
...or telling them to stop posting on HN.
derefr 11y ago
But why type it yourself? Someone's probably argued it better than you ever can, and you can just link to that. It's not like you'll come up with any novel arguments at this point, or even novel presentations of existing arguments. It's all been said, and said well.
pjmlp 11y ago
I have been doing it since 1992, surely won't stop now. :)
dmm 11y ago
I'm writing some tools to capture video streams, doing demuxing/muxing, decoding, etc. All of the libraries that do these things are written in C. Using another language, aside from c++, would involve using some kind of ffi mapping, made by me or someone else. So now I not only have to understand the semantics of c and the libraries I'm using I have to understand the semantics of the ffi mapping. Using the ffmpeg libraries I've had to dig into the source to figure out what it's doing[0]. If I'm using python(or whatever) now I have to do that and figure out what the mapping is doing. And I'm still not safe from segfaults and memory leaks, they're just harder to find and understand. Soft realtime is trivial in C. If I'm capturing a video stream at 10fps I have about 100ms to do everything to handle a frame. This is very easy to do in c. If I'm using a garbage collected language I have 100ms-X, where X is the longest garbage collection pause. I tried doing a similar project in java and found it very difficult. Using any library, for example to output a jpeg, created tons of garbage, resulting in long gc pauses. I tried a bunch of different -XXasdf options that helped but to really solving the problem would involve reimplementing a bunch of stuff to reuse buffers and avoid garbage. I freely admit this may be because of my ignorance of how to do it correctly but again it's trivial to avoid these pauses in c. So I ask sincerely what should I do? I'll gladly throw out the years of painfully acquired c knowledge if you have a better solution for me today. [0] I wonder why avformat_find_stream_info is detecting the from framerate? Good luck :) http://ffmpeg.org/doxygen/trunk/libavformat_2utils_8c_source.html#l03123 http://ffmpeg.org/doxygen/trunk/libavformat_2utils_8c_source...
- lmm 11y ago
  It's not a great situation yeah - I'd heard there was some effort on Ocaml multimedia libraries but I don't know what's happened to that. Mozilla appear to have got some of their video stack implemented in rust but nowhere near the range of what ffmpeg supports. Hopefully in a few years we'll get there, but in the meantime there's no way around running some C code. Even so, I would (and have) still use an FFI and put as much as possible in a safer language. You will get segfaults and memory leaks because you're still calling C code, but you won't be adding more, and IME that's well worth the overhead of an FFI. (Also ffmpeg has had quite a bit of attention from people using valgrind, afl-fuzz etc, so its defect rate is lower than your C code would likely have, unless you're also using those tools). Alternatively, depending on your use case you might be able to separate the decoding/encoding stages from the processing, in which case you can write the code without needing much in the way of libraries, and pipe to/from ffmpeg. Write a quick prototype and see if you can get the performance you need within GC or a small amount of -XXasdf (you might be surprised, particularly with the state of Haskell and OCaml today. I've done literally this from Java, JNI-ing the ffmpeg libraries to do realtime streaming, and had it work fine, and that was five years ago). In the rare case something like that doesn't cut it Rust is what I'd use (there are other options but I know less about them).

tomp 11y ago

The second guide explains some cases for undefined behavior, but it mixes up some things (or explains them poorly)... For example, I understand why signed integer overflow and violating type rules are UB; the first allows the compiler to optimize many inequalities, while the second enables various optimizations that rely on alias analysis. But I can't understand why e.g. use of an uninitialized variable or oversized shift amounts are UB. Unless these also enable some powerful optimizations (the article doesn't mention any, and I can't think of any either), it would be much better to leave them implementation defined, not undefined. In that case, reading an uninitialized variable could result in any value, but it would still make the whole program valid. Edit: should have said "platform-defined" instead of "implementation-defined".

bluetomcat 11y ago
On some architectures, an uninitialized value may contain either a trap representation or a valid value, so enforcing any consistent implementation defined behavior seems impossible.
- tomp 11y ago
  Really? I didn't know that. Which architectures are that (that contain a trap)? Still, crashing would be a valid "implementation-defined" behaviour in my book.
  - bluetomcat 11y ago
    x86 has signalling floating-point NaNs, for example. This means that any attempt to access an uninitialized floating-point value could potentially crash the program (or at least affect the control flow).
    - tomp 11y ago
      Hm... according to the example at http://en.cppreference.com/w/cpp/types/numeric_limits/signaling_NaN http://en.cppreference.com/w/cpp/types/numeric_limits/signal... sNaN doesn't actually trap - it just records a special bit in a FPU exception register. So yes, it can affect the control flow - which is something you should expect reading aribtrary values!

oselhn 11y ago

Problem is that most of the C/C++ programmers do not known about most UB cases. Using C/C++ only if is really needed helps a lot.

pjmlp 11y ago
The worse ones are the ones that think "my compiler == C language".

legulere 11y ago

Instead we should maybe argue how undefined variable and undefined behaviour handling in LLVM is making things worse than they actually could be: Undef variables taking on different values at different use points can lead to e.g. a == a being false. Code paths being possible with undefined values, that wouldn't be possible with any value sounds like a terrible idea to me, possibly exploitable, at least a source for heisenbugs When llvm finds undefined behaviour in a code path (not possible undefined behaviour, but there actually is undefined behaviour), it does strange things instead of just trapping. When the execution reaches that code path, the program already is in an erroneous state, no need to optimize stuff there. For instance: int main(int argc, char argv) { int a = 1; int b = 0; printf("%d", a/b); } compiles to: define i32 @main(i32 %argc, i8 nocapture readnone %argv) #0 { %1 = tail call i32 (i8, ...) @printf(i8* getelementptr inbounds ([3 x i8]* @.str, i64 0, i64 0), i32 undef) #2 ret i32 0 } Why let the program behave buggy when llvm can see that undefined behaviour will occur? Why not just trap instead? That would have zero cost.

lomnakkus 11y ago
> Why let the program behave buggy when llvm can see that undefined behaviour will occur? It's not quite that easy. Because compilers these days have a large number of phases/optimization passes it can be prohibitively difficult to trace back from the location of a potential UB to what might/does trigger it (in the source). So what compilers do is ASSUME that no UB can occur, hence there's no need to generate a check for "b=0". You can run clang with -fsanitize=undefined, though, and get runtime-checks for many UBs. It'll cost you runtime performance, though.

vardump 11y ago

Those who complain about UB should first master asm programming with microcontroller and some modern architecture. Maybe then they'd understand the cost better. Compilers should generate warnings on all compile time UB and maybe even do data flow analysis for possible runtime UB. Although, IMHO, code with a reference to block scope uninitialized variable shouldn't even compile.

Someone 11y ago
It can be hard or even impossible to prove that a variable gets read before it gets written to. People who want to get every last bit of performance out of their systems do want to write code like: int x; ... if(condition1) x = expression1; ... while(expression2) { x = DoSomethingWith(x); } where the intelligent reader knows things are fine, but a compiler has a hard time proving it. For example, a contract implicit in an API may enforce the invariant, but that cannot be deduced from the code.
cyphar 11y ago
> Although, IMHO, code with a reference to block scope uninitialized variable shouldn't even compile. Funny story, that is actually what caused Debian's weak keys fiasco. The code was using uninitialised memory to seed the PRNG and someone removed it because of valgrind warnings. So sometimes UB is exactly what you want. :P