15 ms·

It remains unclear what behaviour compilers currently provide (or should provide) for this. It might be nice if future surveys explicitly asked a followup ques

by nkurz 11y ago

It remains unclear what behaviour compilers currently provide (or should provide) for this. It might be nice if future surveys explicitly asked a followup question "Regardless of the standard or behavior of existing compilers, is there one of these answers that is the 'obviously correct' manner in which compilers should behave? Which one?" If practically all users believe that the same answer is 'obviously correct', compiler writes might want to take this into account when deciding which behavior to implement. For MSVC, one respondent said: "I am aware of a significant divergence between the LLVM community and MSVC here; in general LLVM uses "undefined behaviour" to mean "we can miscompile the program and get better benchmarks", whereas MSVC regards "undefined behaviour" as "we might have a security vulnerability so this is a compile error / build break". First, there is reading an uninitialized variable (i.e. something which does not necessarily have a memory location); that should always be a compile error. Period. Second, there is reading a partially initialised struct (i.e. reading some memory whose contents are only partly defined). That should give a compile error/warning or static analysis warning if detectable. If not detectable it should give the actual contents of the memory (be stable). I am strongly with the MSVC folks on this one - if the compiler can tell at compile time that anything is undefined then it should error out. Security problems are a real problem for the whole industry and should not be included deliberately by compilers." I'm much less familiar with MSVC than the alternatives, but this is a refreshing approach. Yes, give me a mode that refuses to silently rewrite undefined behavior. Is MSVC possibly able to take this approach because it isn't trying to be compliant to modern C standards? Does it actually reduce the ability to apply useful optimizations? Or just a difference in philosophy?

dllthomas 11y ago
While I am quite sympathetic to the "break the build on undefined behavior" position, I have a nit to pick in the quoted passage. Initialization of a variable has no relation to whether it has a memory location. You can legitimately take the address of an uninitialized variable (that is often one step in initializing them, such as when you pass the address to memset), and even an initialized variable may not have a memory location (if it lives only in register, or is compiled away entirely).
- angersock 11y ago
  To pick your nit, note that they specifically said reading an uninitialized or partially initialized variable, not reading the address of the same.
  - dllthomas 11y ago
    I didn't mean to imply (and don't think I did...) that reading an uninitialized variable is not a problem, just that the problem is entirely unrelated to whether it has a memory location.
    - eatonphil 11y ago
      Hate to ask the stupid question, but I've been wondering and it seems to be along the same lines... Why can't this be valid? int f; f = 2; To me this says, there is an int pointer f. Let f point to 2. Is this not possible only because 2 does not occupy memory? I don't see why this couldn't be valid.
      - dllthomas 11y ago
        Assuming you're asking about int *f; *f = 2; The statement, *f = 2; says "dereference f to get a location; set the contents of that location to 2". "Set f to the location of 2" would be spelled f = &2; This is partly a matter of the difference between assignment and definition(/equality). The first of these is invalid C (undefined behavior) because it uses a value (the contents of f) before initialization. The second is invalid C (compile error) because 2 is not an l-value (that is, it does not have a location). Edited to add: I will note that I don't think there is any reason *f = 2; couldn't mean to give f the addess of 2, as by analogy to structural pattern matching, but that the syntax is already taken for something else.
- jacquesm 11y ago
  And you might want to access a memory location legitimately that was not written by this program, for instance a chunk of dual-port memory, a shared memory block, a piece of memory mapped hardware and so on. In those cases some compiler override could provide the solution, while still allowing the compiler to flag all the other cases as errors.
userbinator 11y ago
MSVC will very confidently warn on cases like this by default: int foo() { int bar; return bar + 5; /* C4700: local variable 'bar' used without having been initialized */ } https://msdn.microsoft.com/en-us/library/axhfhh6x.aspx https://msdn.microsoft.com/en-us/library/axhfhh6x.aspx In the same situation, GCC says "may be used uninitialized in this function" if you enable the warning (-Wmaybe-uninitialized), despite this being a trivial and certain case.
- ydcvjk 11y ago
  gcc 4.9.2 gives: warning: 'bar' is used uninitialized in this function Maybe you are using a old version. If you enable -Wall which you always should then this flag is automatically enabled. https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wall-296 https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#inde...
  - anon1385 11y ago
    -Wuninitialized has been broken in GCC for over a decade: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=18501 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=18501
    - anon4 11y ago
      As if we needed more proof that GCC is a fucking joke...
      - deleted 11y ago
        [deleted]
      - ydcvjk 11y ago
        Hey Linus, how are you doing!
- bluecalm 11y ago
  While I like MSVC team approach to this I don't think it's necessarily worse to just warn as long as that warning consistently work. I mean, just add some warning options to your compiling script and you are good to go. I am getting this error from GCC very consistently and as good as compilation error in practice.
Kenji 11y ago
First, there is reading an uninitialized variable (i.e. something which does not necessarily have a memory location); that should always be a compile error. Period. You're out of luck. You have to solve the halting problem to statically analyze whether or not a variable will be used when it's undefined. The reason why this is not solved well is because it's impossible to solve perfectly! Java made another approach that I hate: For example if you have Object a; for(int i = 0; i < 1;++i) a = new Object(); a.toString(); then Java will give you error: variable a might not have been initialized even though it's plain and obvious that a is initialized. Things like that make me mad, it's a non-solution.
- nkurz 11y ago
  I'm not asking for the compiler to consistently identify undefined behavior, only to have a mode where it refuses to silently make 'optimizations' when it does identify UB. The parallel to your example would be if the for loop was "for(int i = 0; i < j;++i)". If the compiler was able to determine that there is a code path whereby j might be undefined, should it be allowed to remove the body of the loop, even in those cases where the programmer knows by other means that "j >= 1"? My request is that it either keep the loop body, or complain about the undefined behavior, but not silently make 'optimizations' based on the fact that it has identified the potential for undefined behavior to occur. Note that I'm just using an 'uninitialized variable' as a hypothetical example. Given a chance, I always compile with -Wall -Wextra, and in practice, GCC, CLang, and ICC (the compilers I use) do a good job of issuing warnings for the use of uninitialized variables. I like this current behavior, but would prefer a philosophical approach that makes warnings like this more rather than less common.
  - kazinator 11y ago
    > If the compiler was able to determine that there is a code path whereby j might be undefined, should it be allowed to remove the body of the loop, even in those cases where the programmer knows by other means that "j >= 1"? No; rather, the correct logic is that compiler must preserve the body of the loop if there exists the possibility that it can be reached by a valid code path without any undefined behavior (j is defined, and so forth). Only if the compiler can prove that no well-defined execution path can reach the body can it remove it. (A bad idea to do without any warning, though. If undefined behavior is confirmed, it should be diagnosed.)
bluecalm 11y ago
Main problem with breaking on undefined behavior is detecting it in the first place. It's not a choice not to warn/break it's an implementation challenge to find those.
- dllthomas 11y ago
  I think one could reliably break (more safely) on undefined behavior by inserting defensive run-time checks. Whether that's worth the performance hit is liable to be domain specific.
kazinator 11y ago
> If practically all users believe that the same answer is 'obviously correct', compiler writes might want to take this into account The problem with this democratic approach is that most users are not qualified such that their opinion is particularly valuable. The small minority who doesn't find it "obviously correct" may actually in some cases be the minority with a clue. It could work if only those users are given a vote who pass a language lawyer exam.
- Guvante 11y ago
  > The problem with this democratic approach is that most users are not qualified such that their opinion is particularly valuable. They aren't advocating a democratic approach, they are saying that the general expectations of users is valuable information. You don't rely exclusively on what the users want, but users expectations for how compilers behave is very valuable behavior, whether it means finding out how to inform users of the real behavior or changing the behavior.
Someone 11y ago
"First, there is reading an uninitialized variable (i.e. something which does not necessarily have a memory location); that should always be a compile error. Period." You cannot implement that efficiently without rejecting valid programs. Consider code like this: int x; if( f()) x = 1; if( g()) h(x); For sufficiently complex functions f and g, there's no reasonable way to decide at compile time whether x will be set whenever g() returns true. For example, f() might always return false because Fermat's last theorem is true. And that 'reasonable' likely isn't a necessary part of that statement.
- userbinator 11y ago
  I think the idea is to give an error when there is absolutely no way that variable could have been initialised when it's read, i.e. cases like this: int x; h(x); The point is not to solve the Halting Problem, but to catch code that is so obviously wrong that no programmer would deliberately write it (unless they were testing the compiler's reaction.)
  - Someone 11y ago
    If so, "that should always be a compile error. Period.” is IMO a poor way to express that idea. On top of that, the C compilers I know more or less have that, as they give warnings for basically cases where Java (with its stricter rules) would refuse to compile equivalent code, and have a flag that turns warnings into errors.
- JupiterMoon 11y ago
  Why does this have to be a valid program? There is obviously a chance that x is undefined at line 3 so why not allow the compiler to throw an error?
  - Ded7xSEoPKYNsDd 11y ago
    Not necessarily. Just because the compiler can't prove that g only returns true iff f also returned true doesn't mean the programmer doesn't know it to actually be true. So changing that behavior means that the compiler now rejects 40 years worth of correctly working legacy code (and some buggy code, as well). Newer languages (e.g. Java, C#) that don't have to support existing code can afford to do what you want and reject programs where a simple heuristic isn't enough to tell whether a variable is initialized or not.
qznc 11y ago
My guess would be that LLVM is not so much motivated by better benchmarks. You could always make it a parameter. The bigger issue usually is: If we mark something as an error, which is actually allowed by the C standard, then lots big old libraries will not compile anymore and nobody is willing to fix all the old code.
- pmelendez 11y ago
  > then lots big old libraries will not compile anymore and nobody is willing to fix all the old code. This is begging for one question in my opinion. Should we keep using old libraries that nobody is maintaining anymore? Isn't that a big security issue?
- Sir_Cmpwn 11y ago
  Can it be put behind a flag? If some library fails to compile, toss the flag into the toolchain invocation and try again?
- anon4 11y ago
  If we miscompile (or outright delete) something which is marked "undefined behaviour" by the C standard, then lots of old libraries will build without errors, but fail in various fun ways during use, including having severe security issues. I think failing the build is better than giving me a library with deleted NULL checks and whatnot.

by nkurz 11y ago

dllthomas 11y ago

While I am quite sympathetic to the "break the build on undefined behavior" position, I have a nit to pick in the quoted passage. Initialization of a variable has no relation to whether it has a memory location. You can legitimately take the address of an uninitialized variable (that is often one step in initializing them, such as when you pass the address to memset), and even an initialized variable may not have a memory location (if it lives only in register, or is compiled away entirely).

angersock 11y ago
To pick your nit, note that they specifically said reading an uninitialized or partially initialized variable, not reading the address of the same.
- dllthomas 11y ago
  I didn't mean to imply (and don't think I did...) that reading an uninitialized variable is not a problem, just that the problem is entirely unrelated to whether it has a memory location.
  - eatonphil 11y ago
    Hate to ask the stupid question, but I've been wondering and it seems to be along the same lines... Why can't this be valid? int f; f = 2; To me this says, there is an int pointer f. Let f point to 2. Is this not possible only because 2 does not occupy memory? I don't see why this couldn't be valid.
    - dllthomas 11y ago
      Assuming you're asking about int *f; *f = 2; The statement, *f = 2; says "dereference f to get a location; set the contents of that location to 2". "Set f to the location of 2" would be spelled f = &2; This is partly a matter of the difference between assignment and definition(/equality). The first of these is invalid C (undefined behavior) because it uses a value (the contents of f) before initialization. The second is invalid C (compile error) because 2 is not an l-value (that is, it does not have a location). Edited to add: I will note that I don't think there is any reason *f = 2; couldn't mean to give f the addess of 2, as by analogy to structural pattern matching, but that the syntax is already taken for something else.
jacquesm 11y ago
And you might want to access a memory location legitimately that was not written by this program, for instance a chunk of dual-port memory, a shared memory block, a piece of memory mapped hardware and so on. In those cases some compiler override could provide the solution, while still allowing the compiler to flag all the other cases as errors.

userbinator 11y ago

MSVC will very confidently warn on cases like this by default: int foo() { int bar; return bar + 5; /* C4700: local variable 'bar' used without having been initialized */ } https://msdn.microsoft.com/en-us/library/axhfhh6x.aspx https://msdn.microsoft.com/en-us/library/axhfhh6x.aspx In the same situation, GCC says "may be used uninitialized in this function" if you enable the warning (-Wmaybe-uninitialized), despite this being a trivial and certain case.

ydcvjk 11y ago
gcc 4.9.2 gives: warning: 'bar' is used uninitialized in this function Maybe you are using a old version. If you enable -Wall which you always should then this flag is automatically enabled. https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wall-296 https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#inde...
- anon1385 11y ago
  -Wuninitialized has been broken in GCC for over a decade: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=18501 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=18501
  - anon4 11y ago
    As if we needed more proof that GCC is a fucking joke...
    - deleted 11y ago
      [deleted]
    - ydcvjk 11y ago
      Hey Linus, how are you doing!
bluecalm 11y ago
While I like MSVC team approach to this I don't think it's necessarily worse to just warn as long as that warning consistently work. I mean, just add some warning options to your compiling script and you are good to go. I am getting this error from GCC very consistently and as good as compilation error in practice.

Kenji 11y ago

First, there is reading an uninitialized variable (i.e. something which does not necessarily have a memory location); that should always be a compile error. Period. You're out of luck. You have to solve the halting problem to statically analyze whether or not a variable will be used when it's undefined. The reason why this is not solved well is because it's impossible to solve perfectly! Java made another approach that I hate: For example if you have Object a; for(int i = 0; i < 1;++i) a = new Object(); a.toString(); then Java will give you error: variable a might not have been initialized even though it's plain and obvious that a is initialized. Things like that make me mad, it's a non-solution.

nkurz 11y ago
I'm not asking for the compiler to consistently identify undefined behavior, only to have a mode where it refuses to silently make 'optimizations' when it does identify UB. The parallel to your example would be if the for loop was "for(int i = 0; i < j;++i)". If the compiler was able to determine that there is a code path whereby j might be undefined, should it be allowed to remove the body of the loop, even in those cases where the programmer knows by other means that "j >= 1"? My request is that it either keep the loop body, or complain about the undefined behavior, but not silently make 'optimizations' based on the fact that it has identified the potential for undefined behavior to occur. Note that I'm just using an 'uninitialized variable' as a hypothetical example. Given a chance, I always compile with -Wall -Wextra, and in practice, GCC, CLang, and ICC (the compilers I use) do a good job of issuing warnings for the use of uninitialized variables. I like this current behavior, but would prefer a philosophical approach that makes warnings like this more rather than less common.
- kazinator 11y ago
  > If the compiler was able to determine that there is a code path whereby j might be undefined, should it be allowed to remove the body of the loop, even in those cases where the programmer knows by other means that "j >= 1"? No; rather, the correct logic is that compiler must preserve the body of the loop if there exists the possibility that it can be reached by a valid code path without any undefined behavior (j is defined, and so forth). Only if the compiler can prove that no well-defined execution path can reach the body can it remove it. (A bad idea to do without any warning, though. If undefined behavior is confirmed, it should be diagnosed.)

bluecalm 11y ago

Main problem with breaking on undefined behavior is detecting it in the first place. It's not a choice not to warn/break it's an implementation challenge to find those.

dllthomas 11y ago
I think one could reliably break (more safely) on undefined behavior by inserting defensive run-time checks. Whether that's worth the performance hit is liable to be domain specific.

kazinator 11y ago

> If practically all users believe that the same answer is 'obviously correct', compiler writes might want to take this into account The problem with this democratic approach is that most users are not qualified such that their opinion is particularly valuable. The small minority who doesn't find it "obviously correct" may actually in some cases be the minority with a clue. It could work if only those users are given a vote who pass a language lawyer exam.

Guvante 11y ago
> The problem with this democratic approach is that most users are not qualified such that their opinion is particularly valuable. They aren't advocating a democratic approach, they are saying that the general expectations of users is valuable information. You don't rely exclusively on what the users want, but users expectations for how compilers behave is very valuable behavior, whether it means finding out how to inform users of the real behavior or changing the behavior.

Someone 11y ago

"First, there is reading an uninitialized variable (i.e. something which does not necessarily have a memory location); that should always be a compile error. Period." You cannot implement that efficiently without rejecting valid programs. Consider code like this: int x; if( f()) x = 1; if( g()) h(x); For sufficiently complex functions f and g, there's no reasonable way to decide at compile time whether x will be set whenever g() returns true. For example, f() might always return false because Fermat's last theorem is true. And that 'reasonable' likely isn't a necessary part of that statement.

userbinator 11y ago
I think the idea is to give an error when there is absolutely no way that variable could have been initialised when it's read, i.e. cases like this: int x; h(x); The point is not to solve the Halting Problem, but to catch code that is so obviously wrong that no programmer would deliberately write it (unless they were testing the compiler's reaction.)
- Someone 11y ago
  If so, "that should always be a compile error. Period.” is IMO a poor way to express that idea. On top of that, the C compilers I know more or less have that, as they give warnings for basically cases where Java (with its stricter rules) would refuse to compile equivalent code, and have a flag that turns warnings into errors.
JupiterMoon 11y ago
Why does this have to be a valid program? There is obviously a chance that x is undefined at line 3 so why not allow the compiler to throw an error?
- Ded7xSEoPKYNsDd 11y ago
  Not necessarily. Just because the compiler can't prove that g only returns true iff f also returned true doesn't mean the programmer doesn't know it to actually be true. So changing that behavior means that the compiler now rejects 40 years worth of correctly working legacy code (and some buggy code, as well). Newer languages (e.g. Java, C#) that don't have to support existing code can afford to do what you want and reject programs where a simple heuristic isn't enough to tell whether a variable is initialized or not.

qznc 11y ago

My guess would be that LLVM is not so much motivated by better benchmarks. You could always make it a parameter. The bigger issue usually is: If we mark something as an error, which is actually allowed by the C standard, then lots big old libraries will not compile anymore and nobody is willing to fix all the old code.

pmelendez 11y ago
> then lots big old libraries will not compile anymore and nobody is willing to fix all the old code. This is begging for one question in my opinion. Should we keep using old libraries that nobody is maintaining anymore? Isn't that a big security issue?
Sir_Cmpwn 11y ago
Can it be put behind a flag? If some library fails to compile, toss the flag into the toolchain invocation and try again?
anon4 11y ago
If we miscompile (or outright delete) something which is marked "undefined behaviour" by the C standard, then lots of old libraries will build without errors, but fail in various fun ways during use, including having severe security issues. I think failing the build is better than giving me a library with deleted NULL checks and whatnot.