7 ms·

See https://news.ycombinator.com/item?id=30024534 https://news.ycombinator.com/item?id=30024534 (by jcranmer) and https://news.ycombinator.com/item?id=30026095

See https://news.ycombinator.com/item?id=30024534 https://news.ycombinator.com/item?id=30024534 (by jcranmer) and https://news.ycombinator.com/item?id=30026095 https://news.ycombinator.com/item?id=30026095 (by me). Does it help?

nkurz 5y ago
Unfortunately, these don't help much. I thought about responding to jcranmer's post instead of yours, but wasn't sure how to approach it. It's a good comment, and I appreciated his attempt, but I feel like he's thoroughly missing the point of the complaints about UB. The complaint isn't that too much UB exists in C, but that compiler writers seem to use the presence of UB as an excuse for being allowed to break code that has worked historically. And the complaint isn't just that the code is broken, but that no apparent care is taken to avoid doing so despite the small amount of gain. It's a worldview mismatch, and I don't know how to bridge it. Your comment seems about the same as the one I responded to. You seem to assume that people who complain about UB in C would have a problem with keeping local variables in registers, but I've never seen anyone actually make this complaint. Take for example the Arxiv paper we are discussing: he doesn't bring this up. This makes me suspect that your mental model of the people who are complaining about UB in C is probably flawed. I understand the technical issue you are gesturing toward, I just don't see it as being something that refutes any of the issues brought up in the Arxiv paper. My hope was that a concrete example might help to clarify, but I do realize this might not be the right forum for that.
- sanxiyn 5y ago
  No, I don't assume people are against register allocation, but any concrete proposal I have seen kind of implies such conclusion. I am trying to understand what people actually want, since they seem clearly different from what people say they want. Okay let's discuss a concrete example. *x = 12345678; f(); return *x; // can you copy propagate 12345678 to here? f() does this: for (int *p = 0; p++; p < MEMSIZE) if (*p == 12345678) *p = 12345679. That is, f scans the memory for 12345678 and replace all instances with 12345679. There is no doubt this actually works that way in assembly. Things like cheat engines do this! C compilers assume this doesn't happen, because it is UB. Hence, portable assembly C compiler can't omit any load. Now I understand there are minority of people who will answer "that's what I want!", but like register allocation, I think people generally want this to optimize. But that necessarily implies memory search-and-replace can't compile in portable assembly manner.
  - mpweiher 5y ago
    > any concrete proposal I have seen kind of implies such conclusion. No it does not. In your example, I personally would prefer it did not propagate the 12345678. Good grief, I wrote the deref there. > C compilers assume this doesn't happen, because it is UB. Incorrectly. IMHO. > but like register allocation, You are silently equating int x; with a memory deref. There is no need to do this. Anyway, here is part of the rationale for C: "Although it strove to give programmers the opportunity to write truly portable programs, the Committee did not want to force programmers into writing portably, to preclude the use of C as a ``high-level assembler'': the ability to write machine-specific code is one of the strengths of C. It is this principle which largely motivates drawing the distinction between strictly conforming program and conforming program" http://port70.net/~nsz/c/c89/rationale/a.html#1-5 http://port70.net/~nsz/c/c89/rationale/a.html#1-5 So the idea of C as a portable assembler is not some strange idea proposed by ignorant people who don't understand C, but an idea that was fundamental to C and fundamental to the people who created the ANSI/ISO C standard. But hey, what do they know?
  - mjevans 5y ago
    A programmer wrote things in a specific order for a specific reason. Lets instead assume that the variable assignments above are to some global configuration variables and then f() also references those and the behavior of f() changes based on the previously written code. The objections from the 'C as portable assembler' camp are: * Re-ordering the order of operations across context switch bounds (curly braces and function calls). -- re-ordering non-volatile store / loads within a context is fine, and shouldn't generate warnings. * Eliminating written instructions (not calling f()) based on optimizations. -- Modification to computed work should always generate a warning so the optimization can be applied to the source code, or bugs corrected.
    - UncleMeat 5y ago
      > A programmer wrote things in a specific order for a specific reason. Is it not possible that the programmer introduced a bug? Consider the bug that caused the 911 glitch in Android phones recently. An unstable comparator was defined in a type, violating the contract that Comparable has with Java's sorting algorithms. When Java detects that this implementation violates the assumptions its sorting algorithms make, it throws an exception. Should it not do this and instead say that the programmer wrote that specific comparator on purpose and it should loop forever or produce an incorrect sort? I think most people would say "no". So why is the contract around pointer dereferencing meaningfully different?
- comp54321 5y ago
  There's not a bright line between optimization passes being aware of UB in unreasonable vs "exploitative" ways. The principled way to think about optimization is that an optimization pass can assume certain invariants about the code but must preserve other invariants through the transformation as long as the assumptions hold. I think basically any invariant you assume in a memory-unsafe language C could be invalidated by real code. A lot of the things people complain about are cases where the compiler got better at reasoning through valid transformations to the code, not cases where the compiler . E.g. I'm sure that even very old compilers have had optimization passes that remove redundant NULL checks that logically depend on dereferencing NULL pointers being undefined. But these probably were more ad-hoc and could only reason about relatively simple cases and constrained scopes. Another example is integer overflow. I'd bet a lot of older compilers have loop optimizations that somehow depend on the loop counter not overflowing, or pointer addresses not wrapping around. I think it's perfectly reasonable to say that C's definition of undefined behaviour is too expansive and too difficult to avoid in real code. But I think that a lot of the complaints end up being equivalent to "don't depend on UB in ways that I notice in my code" or "don't depend on UB in ways that compilers from the 90s didn't". That's not something you can practically apply or verify when building a compiler because there isn't a bright line between assuming an invariant in reasonable vs unreasonable ways, unless you can define a narrow invariant that distinguishes them.
- jcranmer 5y ago
  My experience with a lot of the UB-is-bad crowd is that they don't have much of an appreciation for semantics in general. That is to say, they tend to react to particular compiler behavior, and they don't have any suggestions for how to rectify the situation in a way that preserves consistent semantics. And when you try to pin down the semantics, you usually end up with a compiler that has to "do what I mean, not what I said." A salient example that people often try on me, that I don't find persuasive, is the null pointer check: void foo(int *x) { int val = *x; if (!x) return; /* do real stuff */ } "The compiler is stupid for eliminating that null check!" "So you want the code to crash because of a null pointer dereference then." "No, no, no! Do the check first, and dereference only when it's not null!" "That's not what you said..."
  - mzs 5y ago
    On a lot of systems load from address zero doesn't segfault, I'm fine with the cpu loading it. I'm disappointed that the new compiler removed the check someone wrote a dozen years ago to prevent the cpu from overwriting something important though.
    - jcranmer 5y ago
      In the case where loads from address zero are legal, the dereference-means-non-null-pointer assumption is disabled.
      - mzs 5y ago
        Seems it wasn't disabled in 2009 https://github.com/torvalds/linux/commit/a3ca86aea507904148870946d599e07a340b39bf https://github.com/torvalds/linux/commit/a3ca86aea5079041488...
        jcranmer 5y ago
        That case wasn't one of the cases where null pointers are valid memory, judging from the commit message. (There was a case where gcc had a bug where it didn't disable that check properly, but this doesn't appear to be that case.)
- mustache_kimono 5y ago
  > ...compiler writers seem to use the presence of UB as an excuse for being allowed to break code that has worked historically. I may be dense, but I just don't understand this. Programmer compiles old C with a new compiler with certain optimizations set to on, and then is disappointed when the new compiler optimizes UB into some unsafe UB goofiness. I think expecting different behavior re: UB is folly. The language is explicit that this is the greyest of grey areas. An aside -- what is so funny to me, not re: you, but re: some C programmers and Rust, is I've heard, just this week, 1) "Why do you need Rust? Unsafe memory behavior -- that's actually the programmers fault", 2) "Rust doesn't have a spec, a language must have a spec, a spec is the only thing between us and anarchy" (yet C has a spec and this is the suck), and 3) "Rust is too fast moving to use for anything because I can't compile something I wrote 3 months ago on a new compiler and have it work" (which seems, mostly, apocryphal.) > It's a worldview mismatch, and I don't know how to bridge it. I made the aside above only because -- maybe C and its values are the problem.

nkurz 5y ago

Unfortunately, these don't help much. I thought about responding to jcranmer's post instead of yours, but wasn't sure how to approach it. It's a good comment, and I appreciated his attempt, but I feel like he's thoroughly missing the point of the complaints about UB. The complaint isn't that too much UB exists in C, but that compiler writers seem to use the presence of UB as an excuse for being allowed to break code that has worked historically. And the complaint isn't just that the code is broken, but that no apparent care is taken to avoid doing so despite the small amount of gain. It's a worldview mismatch, and I don't know how to bridge it. Your comment seems about the same as the one I responded to. You seem to assume that people who complain about UB in C would have a problem with keeping local variables in registers, but I've never seen anyone actually make this complaint. Take for example the Arxiv paper we are discussing: he doesn't bring this up. This makes me suspect that your mental model of the people who are complaining about UB in C is probably flawed. I understand the technical issue you are gesturing toward, I just don't see it as being something that refutes any of the issues brought up in the Arxiv paper. My hope was that a concrete example might help to clarify, but I do realize this might not be the right forum for that.

sanxiyn 5y ago
No, I don't assume people are against register allocation, but any concrete proposal I have seen kind of implies such conclusion. I am trying to understand what people actually want, since they seem clearly different from what people say they want. Okay let's discuss a concrete example. *x = 12345678; f(); return *x; // can you copy propagate 12345678 to here? f() does this: for (int *p = 0; p++; p < MEMSIZE) if (*p == 12345678) *p = 12345679. That is, f scans the memory for 12345678 and replace all instances with 12345679. There is no doubt this actually works that way in assembly. Things like cheat engines do this! C compilers assume this doesn't happen, because it is UB. Hence, portable assembly C compiler can't omit any load. Now I understand there are minority of people who will answer "that's what I want!", but like register allocation, I think people generally want this to optimize. But that necessarily implies memory search-and-replace can't compile in portable assembly manner.
- mpweiher 5y ago
  > any concrete proposal I have seen kind of implies such conclusion. No it does not. In your example, I personally would prefer it did not propagate the 12345678. Good grief, I wrote the deref there. > C compilers assume this doesn't happen, because it is UB. Incorrectly. IMHO. > but like register allocation, You are silently equating int x; with a memory deref. There is no need to do this. Anyway, here is part of the rationale for C: "Although it strove to give programmers the opportunity to write truly portable programs, the Committee did not want to force programmers into writing portably, to preclude the use of C as a ``high-level assembler'': the ability to write machine-specific code is one of the strengths of C. It is this principle which largely motivates drawing the distinction between strictly conforming program and conforming program" http://port70.net/~nsz/c/c89/rationale/a.html#1-5 http://port70.net/~nsz/c/c89/rationale/a.html#1-5 So the idea of C as a portable assembler is not some strange idea proposed by ignorant people who don't understand C, but an idea that was fundamental to C and fundamental to the people who created the ANSI/ISO C standard. But hey, what do they know?
- mjevans 5y ago
  A programmer wrote things in a specific order for a specific reason. Lets instead assume that the variable assignments above are to some global configuration variables and then f() also references those and the behavior of f() changes based on the previously written code. The objections from the 'C as portable assembler' camp are: * Re-ordering the order of operations across context switch bounds (curly braces and function calls). -- re-ordering non-volatile store / loads within a context is fine, and shouldn't generate warnings. * Eliminating written instructions (not calling f()) based on optimizations. -- Modification to computed work should always generate a warning so the optimization can be applied to the source code, or bugs corrected.
  - UncleMeat 5y ago
    > A programmer wrote things in a specific order for a specific reason. Is it not possible that the programmer introduced a bug? Consider the bug that caused the 911 glitch in Android phones recently. An unstable comparator was defined in a type, violating the contract that Comparable has with Java's sorting algorithms. When Java detects that this implementation violates the assumptions its sorting algorithms make, it throws an exception. Should it not do this and instead say that the programmer wrote that specific comparator on purpose and it should loop forever or produce an incorrect sort? I think most people would say "no". So why is the contract around pointer dereferencing meaningfully different?
comp54321 5y ago
There's not a bright line between optimization passes being aware of UB in unreasonable vs "exploitative" ways. The principled way to think about optimization is that an optimization pass can assume certain invariants about the code but must preserve other invariants through the transformation as long as the assumptions hold. I think basically any invariant you assume in a memory-unsafe language C could be invalidated by real code. A lot of the things people complain about are cases where the compiler got better at reasoning through valid transformations to the code, not cases where the compiler . E.g. I'm sure that even very old compilers have had optimization passes that remove redundant NULL checks that logically depend on dereferencing NULL pointers being undefined. But these probably were more ad-hoc and could only reason about relatively simple cases and constrained scopes. Another example is integer overflow. I'd bet a lot of older compilers have loop optimizations that somehow depend on the loop counter not overflowing, or pointer addresses not wrapping around. I think it's perfectly reasonable to say that C's definition of undefined behaviour is too expansive and too difficult to avoid in real code. But I think that a lot of the complaints end up being equivalent to "don't depend on UB in ways that I notice in my code" or "don't depend on UB in ways that compilers from the 90s didn't". That's not something you can practically apply or verify when building a compiler because there isn't a bright line between assuming an invariant in reasonable vs unreasonable ways, unless you can define a narrow invariant that distinguishes them.
jcranmer 5y ago
My experience with a lot of the UB-is-bad crowd is that they don't have much of an appreciation for semantics in general. That is to say, they tend to react to particular compiler behavior, and they don't have any suggestions for how to rectify the situation in a way that preserves consistent semantics. And when you try to pin down the semantics, you usually end up with a compiler that has to "do what I mean, not what I said." A salient example that people often try on me, that I don't find persuasive, is the null pointer check: void foo(int *x) { int val = *x; if (!x) return; /* do real stuff */ } "The compiler is stupid for eliminating that null check!" "So you want the code to crash because of a null pointer dereference then." "No, no, no! Do the check first, and dereference only when it's not null!" "That's not what you said..."
- mzs 5y ago
  On a lot of systems load from address zero doesn't segfault, I'm fine with the cpu loading it. I'm disappointed that the new compiler removed the check someone wrote a dozen years ago to prevent the cpu from overwriting something important though.
  - jcranmer 5y ago
    In the case where loads from address zero are legal, the dereference-means-non-null-pointer assumption is disabled.
    - mzs 5y ago
      Seems it wasn't disabled in 2009 https://github.com/torvalds/linux/commit/a3ca86aea507904148870946d599e07a340b39bf https://github.com/torvalds/linux/commit/a3ca86aea5079041488...
      - jcranmer 5y ago
        That case wasn't one of the cases where null pointers are valid memory, judging from the commit message. (There was a case where gcc had a bug where it didn't disable that check properly, but this doesn't appear to be that case.)
mustache_kimono 5y ago
> ...compiler writers seem to use the presence of UB as an excuse for being allowed to break code that has worked historically. I may be dense, but I just don't understand this. Programmer compiles old C with a new compiler with certain optimizations set to on, and then is disappointed when the new compiler optimizes UB into some unsafe UB goofiness. I think expecting different behavior re: UB is folly. The language is explicit that this is the greyest of grey areas. An aside -- what is so funny to me, not re: you, but re: some C programmers and Rust, is I've heard, just this week, 1) "Why do you need Rust? Unsafe memory behavior -- that's actually the programmers fault", 2) "Rust doesn't have a spec, a language must have a spec, a spec is the only thing between us and anarchy" (yet C has a spec and this is the suck), and 3) "Rust is too fast moving to use for anything because I can't compile something I wrote 3 months ago on a new compiler and have it work" (which seems, mostly, apocryphal.) > It's a worldview mismatch, and I don't know how to bridge it. I made the aside above only because -- maybe C and its values are the problem.