6 ms·

On the topic of compilers being hard: In my repositories I'm carrying around a 5 year old workaround (https://github.com/Tarsnap/libcperciva/commit/92e666e59503

On the topic of compilers being hard: In my repositories I'm carrying around a 5 year old workaround (https://github.com/Tarsnap/libcperciva/commit/92e666e59503de37b958c7e70e56ae71b281b036 https://github.com/Tarsnap/libcperciva/commit/92e666e59503de...) for a 6 year old bug (https://bugs.llvm.org/show_bug.cgi?id=21183 https://bugs.llvm.org/show_bug.cgi?id=21183) in clang/LLVM. You might think that "incorrect code gets generated" would be a maximum-priority must-fix-before-the-next-release bug in a compiler, but apparently not.

thegrandpoobah 6y ago
Actually shocked to hear this considering the number of people and companies currently working on LLVM.
- drivebycomment 6y ago
  Not that surprising given that it seems specific to asan mode and only in the presence of setjmp/longjmp. I.e. this cant be that important unless the same bug shows also in non-asan Setjmp/longjmp case. Setjmp/longjmp are often a source of trouble for the register allocator or related optimization due to its unique properties. Often the optimizing compilers treat set/longjmp as "give up almost everything around those functions" but maybe llvm is trying hard to model it precisely and failing. But I haven't personally looked at the relevant code in llvm so this is mostly a speculation.
  - h_anna_h 6y ago
    Regarding asan bugs, if you do not free all of your memory before exiting main it will complain about stuff and end up calling _exit I think which does not flush the buffers.
  - cperciva 6y ago
    The bug shows up without asan. We literally tripped over it in production with code compiled with "clang -O2".
    - drivebycomment 6y ago
      Then I agree that this is surprising. That means clang/llvm is not modeling set/longjmp correctly in general and that is not acceptable as a C compiler.
    - scatters 6y ago
      That's apparent from your ticket (which is marked as a duplicate) but on the one you linked to, all the offending compiler lines contain asan. Is there a chance this is impacting prioritization?
      - cperciva 6y ago
        Could be. I'm not an LLVM developer so I don't know how they prioritize issues.
- jcranmer 6y ago
  The code in question is related to setjmp/longjmp. As a general rule of thumb, this is something where the gut reaction to any bug report is going to be "you're doing it wrong"--the C standard actually provides a lot less guarantees here than most people believe it does. I haven't read the verbiage and the bug report in question in enough detail to ascertain if this is true or not, but it isn't in the least bit surprising that most people see setjmp/longjmp and immediately table the issue as being low-priority.
  - usefulcat 6y ago
    Having read the comments in the llvm bug report, it seems pretty clear that it is a real bug and that they have at least an idea of where the bug is.
titzer 6y ago
No, performance is maximum priority.
- naikrovek 6y ago
  That's a "great on the surface, bad when you think about it" thing if I've ever seen one. Who cares how fast the code is, if it isn't the correct code?
  - xmprt 6y ago
    I have solved the halting problem. Every program you write compiles and executes as HALT; and therefore it's possible to determine if a program halts! It might not be correct but it sure is fast!
    - mrybczyn 6y ago
      https://en.wikipedia.org/wiki/Full_employment_theorem https://en.wikipedia.org/wiki/Full_employment_theorem You've broken the full employment theorem for compiler writers.
  - qznc 6y ago
    If people cared about correctness, they would use CompCert instead of GCC or Clang.
    - tobylane 6y ago
      CompCert doesn’t guarantee these two functions will work. Thanks, didn’t know about that project.
    - pjmlp 6y ago
      Or create OSes where applications are all written in managed languages, and the use of GCC and clang is gate keeped to the OS vendor and native libraries, just an idea.
      - 6y ago
ubadair 6y ago
Wow, this is the first time I've seen the computed goto extension. Delightfully gross! https://gcc.gnu.org/onlinedocs/gcc/Labels-as-Values.html https://gcc.gnu.org/onlinedocs/gcc/Labels-as-Values.html
- KMag 6y ago
  Computed gotos are commonly used in main loops of bytecode interpreters to get a separate dispatch (indirect jump) at the end of handling each bytecode, so the CPU's branch target buffer has a better chance of predicting the indirect jump target for the next bytecode based on the previous bytecode. For instance, CPython's dispatch loop uses computed goto on supported platforms, and Erlang's BEAM VM (in non-JIT mode) uses computed branch labels to convert bytecodes into branch addresses at bytecode load time. I suppose, for twice the latency, CPU designers could implement (a hash of) the previous branch address from the source instruction pointer as part of the BTB tag, similar to using the global branch history as part of the state in the branch predictor. Presumably, the global branch history could also be used in the BTB tag to give some hint as to which bytecode we've just finished executing. Though, where it really counts, interpreter writers are already often using computed gotos, reducing the reward to cost ratio for implementing such specialized BTB improvements. On the other hand, while (1) { switch (...) { ... } } is probably rare enough (and almost certainly in a hot loop) that a very specific optimization flag might be better than a syntactic extension. Granted, an optimization flag doesn't work for Erlang's threaded code use case.
- __s 6y ago
  You'll also find them used in CPython's ceval.c I use them in both my C befunge implementations: https://github.com/serprex/Befunge/blob/c97c8e63a4eb262f3a60dac740a1de3bf1f1bf6f/marsh.c#L83 https://github.com/serprex/Befunge/blob/c97c8e63a4eb262f3a60... https://github.com/serprex/Befunge/blob/c97c8e63a4eb262f3a60dac740a1de3bf1f1bf6f/bejit.c#L232 https://github.com/serprex/Befunge/blob/c97c8e63a4eb262f3a60...
vintagedave 6y ago
Are you sure it's not fixed? https://reviews.llvm.org/D11495 https://reviews.llvm.org/D11495 was fixed on 29/7/2015.
- cperciva 6y ago
  That's marked as being Asan-specific, and the bug isn't. So... I don't think so?

thegrandpoobah 6y ago

Actually shocked to hear this considering the number of people and companies currently working on LLVM.

drivebycomment 6y ago
Not that surprising given that it seems specific to asan mode and only in the presence of setjmp/longjmp. I.e. this cant be that important unless the same bug shows also in non-asan Setjmp/longjmp case. Setjmp/longjmp are often a source of trouble for the register allocator or related optimization due to its unique properties. Often the optimizing compilers treat set/longjmp as "give up almost everything around those functions" but maybe llvm is trying hard to model it precisely and failing. But I haven't personally looked at the relevant code in llvm so this is mostly a speculation.
- h_anna_h 6y ago
  Regarding asan bugs, if you do not free all of your memory before exiting main it will complain about stuff and end up calling _exit I think which does not flush the buffers.
- cperciva 6y ago
  The bug shows up without asan. We literally tripped over it in production with code compiled with "clang -O2".
  - drivebycomment 6y ago
    Then I agree that this is surprising. That means clang/llvm is not modeling set/longjmp correctly in general and that is not acceptable as a C compiler.
  - scatters 6y ago
    That's apparent from your ticket (which is marked as a duplicate) but on the one you linked to, all the offending compiler lines contain asan. Is there a chance this is impacting prioritization?
    - cperciva 6y ago
      Could be. I'm not an LLVM developer so I don't know how they prioritize issues.
jcranmer 6y ago
The code in question is related to setjmp/longjmp. As a general rule of thumb, this is something where the gut reaction to any bug report is going to be "you're doing it wrong"--the C standard actually provides a lot less guarantees here than most people believe it does. I haven't read the verbiage and the bug report in question in enough detail to ascertain if this is true or not, but it isn't in the least bit surprising that most people see setjmp/longjmp and immediately table the issue as being low-priority.
- usefulcat 6y ago
  Having read the comments in the llvm bug report, it seems pretty clear that it is a real bug and that they have at least an idea of where the bug is.

titzer 6y ago

No, performance is maximum priority.

naikrovek 6y ago
That's a "great on the surface, bad when you think about it" thing if I've ever seen one. Who cares how fast the code is, if it isn't the correct code?
- xmprt 6y ago
  I have solved the halting problem. Every program you write compiles and executes as HALT; and therefore it's possible to determine if a program halts! It might not be correct but it sure is fast!
  - mrybczyn 6y ago
    https://en.wikipedia.org/wiki/Full_employment_theorem https://en.wikipedia.org/wiki/Full_employment_theorem You've broken the full employment theorem for compiler writers.
- qznc 6y ago
  If people cared about correctness, they would use CompCert instead of GCC or Clang.
  - tobylane 6y ago
    CompCert doesn’t guarantee these two functions will work. Thanks, didn’t know about that project.
  - pjmlp 6y ago
    Or create OSes where applications are all written in managed languages, and the use of GCC and clang is gate keeped to the OS vendor and native libraries, just an idea.
    - 6y ago

ubadair 6y ago

Wow, this is the first time I've seen the computed goto extension. Delightfully gross! https://gcc.gnu.org/onlinedocs/gcc/Labels-as-Values.html https://gcc.gnu.org/onlinedocs/gcc/Labels-as-Values.html

KMag 6y ago
Computed gotos are commonly used in main loops of bytecode interpreters to get a separate dispatch (indirect jump) at the end of handling each bytecode, so the CPU's branch target buffer has a better chance of predicting the indirect jump target for the next bytecode based on the previous bytecode. For instance, CPython's dispatch loop uses computed goto on supported platforms, and Erlang's BEAM VM (in non-JIT mode) uses computed branch labels to convert bytecodes into branch addresses at bytecode load time. I suppose, for twice the latency, CPU designers could implement (a hash of) the previous branch address from the source instruction pointer as part of the BTB tag, similar to using the global branch history as part of the state in the branch predictor. Presumably, the global branch history could also be used in the BTB tag to give some hint as to which bytecode we've just finished executing. Though, where it really counts, interpreter writers are already often using computed gotos, reducing the reward to cost ratio for implementing such specialized BTB improvements. On the other hand, while (1) { switch (...) { ... } } is probably rare enough (and almost certainly in a hot loop) that a very specific optimization flag might be better than a syntactic extension. Granted, an optimization flag doesn't work for Erlang's threaded code use case.
__s 6y ago
You'll also find them used in CPython's ceval.c I use them in both my C befunge implementations: https://github.com/serprex/Befunge/blob/c97c8e63a4eb262f3a60dac740a1de3bf1f1bf6f/marsh.c#L83 https://github.com/serprex/Befunge/blob/c97c8e63a4eb262f3a60... https://github.com/serprex/Befunge/blob/c97c8e63a4eb262f3a60dac740a1de3bf1f1bf6f/bejit.c#L232 https://github.com/serprex/Befunge/blob/c97c8e63a4eb262f3a60...

vintagedave 6y ago

Are you sure it's not fixed? https://reviews.llvm.org/D11495 https://reviews.llvm.org/D11495 was fixed on 29/7/2015.

cperciva 6y ago
That's marked as being Asan-specific, and the bug isn't. So... I don't think so?