3 ms·

The issue is that basic math doesn't work. I don't dispute that no test suite can catch everything, but yes I do expect that any reasonable test suite that comp

by remram 4y ago

The issue is that basic math doesn't work. I don't dispute that no test suite can catch everything, but yes I do expect that any reasonable test suite that compiles and runs code would have caught this.

Dylan16807 4y ago
Basic math that does multiple operations on the same 8 or 16 bit value.
- gus_massa 4y ago
  I agree. I'm still confused, but I hope this is correct. IIUC the problem here is that they do something like (a // b ) * b where // is the integer division. They are not breaking all math operations, just math operations that have a very specific pattern. Also, I'm not sure if it's important that 128 <= b <= 255, because the main problem seams to be they are extending a number and the leading 1 in the binary representation is a problem (???). So you must test a very specific math pattern, with very specific numbers, and then hope that other parts of the optimizer don't optimize the problem away before the bug in the compiler makes a mistake. I don't know too much about the automatic test suit in Haskell, but I know more about the automatic test suit in Racket. In Racket there are a lot of test written manually, and some randomized test, and also tests for many of the packages. But I really doubt there is a test somewhere for this specific case. [Is there a cross language regression test collection? Something that collects weird cases, so other languages maintainers can take a look, translate the code and add it to the tests.]
  - bgamari 4y ago
    That is correct. The problem is only triggered by very particular patterns of integer operations which happen to get lowered to C-- where an operand is bound to a local register; this register then gets inappropriately mutated by the code generated by the backend. As you say, GHC, like most language implementations, has an extensive [testsuite](https://gitlab.haskell.org/ghc/ghc/-/tree/master/testsuite https://gitlab.haskell.org/ghc/ghc/-/tree/master/testsuite) (10k tests which result in over 30k testcases). Sadly, none of these tests covered this particular case. Moreover, we also have a systematic [property testing framework](https://gitlab.haskell.org/ghc/test-primops https://gitlab.haskell.org/ghc/test-primops) testing the backends and primitive operations provided by the compiler. This testsuite also didn't test this form of program since it focuses on testing the expression fragment of the Cmm language (and therefore won't generate local register bindings). This is a blind spot that I am going to look into addressing.

by remram 4y ago

Dylan16807 4y ago

Basic math that does multiple operations on the same 8 or 16 bit value.

gus_massa 4y ago
I agree. I'm still confused, but I hope this is correct. IIUC the problem here is that they do something like (a // b ) * b where // is the integer division. They are not breaking all math operations, just math operations that have a very specific pattern. Also, I'm not sure if it's important that 128 <= b <= 255, because the main problem seams to be they are extending a number and the leading 1 in the binary representation is a problem (???). So you must test a very specific math pattern, with very specific numbers, and then hope that other parts of the optimizer don't optimize the problem away before the bug in the compiler makes a mistake. I don't know too much about the automatic test suit in Haskell, but I know more about the automatic test suit in Racket. In Racket there are a lot of test written manually, and some randomized test, and also tests for many of the packages. But I really doubt there is a test somewhere for this specific case. [Is there a cross language regression test collection? Something that collects weird cases, so other languages maintainers can take a look, translate the code and add it to the tests.]
- bgamari 4y ago
  That is correct. The problem is only triggered by very particular patterns of integer operations which happen to get lowered to C-- where an operand is bound to a local register; this register then gets inappropriately mutated by the code generated by the backend. As you say, GHC, like most language implementations, has an extensive [testsuite](https://gitlab.haskell.org/ghc/ghc/-/tree/master/testsuite https://gitlab.haskell.org/ghc/ghc/-/tree/master/testsuite) (10k tests which result in over 30k testcases). Sadly, none of these tests covered this particular case. Moreover, we also have a systematic [property testing framework](https://gitlab.haskell.org/ghc/test-primops https://gitlab.haskell.org/ghc/test-primops) testing the backends and primitive operations provided by the compiler. This testsuite also didn't test this form of program since it focuses on testing the expression fragment of the Cmm language (and therefore won't generate local register bindings). This is a blind spot that I am going to look into addressing.