6 ms·

GHC 9.4.2 regresses being able to do math on aarch64

103 points by nequo 4y ago

remram 4y ago
This happens in a released version? Without tests catching it?
- Confiks 4y ago
  Issue created 3 months ago [1]: "CCall testsuite doesn't test `signed` arguments and results. This would have helped catch [a previous ghc regression on aarch64]". https://gitlab.haskell.org/ghc/test-primops/-/issues/3 https://gitlab.haskell.org/ghc/test-primops/-/issues/3
- chowells 4y ago
  Would your aarch64 test suite have caught it? Testing is fine and all, but it can only find bugs the test authors predicted. There will always be some issues that test suites miss. It's obvious to add regression tests for this and related issues now that they've been found. And great! That's what regression tests are for. But testing is always going to be fundamentally incomplete in that it can only cover non-obvious things after they've already been encountered.
  - mort96 4y ago
    I would've expected a compiler to have a huge integration/regression test suite which just consists of other pieces of software's test suites. Grab the N most popular Haskell programs with test suites, compile them, run their test suites, verify that they still pass on the new version.
    - sandermvanvliet 4y ago
      Isn’t rust doing something like that? Can’t seem to find the article right now but I remember reading something about this approach.
      - steveklabnik 4y ago
        Yes, “crater” is the name of the tool.
      - estebank 4y ago
        > Crater is a tool for compiling and running tests for every crate on crates.io (and a few on GitHub). It is mainly used for checking the extent of breakage when implementing potentially breaking changes and ensuring lack of breakage by running beta vs stable compiler versions. https://rustc-dev-guide.rust-lang.org/tests/crater.html https://rustc-dev-guide.rust-lang.org/tests/crater.html https://github.com/rust-lang/crater https://github.com/rust-lang/crater https://crater.rust-lang.org/ https://crater.rust-lang.org/ Even then such a bug as GHC's can conceivably not be detected.
        moomin 4y ago
        It would be much harder to achieve this with GHC since the package manager is far from deterministic.
        bgamari 4y ago
        I'm not sure what you mean by this. `cabal-install` is quite deterministic; you can even freeze the index date for full build plan determinism. The largest challenge in this area is the fact that core libraries' interface sometimes change. Our [head.hackage](https://gitlab.haskell.org/ghc/head.hackage/ https://gitlab.haskell.org/ghc/head.hackage/) infrastructure is one way of addressing this. I agree that it would be great to leverage `head.hackage` to test GHC against a larger body of packages.
        moomin 4y ago
        It definitely _can_ be deterministic, but my experience of pulling down packages without a frozen index is that it rarely compiles. This, inevitably is because the plan depends on the current state of hackage. A combination of “it might pick a different minor version”, “different versions can cause cascading effects”, “many libraries have breaking changes” and “Haskell is extremely unforgiving of potential breaks” makes it very difficult to use without either a) being really good or b) downloading a freeze index. (I download freeze indexes.) The whole core libraries thing is difficult, in that I can see it’s a real problem for everyone constantly updating their packages (and consuming them), but also that most of the changes are actually good ideas.
    - whateveracct 4y ago
      nixpkgs
  - remram 4y ago
    The issue is that basic math doesn't work. I don't dispute that no test suite can catch everything, but yes I do expect that any reasonable test suite that compiles and runs code would have caught this.
    - Dylan16807 4y ago
      Basic math that does multiple operations on the same 8 or 16 bit value.
      - gus_massa 4y ago
        I agree. I'm still confused, but I hope this is correct. IIUC the problem here is that they do something like (a // b ) * b where // is the integer division. They are not breaking all math operations, just math operations that have a very specific pattern. Also, I'm not sure if it's important that 128 <= b <= 255, because the main problem seams to be they are extending a number and the leading 1 in the binary representation is a problem (???). So you must test a very specific math pattern, with very specific numbers, and then hope that other parts of the optimizer don't optimize the problem away before the bug in the compiler makes a mistake. I don't know too much about the automatic test suit in Haskell, but I know more about the automatic test suit in Racket. In Racket there are a lot of test written manually, and some randomized test, and also tests for many of the packages. But I really doubt there is a test somewhere for this specific case. [Is there a cross language regression test collection? Something that collects weird cases, so other languages maintainers can take a look, translate the code and add it to the tests.]
        bgamari 4y ago
        That is correct. The problem is only triggered by very particular patterns of integer operations which happen to get lowered to C-- where an operand is bound to a local register; this register then gets inappropriately mutated by the code generated by the backend. As you say, GHC, like most language implementations, has an extensive [testsuite](https://gitlab.haskell.org/ghc/ghc/-/tree/master/testsuite https://gitlab.haskell.org/ghc/ghc/-/tree/master/testsuite) (10k tests which result in over 30k testcases). Sadly, none of these tests covered this particular case. Moreover, we also have a systematic [property testing framework](https://gitlab.haskell.org/ghc/test-primops https://gitlab.haskell.org/ghc/test-primops) testing the backends and primitive operations provided by the compiler. This testsuite also didn't test this form of program since it focuses on testing the expression fragment of the Cmm language (and therefore won't generate local register bindings). This is a blind spot that I am going to look into addressing.
- whateveracct 4y ago
  It's pretty advanced in the cycle for something this nasty, but 9.4.2 is still "beta"-y. Usually you wait for a x.x.4 release to bump the MV. This sort of thing could be more explicit though.
carterschonwald 4y ago
Ouch, this is a nasty bug. Looks like it’s a secondary consequence of some arm specific issue in the 9.4 support for sized word primitive unlifted types. Good thing folks caught it relatively early in the life cycle of ghc 9.4! (Ghc 8.10.7 is the most recent bug fix release in the 8.10 series, and I try to make sure I support it in my own hackery) This looks to have a pretty straightforward fix hopefully. In the space of miscompilation bugs this seems pretty easy to catch. A pretty nasty one happened in ghc 7.6/7.8 , where the register allocator didn’t realize float and double registers were the same registers, so code that mixed the type had total garbage!
T-R 4y ago
Alexis also just posted a video on the GHC optimization pipeline: https://www.youtube.com/watch?v=fdyh3YQ-ZWI https://www.youtube.com/watch?v=fdyh3YQ-ZWI
ysangkok 4y ago
Interesting property test proposed which would catch this: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/9149/diffs https://gitlab.haskell.org/ghc/ghc/-/merge_requests/9149/dif...
NextHendrix 4y ago
Are there significant projects that run Haskell on arm specifically? Smartphone apps or just new Macs?
- lynguist 4y ago
  Almost every newly deployed datacenter CPU is ARM.
  - hardwaresofton 4y ago
    Do you have a source on this? Would love to learn how much this has shifted…
    - not2b 4y ago
      I think this is premature. https://www.datacenterknowledge.com/arm/arm-chips-gaining-data-centers-still-single-digits https://www.datacenterknowledge.com/arm/arm-chips-gaining-da...
      - derefr 4y ago
        Guessing the split here is between internal and external use-cases. Maybe 7% of e.g. Amazon EC2 is ARM, because that's the percentage of orgs AWS and other cloud vendors predict are "ready" for arm. But internally? How much of Amazon S3 is ARM? How much of Cloudflare? Google Search? Netflix? I'd expect pretty high numbers in these vertically-integrated DC niches — they can literally do whatever they want to solve these problems, and one of the largest KPIs for any of these services would be performance + scalability (i.e. requests served per OpEx dollar.)
      - lynguist 4y ago
        Why do you think it’s gaining market share? Because more of ARM is deployed than non-ARM. I couldn’t find the direct source any more, but I remember that in the year 2020 AWS deployed more ARM than non-ARM, and today it is at around 80% ARM and 20% non-ARM for a newly deployed system. You only get to single digit market share because of the enormous amount of non-ARM that still exists and is being used. ARM is growing faster than non-ARM. That’s what I’ve said.
sidkshatriya 4y ago
Sometimes when a codebase becomes too complex to comprehend, it starts failing in embarrassing ways. It is time to ask if GHC itself has become too complicated.
- whateveracct 4y ago
  I hate this kind of talk. Feels very FUD-y to me. It isn't time to ask that question though. At least, I don't think so and I've been using GHC Haskell for a decade now and for many different commercial use-cases to boot.
  - nh23423fefe 4y ago
    yeah zero value question. empty assertions of complexity and complication with faux modesty of just asking questions.
    - sidkshatriya 4y ago
      Complexity is positively correlated with bugs. Many studies have shown that. GHC is complex, perhaps too complex. I don't track GHC closely but it seems GHC does seem to suffer from more than a fair share of embarrassing compilation bugs. This feeling is derived anecdotally so I may be wrong. Happy to be corrected. Someone more conversant with GHC will have to chime in and agree/disagree. Don't let your love of Haskell get in the way of providing an objective opinion :-)
      - whateveracct 4y ago
        This is reductive thinking under the guise of "objectivity" imo. What does "complex" even mean here? How do you measure it or quantify it? You can't - and then your "objective" house of cards comes tumbling down. You can't correlate anything with hand waves. GHC is doing fine. I say this as someone more tuned into it than you. Moralizing and catastrophizing about this bug is a waste of everyone's time. The bug is fixed, the process inefficiencies that contributed to it happening are being corrected. It's software development 101.

103 points by nequo 4y ago

remram 4y ago

This happens in a released version? Without tests catching it?

Confiks 4y ago
Issue created 3 months ago [1]: "CCall testsuite doesn't test `signed` arguments and results. This would have helped catch [a previous ghc regression on aarch64]". https://gitlab.haskell.org/ghc/test-primops/-/issues/3 https://gitlab.haskell.org/ghc/test-primops/-/issues/3
chowells 4y ago
Would your aarch64 test suite have caught it? Testing is fine and all, but it can only find bugs the test authors predicted. There will always be some issues that test suites miss. It's obvious to add regression tests for this and related issues now that they've been found. And great! That's what regression tests are for. But testing is always going to be fundamentally incomplete in that it can only cover non-obvious things after they've already been encountered.
- mort96 4y ago
  I would've expected a compiler to have a huge integration/regression test suite which just consists of other pieces of software's test suites. Grab the N most popular Haskell programs with test suites, compile them, run their test suites, verify that they still pass on the new version.
  - sandermvanvliet 4y ago
    Isn’t rust doing something like that? Can’t seem to find the article right now but I remember reading something about this approach.
    - steveklabnik 4y ago
      Yes, “crater” is the name of the tool.
    - estebank 4y ago
      > Crater is a tool for compiling and running tests for every crate on crates.io (and a few on GitHub). It is mainly used for checking the extent of breakage when implementing potentially breaking changes and ensuring lack of breakage by running beta vs stable compiler versions. https://rustc-dev-guide.rust-lang.org/tests/crater.html https://rustc-dev-guide.rust-lang.org/tests/crater.html https://github.com/rust-lang/crater https://github.com/rust-lang/crater https://crater.rust-lang.org/ https://crater.rust-lang.org/ Even then such a bug as GHC's can conceivably not be detected.
      - moomin 4y ago
        It would be much harder to achieve this with GHC since the package manager is far from deterministic.
        bgamari 4y ago
        I'm not sure what you mean by this. `cabal-install` is quite deterministic; you can even freeze the index date for full build plan determinism. The largest challenge in this area is the fact that core libraries' interface sometimes change. Our [head.hackage](https://gitlab.haskell.org/ghc/head.hackage/ https://gitlab.haskell.org/ghc/head.hackage/) infrastructure is one way of addressing this. I agree that it would be great to leverage `head.hackage` to test GHC against a larger body of packages.
        moomin 4y ago
        It definitely _can_ be deterministic, but my experience of pulling down packages without a frozen index is that it rarely compiles. This, inevitably is because the plan depends on the current state of hackage. A combination of “it might pick a different minor version”, “different versions can cause cascading effects”, “many libraries have breaking changes” and “Haskell is extremely unforgiving of potential breaks” makes it very difficult to use without either a) being really good or b) downloading a freeze index. (I download freeze indexes.) The whole core libraries thing is difficult, in that I can see it’s a real problem for everyone constantly updating their packages (and consuming them), but also that most of the changes are actually good ideas.
  - whateveracct 4y ago
    nixpkgs
- remram 4y ago
  The issue is that basic math doesn't work. I don't dispute that no test suite can catch everything, but yes I do expect that any reasonable test suite that compiles and runs code would have caught this.
  - Dylan16807 4y ago
    Basic math that does multiple operations on the same 8 or 16 bit value.
    - gus_massa 4y ago
      I agree. I'm still confused, but I hope this is correct. IIUC the problem here is that they do something like (a // b ) * b where // is the integer division. They are not breaking all math operations, just math operations that have a very specific pattern. Also, I'm not sure if it's important that 128 <= b <= 255, because the main problem seams to be they are extending a number and the leading 1 in the binary representation is a problem (???). So you must test a very specific math pattern, with very specific numbers, and then hope that other parts of the optimizer don't optimize the problem away before the bug in the compiler makes a mistake. I don't know too much about the automatic test suit in Haskell, but I know more about the automatic test suit in Racket. In Racket there are a lot of test written manually, and some randomized test, and also tests for many of the packages. But I really doubt there is a test somewhere for this specific case. [Is there a cross language regression test collection? Something that collects weird cases, so other languages maintainers can take a look, translate the code and add it to the tests.]
      - bgamari 4y ago
        That is correct. The problem is only triggered by very particular patterns of integer operations which happen to get lowered to C-- where an operand is bound to a local register; this register then gets inappropriately mutated by the code generated by the backend. As you say, GHC, like most language implementations, has an extensive [testsuite](https://gitlab.haskell.org/ghc/ghc/-/tree/master/testsuite https://gitlab.haskell.org/ghc/ghc/-/tree/master/testsuite) (10k tests which result in over 30k testcases). Sadly, none of these tests covered this particular case. Moreover, we also have a systematic [property testing framework](https://gitlab.haskell.org/ghc/test-primops https://gitlab.haskell.org/ghc/test-primops) testing the backends and primitive operations provided by the compiler. This testsuite also didn't test this form of program since it focuses on testing the expression fragment of the Cmm language (and therefore won't generate local register bindings). This is a blind spot that I am going to look into addressing.
whateveracct 4y ago
It's pretty advanced in the cycle for something this nasty, but 9.4.2 is still "beta"-y. Usually you wait for a x.x.4 release to bump the MV. This sort of thing could be more explicit though.

carterschonwald 4y ago

Ouch, this is a nasty bug. Looks like it’s a secondary consequence of some arm specific issue in the 9.4 support for sized word primitive unlifted types. Good thing folks caught it relatively early in the life cycle of ghc 9.4! (Ghc 8.10.7 is the most recent bug fix release in the 8.10 series, and I try to make sure I support it in my own hackery) This looks to have a pretty straightforward fix hopefully. In the space of miscompilation bugs this seems pretty easy to catch. A pretty nasty one happened in ghc 7.6/7.8 , where the register allocator didn’t realize float and double registers were the same registers, so code that mixed the type had total garbage!

T-R 4y ago

Alexis also just posted a video on the GHC optimization pipeline: https://www.youtube.com/watch?v=fdyh3YQ-ZWI https://www.youtube.com/watch?v=fdyh3YQ-ZWI

ysangkok 4y ago

Interesting property test proposed which would catch this: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/9149/diffs https://gitlab.haskell.org/ghc/ghc/-/merge_requests/9149/dif...

NextHendrix 4y ago

Are there significant projects that run Haskell on arm specifically? Smartphone apps or just new Macs?

lynguist 4y ago
Almost every newly deployed datacenter CPU is ARM.
- hardwaresofton 4y ago
  Do you have a source on this? Would love to learn how much this has shifted…
  - not2b 4y ago
    I think this is premature. https://www.datacenterknowledge.com/arm/arm-chips-gaining-data-centers-still-single-digits https://www.datacenterknowledge.com/arm/arm-chips-gaining-da...
    - derefr 4y ago
      Guessing the split here is between internal and external use-cases. Maybe 7% of e.g. Amazon EC2 is ARM, because that's the percentage of orgs AWS and other cloud vendors predict are "ready" for arm. But internally? How much of Amazon S3 is ARM? How much of Cloudflare? Google Search? Netflix? I'd expect pretty high numbers in these vertically-integrated DC niches — they can literally do whatever they want to solve these problems, and one of the largest KPIs for any of these services would be performance + scalability (i.e. requests served per OpEx dollar.)
    - lynguist 4y ago
      Why do you think it’s gaining market share? Because more of ARM is deployed than non-ARM. I couldn’t find the direct source any more, but I remember that in the year 2020 AWS deployed more ARM than non-ARM, and today it is at around 80% ARM and 20% non-ARM for a newly deployed system. You only get to single digit market share because of the enormous amount of non-ARM that still exists and is being used. ARM is growing faster than non-ARM. That’s what I’ve said.

sidkshatriya 4y ago

Sometimes when a codebase becomes too complex to comprehend, it starts failing in embarrassing ways. It is time to ask if GHC itself has become too complicated.

whateveracct 4y ago
I hate this kind of talk. Feels very FUD-y to me. It isn't time to ask that question though. At least, I don't think so and I've been using GHC Haskell for a decade now and for many different commercial use-cases to boot.
- nh23423fefe 4y ago
  yeah zero value question. empty assertions of complexity and complication with faux modesty of just asking questions.
  - sidkshatriya 4y ago
    Complexity is positively correlated with bugs. Many studies have shown that. GHC is complex, perhaps too complex. I don't track GHC closely but it seems GHC does seem to suffer from more than a fair share of embarrassing compilation bugs. This feeling is derived anecdotally so I may be wrong. Happy to be corrected. Someone more conversant with GHC will have to chime in and agree/disagree. Don't let your love of Haskell get in the way of providing an objective opinion :-)
    - whateveracct 4y ago
      This is reductive thinking under the guise of "objectivity" imo. What does "complex" even mean here? How do you measure it or quantify it? You can't - and then your "objective" house of cards comes tumbling down. You can't correlate anything with hand waves. GHC is doing fine. I say this as someone more tuned into it than you. Moralizing and catastrophizing about this bug is a waste of everyone's time. The bug is fixed, the process inefficiencies that contributed to it happening are being corrected. It's software development 101.