3 ms·

Sometimes when a codebase becomes too complex to comprehend, it starts failing in embarrassing ways. It is time to ask if GHC itself has become too complicated

Sometimes when a codebase becomes too complex to comprehend, it starts failing in embarrassing ways. It is time to ask if GHC itself has become too complicated.

whateveracct 4y ago
I hate this kind of talk. Feels very FUD-y to me. It isn't time to ask that question though. At least, I don't think so and I've been using GHC Haskell for a decade now and for many different commercial use-cases to boot.
- nh23423fefe 4y ago
  yeah zero value question. empty assertions of complexity and complication with faux modesty of just asking questions.
  - sidkshatriya 4y ago
    Complexity is positively correlated with bugs. Many studies have shown that. GHC is complex, perhaps too complex. I don't track GHC closely but it seems GHC does seem to suffer from more than a fair share of embarrassing compilation bugs. This feeling is derived anecdotally so I may be wrong. Happy to be corrected. Someone more conversant with GHC will have to chime in and agree/disagree. Don't let your love of Haskell get in the way of providing an objective opinion :-)
    - whateveracct 4y ago
      This is reductive thinking under the guise of "objectivity" imo. What does "complex" even mean here? How do you measure it or quantify it? You can't - and then your "objective" house of cards comes tumbling down. You can't correlate anything with hand waves. GHC is doing fine. I say this as someone more tuned into it than you. Moralizing and catastrophizing about this bug is a waste of everyone's time. The bug is fixed, the process inefficiencies that contributed to it happening are being corrected. It's software development 101.
    - xdavidliu 4y ago
      do you feel the same way about other libraries or software? Do you have reasoning that you can articulate that GHC is unnecessarily complex in ways that other comparable software is not?
    - bgamari 4y ago
      GHC is indeed a large piece of software and, like all software, has its share of bugs. Some of these bugs are indeed attributable to GHC's complexity or, rather, the complexity of its domain: compiling high-level languages to efficient machine code is not easy. To determine whether GHC is too complex we need a definition of what constitutes "too complex". I think a reasonable definition would be "too complex to be managed via abstraction". In general I think GHC does a reasonably good job here. There is a reason why Haskell programs passes through four different intermediate languages as it works its way through GHC. Each of these languages, and associated compiler passes, can focus on a specific set of needs constrained by a specific set of invariants. Where this approach to encapsulating complexity can fall short is where we must maintain hard-to-check relationships between the state of the machine at runtime and the compiler. This is the source of many of the bugs which are most readily attributable to GHC's complexity (e.g. [#13615](https://gitlab.haskell.org/ghc/ghc/-/issues/13615 https://gitlab.haskell.org/ghc/ghc/-/issues/13615) and [#14346](https://gitlab.haskell.org/ghc/ghc/-/issues/14346 https://gitlab.haskell.org/ghc/ghc/-/issues/14346) come to mind here). However, the sign-extension issue noted in this thread is, in my opinion, not one of these issues. In many ways it is a very boring bug, arising from a simple misunderstanding of the semantics of a function in the code generator (namely, that `getSomeReg` isn't guaranteed to return a fresh local register). There are a number of ways that this could have been caught: better code review, more thorough tests, better use of type safety in the backend. It's certainly an serious (and, frankly, embarrassing) bug, but I don't think it is a sign that the compiler has accrued more complexity than can be managed.
axman6 4y ago
Put simply: No. The native code generator for aarch64 is relatively new, and is bound to have bugs in it that haven't been caught, as others have mentioned, because no one thought to test for this specific thing. There's already a patch for this bug, and a new regression test to catch it. These things happen in all compilers, and all software, relatively often. I don't understand how a bug in GHC has blown up more than any bug in any recent GCC or clang - probably because the bug reporter decided that hyperbole was going to be their tool of choice instead of behaving like an adult.
- protomikron 4y ago
  > I don't understand how a bug in GHC has blown up more than any bug in any recent GCC or clang - probably because the bug reporter decided that hyperbole was going to be their tool of choice instead of behaving like an adult. It didn't blow up more or less than bugs in other language implementations. When there are serious bugs in GCC, LLVM the JVM, etc. it always goes to HN's front page.
  - account42 4y ago
    Most (even serious) bugs don't even get submitted here not to mention reaching the front page.
- whateveracct 4y ago
  Also, aarch64 Haskell is almost exclusively not in production (although there's probably some iOS/macOS apps out there - very cool if so!). This bug shows up on devs' M1 MBPs and the CI used to build their Nix cache. It's a nasty bug, but in reality it isn't some P0 monstrosity despite how "eXtReMeLy aLaRmInG" it seems.

Sometimes when a codebase becomes too complex to comprehend, it starts failing in embarrassing ways. It is time to ask if GHC itself has become too complicated.

whateveracct 4y ago

I hate this kind of talk. Feels very FUD-y to me. It isn't time to ask that question though. At least, I don't think so and I've been using GHC Haskell for a decade now and for many different commercial use-cases to boot.

nh23423fefe 4y ago
yeah zero value question. empty assertions of complexity and complication with faux modesty of just asking questions.
- sidkshatriya 4y ago
  Complexity is positively correlated with bugs. Many studies have shown that. GHC is complex, perhaps too complex. I don't track GHC closely but it seems GHC does seem to suffer from more than a fair share of embarrassing compilation bugs. This feeling is derived anecdotally so I may be wrong. Happy to be corrected. Someone more conversant with GHC will have to chime in and agree/disagree. Don't let your love of Haskell get in the way of providing an objective opinion :-)
  - whateveracct 4y ago
    This is reductive thinking under the guise of "objectivity" imo. What does "complex" even mean here? How do you measure it or quantify it? You can't - and then your "objective" house of cards comes tumbling down. You can't correlate anything with hand waves. GHC is doing fine. I say this as someone more tuned into it than you. Moralizing and catastrophizing about this bug is a waste of everyone's time. The bug is fixed, the process inefficiencies that contributed to it happening are being corrected. It's software development 101.
  - xdavidliu 4y ago
    do you feel the same way about other libraries or software? Do you have reasoning that you can articulate that GHC is unnecessarily complex in ways that other comparable software is not?
  - bgamari 4y ago
    GHC is indeed a large piece of software and, like all software, has its share of bugs. Some of these bugs are indeed attributable to GHC's complexity or, rather, the complexity of its domain: compiling high-level languages to efficient machine code is not easy. To determine whether GHC is too complex we need a definition of what constitutes "too complex". I think a reasonable definition would be "too complex to be managed via abstraction". In general I think GHC does a reasonably good job here. There is a reason why Haskell programs passes through four different intermediate languages as it works its way through GHC. Each of these languages, and associated compiler passes, can focus on a specific set of needs constrained by a specific set of invariants. Where this approach to encapsulating complexity can fall short is where we must maintain hard-to-check relationships between the state of the machine at runtime and the compiler. This is the source of many of the bugs which are most readily attributable to GHC's complexity (e.g. [#13615](https://gitlab.haskell.org/ghc/ghc/-/issues/13615 https://gitlab.haskell.org/ghc/ghc/-/issues/13615) and [#14346](https://gitlab.haskell.org/ghc/ghc/-/issues/14346 https://gitlab.haskell.org/ghc/ghc/-/issues/14346) come to mind here). However, the sign-extension issue noted in this thread is, in my opinion, not one of these issues. In many ways it is a very boring bug, arising from a simple misunderstanding of the semantics of a function in the code generator (namely, that `getSomeReg` isn't guaranteed to return a fresh local register). There are a number of ways that this could have been caught: better code review, more thorough tests, better use of type safety in the backend. It's certainly an serious (and, frankly, embarrassing) bug, but I don't think it is a sign that the compiler has accrued more complexity than can be managed.

axman6 4y ago

Put simply: No. The native code generator for aarch64 is relatively new, and is bound to have bugs in it that haven't been caught, as others have mentioned, because no one thought to test for this specific thing. There's already a patch for this bug, and a new regression test to catch it. These things happen in all compilers, and all software, relatively often. I don't understand how a bug in GHC has blown up more than any bug in any recent GCC or clang - probably because the bug reporter decided that hyperbole was going to be their tool of choice instead of behaving like an adult.

protomikron 4y ago
> I don't understand how a bug in GHC has blown up more than any bug in any recent GCC or clang - probably because the bug reporter decided that hyperbole was going to be their tool of choice instead of behaving like an adult. It didn't blow up more or less than bugs in other language implementations. When there are serious bugs in GCC, LLVM the JVM, etc. it always goes to HN's front page.
- account42 4y ago
  Most (even serious) bugs don't even get submitted here not to mention reaching the front page.
whateveracct 4y ago
Also, aarch64 Haskell is almost exclusively not in production (although there's probably some iOS/macOS apps out there - very cool if so!). This bug shows up on devs' M1 MBPs and the CI used to build their Nix cache. It's a nasty bug, but in reality it isn't some P0 monstrosity despite how "eXtReMeLy aLaRmInG" it seems.