5 ms·

The headline is a bit too generic. This organization is being founded with the support of Simon Peyton Jones and has corporate backing. It appears that the inte

The headline is a bit too generic. This organization is being founded with the support of Simon Peyton Jones and has corporate backing. It appears that the intent is to focus on pain points in the Haskell toolchain and libraries.

Blikkentrekker 6y ago
To me, the major major unsung pain point with Haskell and any non-strict language that makes it unsuitable for most applications is it's inability to use dynamic libraries as we're all used to. For an Haskell implementation to have any efficiency, it must re-order the order of execution inside of function calls as it sees fit, which it can do due to the non-strict semantics. This is fine in theory, but dynamic libraries aren't built on that assumption; they're, naturally, built upon the idea that they are a black box and that the consumer simply calls the function, obtains the result, and does not care nor has access to the insides. A Haskell implementation must have knowledge thereof, thus, the practical effect is that with Haskell it's impossible to, say, fix a security problem in a dynamic library and simply have all consumers seamlessly benefit from that fix — every consumer must be recompiled with the new library's source. And this problem is very seldom raised in discussions — dynamic libraries are fundamentally designed on the assumption of strict semantics.
- simonfxr 6y ago
  I don't see how strict vs. lazy has anything to do with linkage. I guess you are talking about foreign functions defined in a dynamic library. Calling a foreign function (regardless of linkage) will just evaluate the arguments and pass them according to the calling convention of the foreign ABI. If you link to library dynamically and it gets updated (in an ABI compatible way) you don't have to recompile anything, it just works as you would expect.
  - Blikkentrekker 6y ago
    No, the issue does not apply to the f.f.i., because those are called with strict semantics, as C expects that. It's about dynamic libraries written in Haskell itself. Try compiling a Haskell program in GHC with the `-dynamic` flag, and then update any of the Haskell libraries it is linked against, after this, the program will fail to start with a linker error, and must be recompiled itself. https://www.reddit.com/r/archlinux/comments/7jtemw/which_package_do_you_hate_the_most/dra5q7v/ https://www.reddit.com/r/archlinux/comments/7jtemw/which_pac... See this discussion there which explains why, in essence, dynamic linking is not a feasible route with Haskell. Arch Linux currently does this, or at least at the time of writing there, which leads to all Arch Haskell packages being required to be recompiled and reinstalled, if but a simple library be updated that they all use.
    - tome 6y ago
      Yes, but this doesn't seem to have anything to do with lazy evaluation, rather it is because GHC is quite close to a whole-program optimizer.
      - Blikkentrekker 6y ago
        Optimizing lazy evaluation requires this kind of hole-program optimizations. Non-strict semantics would be prohibitively slow without these optimizations. You won't find a Haskell compiler that doesn't do this.
    - simonfxr 6y ago
      Okay you are right, GHC does not produce a stable ABI with dynamic libraries. But it could do so in principal, no problem with laziness itself. It just would be a lot slower without all the cross module inlining and so GHC chooses not to. The archlinux haskell packagers could just ship programs as statically linked binaries and not recompile them on each minor dependency upgrade (except if there is a security upgrade). The binaries would not have any dependency on haskell packages themselves and you would not be forced to upgrade the whole package graph when some dependency is bumped, that would also save a lot of download bandwidth. I'm not sure that anyone really needs haskell packages (besides standalone programs) from the system package manager. No haskell dev I know builds their programs this way.
      - Blikkentrekker 6y ago
        > Okay you are right, GHC does not produce a stable ABI with dynamic libraries. But it could do so in principal, no problem with laziness itself. It just would be a lot slower without all the cross module inlining and so GHC chooses not to. It could do so if it were to be willing to sacrifice having even a modicum of performance. Lazy evaluation is incredibly slow without these. The theory behind lazy evaluation, is that it can gain back some of the performance it inhærently loses by allowing these kinds of aggressive hole-program optimizations because code can freely be re-ordered as no code has side-effects. Without taking advantage of this re-ordering, one is only met with the pœnalties. Note that in Haskell very fundamental primitive logical operations such as `||`, `&&` and `if` receive no special treatment from the implementation and are ordinary functions. If these were not allowed to be optimized as such, and they would be provided as a library, then the code would be prohibitively slow and would essentially require that a thunk be passed at every instance of a normal logical construct. > The archlinux haskell packagers could just ship programs as statically linked binaries and not recompile them on each minor dependency upgrade (except if there is a security upgrade). The binaries would not have any dependency on haskell packages themselves and you would not be forced to upgrade the whole package graph when some dependency is bumped, that would also save a lot of download bandwidth. I'm not sure that anyone really needs haskell packages (besides standalone programs) from the system package manager. No haskell dev I know builds their programs this way. Indeed — no one does, because the unsung pain with Haskell is that it's impossible. With strict languages one has the option for dynamic linking, and dynamic linking seems to be what has won out due to it's convenient “update once; update everywhere.” benefits — that is not available to non-strict languages, for which static linking is the only viable way of distribution.
- lalaithion 6y ago
  The trend in the last 10 years has been away from dynamic linking and toward static linking; Rust, Go, Nim and Zig all have static linking as the default.
  - a1369209993 6y ago
    > Rust, [...] have static linking as the default. Citation needed? Something like 20% of the reason I gave up on Rust was precisely that there was no documented way to force it to generate a static executable.
    - lalaithion 6y ago
      To clarify for Rust; Rust statically links other Rust code into your program by default. That's the relevant comparison in my post, because Haskell also links dynamically to glibc by default, but it statically links with other Haskell code. Presumably that's what Blikkentrekker meant. If you want to have a 100% static executable, that's been possible for a while now: https://doc.rust-lang.org/edition-guide/rust-2018/platform-and-target-support/musl-support-for-fully-static-binaries.html https://doc.rust-lang.org/edition-guide/rust-2018/platform-a...
      - a1369209993 6y ago
        > > no documented way to force [the reference implementation, with the standard libraries (in particular libc) that the default system semantics are dependent on] to generate a static executable. > https://doc.rust-lang.org/edition-guide/rust-2018/platform-and-target-support/musl-support-for-fully-static-binaries.html https://doc.rust-lang.org/edition-guide/rust-2018/platform-a... Yes, I'm aware of that; the fact that they continued using glibc as their default (aka canonical) OS/syscall interface after discovering that it was so impossible to statically link that people had to retarget code to a whole second libc was one of the final nails in the proverbial coffin for me. > because Haskell also links dynamically to glibc by default, but it statically links with other Haskell code. Not particularly relevant, since my inquery was mainly about Rust, but on my system I get: $ echo 'main = pure ()' > foo.hs $ ghc foo.hs [1 of 1] Compiling Main ( foo.hs, /tmp/tmp.icUr2pqZXx/Main.o ) Linking foo ... $ file foo foo: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped Admittedly, it's entirely possible that I fixed ghc and/or gcc at some point and forgot (I've fixed other bugs, but I don't recall fixing this one).
- tome 6y ago
  I'm afraid I don't get the connection between non-strict semantics and dynamic linking at all. As far as I know they are completely orthogonal. I didn't follow your reasoning. Could you expand?
  - Blikkentrekker 6y ago
    They're very related. Due to Haskell's non-strict semantics, if a dynamic library, written in Haskell, that some Haskell program is linked to is recompiled, then the code that is linked to it must also be recompiled.
    - pklausler 6y ago
      Why is this a necessary consequence of lazy execution?
    - tome 6y ago
      I'm having trouble grasping that. Why would non-strict semantics imply that code must be recompiled if a dynamic library changes?
      - Blikkentrekker 6y ago
        With non-strict semantics, function execution is re-ordered and turned inside out. For instance let us say we have pseudocode: let list = [1,2,3,4,5]; let new_list = do_something(list); lew newer_list = do_something_else(new_list); Both these functions consume and return lists. In a strict language, this process of iterating over the entire list internally in these two functions would happen twice; this would be wasteful. A non-strict language however is permitted to take the side-effect-less code apart, and restructure it in such a way that the list is only traversed once, essentially combining these two functions into one function and letting what each does run successively, for this purpose it is permitted to generate a new function that combines what both does, and doing so iterating the list only once, and optimizers are typically clever enough to be able to do so. Suppose that both of these functions be library function black boxes, provided by a dynamic library, then the implementation can no longer do this as it calls both from a library. The way Haskell is written is completely dependent on this; it would be awfully slow if the compiler weren't allowed to do this. The result in practice is that updating a dynamic library written in Haskell requires that all it's consumers be recompiled to compile against the new version. Or otherwise said: there is no concept of a stable a.b.i. in Haskell.
- lallysingh 6y ago
  Do you mean no dynamic lib support for Haskell code, separately from C libraries that can be dynamically linked?
  - Blikkentrekker 6y ago
    There is support for it, but no support for a stable a.b.i., as such with every minor change to the library, nothing linked to it can run without a recompilation. This is due to the non-strict semantics. Consider that in a C library, or any other strict language, the function loaded in the dynamic library is called by first evaluating all of the arguments, and then calling the function by value with the evaluated arguments. This is not how Haskell operates, where the arguments are, in effect, placed inside the function body directly and may, or may not be evaluated, as such a function can't be a black box that can arbitrarily change it's internal workings, so long as the outside a.b.i. remain stable — a program compiled against a dynamic library is fundamentally compiled against the inner workings of functions, rather than simply against the stable interface they proffer to the outside world.
egonschiele 6y ago
That is great news. Haskell toolchain could use some help. I wonder if they'll work on the documentation and error messaging too. I think people are scared off when they realize that some of the libraries have no docs and you just have to read the type signatures.
- sfvisser 6y ago
  Definitely agree with you that more documentation, especially with examples, would be great. However, reading just the types of a well written Haskell API probably gives me greater inside in its usage than most documentation I’ve read on npm packages. (To name an example I’m familiar with) There are so many very general patterns and idioms in Haskell being used all over the place that reading the signatures in the API docs can be extremely valuable.
  - tome 6y ago
    I program roughly equal amounts in Python and Haskell and every time I read Python documentation I wish it were as good as Haskell's. Having type signatures goes a long, long way to understanding an API. Python replaces that with rambling, unclear, prose. The same goes for Python's standard library and popular libraries like flask, pandas and matplotlib. (Haskell's direct links to source are also good, but there's no fundamental reason Python couldn't have those. I've never seen it though.)
pbowyer 6y ago
> It appears that the intent is to focus on pain points in the Haskell toolchain and libraries. Good. I set myself the challenge of compiling a Haskell program [1] during the Christmas holidays. It was meant to be a "one mince pie" challenge, but after an hour I discovered the VM I used didn't have enough RAM (during compilation we were approaching 4GB), then I ran out of disk space as stack approaches 5GB & I had other stuff installed. Once a few hours had gone by (this program isn't fast to compile) I had a working program. I now have to figure out if I can distribute just the resulting binary to other servers, or if it needs other software like GHC installing. Having finished the pack of mince pies, that can wait to another day. I know when I first started compiling C/C++ software there was a learning curve and it took hours the first time, but I found it easier to get started. With Haskell, the way one version of GHC is installed first and then Stack installs a completely isolated version is confusing; plus the inscrutable error messages (haven't got it to hand, but one means OOM but doesn't say that - it takes a Google to find the GitHub issue to work that out). And this is before I try and experiment/decide to learn some Haskell. Apart from the error messages they're not issues with Haskell per se, but they contribute to the experience of it. 1. https://github.com/facebook/duckling https://github.com/facebook/duckling
- smadge 6y ago
  I upgraded my personal VPS from 1GB to 2GB RAM when I couldn't compile some Haskell standard library modules for a simple project.
- mrkeen 6y ago
  When I started, I took the command I used for compiling c programs and substituted a letter. gcc -Wall -O3 -o ./main main.c ghc -Wall -O3 -o ./main main.hs I still compile that way every now and then.
  - pbowyer 6y ago
    Good to know, thanks. The README says to use stack, so I've stuck with that so far. I'll look for instructions for using ghc

Blikkentrekker 6y ago

To me, the major major unsung pain point with Haskell and any non-strict language that makes it unsuitable for most applications is it's inability to use dynamic libraries as we're all used to. For an Haskell implementation to have any efficiency, it must re-order the order of execution inside of function calls as it sees fit, which it can do due to the non-strict semantics. This is fine in theory, but dynamic libraries aren't built on that assumption; they're, naturally, built upon the idea that they are a black box and that the consumer simply calls the function, obtains the result, and does not care nor has access to the insides. A Haskell implementation must have knowledge thereof, thus, the practical effect is that with Haskell it's impossible to, say, fix a security problem in a dynamic library and simply have all consumers seamlessly benefit from that fix — every consumer must be recompiled with the new library's source. And this problem is very seldom raised in discussions — dynamic libraries are fundamentally designed on the assumption of strict semantics.

simonfxr 6y ago
I don't see how strict vs. lazy has anything to do with linkage. I guess you are talking about foreign functions defined in a dynamic library. Calling a foreign function (regardless of linkage) will just evaluate the arguments and pass them according to the calling convention of the foreign ABI. If you link to library dynamically and it gets updated (in an ABI compatible way) you don't have to recompile anything, it just works as you would expect.
- Blikkentrekker 6y ago
  No, the issue does not apply to the f.f.i., because those are called with strict semantics, as C expects that. It's about dynamic libraries written in Haskell itself. Try compiling a Haskell program in GHC with the `-dynamic` flag, and then update any of the Haskell libraries it is linked against, after this, the program will fail to start with a linker error, and must be recompiled itself. https://www.reddit.com/r/archlinux/comments/7jtemw/which_package_do_you_hate_the_most/dra5q7v/ https://www.reddit.com/r/archlinux/comments/7jtemw/which_pac... See this discussion there which explains why, in essence, dynamic linking is not a feasible route with Haskell. Arch Linux currently does this, or at least at the time of writing there, which leads to all Arch Haskell packages being required to be recompiled and reinstalled, if but a simple library be updated that they all use.
  - tome 6y ago
    Yes, but this doesn't seem to have anything to do with lazy evaluation, rather it is because GHC is quite close to a whole-program optimizer.
    - Blikkentrekker 6y ago
      Optimizing lazy evaluation requires this kind of hole-program optimizations. Non-strict semantics would be prohibitively slow without these optimizations. You won't find a Haskell compiler that doesn't do this.
  - simonfxr 6y ago
    Okay you are right, GHC does not produce a stable ABI with dynamic libraries. But it could do so in principal, no problem with laziness itself. It just would be a lot slower without all the cross module inlining and so GHC chooses not to. The archlinux haskell packagers could just ship programs as statically linked binaries and not recompile them on each minor dependency upgrade (except if there is a security upgrade). The binaries would not have any dependency on haskell packages themselves and you would not be forced to upgrade the whole package graph when some dependency is bumped, that would also save a lot of download bandwidth. I'm not sure that anyone really needs haskell packages (besides standalone programs) from the system package manager. No haskell dev I know builds their programs this way.
    - Blikkentrekker 6y ago
      > Okay you are right, GHC does not produce a stable ABI with dynamic libraries. But it could do so in principal, no problem with laziness itself. It just would be a lot slower without all the cross module inlining and so GHC chooses not to. It could do so if it were to be willing to sacrifice having even a modicum of performance. Lazy evaluation is incredibly slow without these. The theory behind lazy evaluation, is that it can gain back some of the performance it inhærently loses by allowing these kinds of aggressive hole-program optimizations because code can freely be re-ordered as no code has side-effects. Without taking advantage of this re-ordering, one is only met with the pœnalties. Note that in Haskell very fundamental primitive logical operations such as `||`, `&&` and `if` receive no special treatment from the implementation and are ordinary functions. If these were not allowed to be optimized as such, and they would be provided as a library, then the code would be prohibitively slow and would essentially require that a thunk be passed at every instance of a normal logical construct. > The archlinux haskell packagers could just ship programs as statically linked binaries and not recompile them on each minor dependency upgrade (except if there is a security upgrade). The binaries would not have any dependency on haskell packages themselves and you would not be forced to upgrade the whole package graph when some dependency is bumped, that would also save a lot of download bandwidth. I'm not sure that anyone really needs haskell packages (besides standalone programs) from the system package manager. No haskell dev I know builds their programs this way. Indeed — no one does, because the unsung pain with Haskell is that it's impossible. With strict languages one has the option for dynamic linking, and dynamic linking seems to be what has won out due to it's convenient “update once; update everywhere.” benefits — that is not available to non-strict languages, for which static linking is the only viable way of distribution.
lalaithion 6y ago
The trend in the last 10 years has been away from dynamic linking and toward static linking; Rust, Go, Nim and Zig all have static linking as the default.
- a1369209993 6y ago
  > Rust, [...] have static linking as the default. Citation needed? Something like 20% of the reason I gave up on Rust was precisely that there was no documented way to force it to generate a static executable.
  - lalaithion 6y ago
    To clarify for Rust; Rust statically links other Rust code into your program by default. That's the relevant comparison in my post, because Haskell also links dynamically to glibc by default, but it statically links with other Haskell code. Presumably that's what Blikkentrekker meant. If you want to have a 100% static executable, that's been possible for a while now: https://doc.rust-lang.org/edition-guide/rust-2018/platform-and-target-support/musl-support-for-fully-static-binaries.html https://doc.rust-lang.org/edition-guide/rust-2018/platform-a...
    - a1369209993 6y ago
      > > no documented way to force [the reference implementation, with the standard libraries (in particular libc) that the default system semantics are dependent on] to generate a static executable. > https://doc.rust-lang.org/edition-guide/rust-2018/platform-and-target-support/musl-support-for-fully-static-binaries.html https://doc.rust-lang.org/edition-guide/rust-2018/platform-a... Yes, I'm aware of that; the fact that they continued using glibc as their default (aka canonical) OS/syscall interface after discovering that it was so impossible to statically link that people had to retarget code to a whole second libc was one of the final nails in the proverbial coffin for me. > because Haskell also links dynamically to glibc by default, but it statically links with other Haskell code. Not particularly relevant, since my inquery was mainly about Rust, but on my system I get: $ echo 'main = pure ()' > foo.hs $ ghc foo.hs [1 of 1] Compiling Main ( foo.hs, /tmp/tmp.icUr2pqZXx/Main.o ) Linking foo ... $ file foo foo: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped Admittedly, it's entirely possible that I fixed ghc and/or gcc at some point and forgot (I've fixed other bugs, but I don't recall fixing this one).
tome 6y ago
I'm afraid I don't get the connection between non-strict semantics and dynamic linking at all. As far as I know they are completely orthogonal. I didn't follow your reasoning. Could you expand?
- Blikkentrekker 6y ago
  They're very related. Due to Haskell's non-strict semantics, if a dynamic library, written in Haskell, that some Haskell program is linked to is recompiled, then the code that is linked to it must also be recompiled.
  - pklausler 6y ago
    Why is this a necessary consequence of lazy execution?
  - tome 6y ago
    I'm having trouble grasping that. Why would non-strict semantics imply that code must be recompiled if a dynamic library changes?
    - Blikkentrekker 6y ago
      With non-strict semantics, function execution is re-ordered and turned inside out. For instance let us say we have pseudocode: let list = [1,2,3,4,5]; let new_list = do_something(list); lew newer_list = do_something_else(new_list); Both these functions consume and return lists. In a strict language, this process of iterating over the entire list internally in these two functions would happen twice; this would be wasteful. A non-strict language however is permitted to take the side-effect-less code apart, and restructure it in such a way that the list is only traversed once, essentially combining these two functions into one function and letting what each does run successively, for this purpose it is permitted to generate a new function that combines what both does, and doing so iterating the list only once, and optimizers are typically clever enough to be able to do so. Suppose that both of these functions be library function black boxes, provided by a dynamic library, then the implementation can no longer do this as it calls both from a library. The way Haskell is written is completely dependent on this; it would be awfully slow if the compiler weren't allowed to do this. The result in practice is that updating a dynamic library written in Haskell requires that all it's consumers be recompiled to compile against the new version. Or otherwise said: there is no concept of a stable a.b.i. in Haskell.
lallysingh 6y ago
Do you mean no dynamic lib support for Haskell code, separately from C libraries that can be dynamically linked?
- Blikkentrekker 6y ago
  There is support for it, but no support for a stable a.b.i., as such with every minor change to the library, nothing linked to it can run without a recompilation. This is due to the non-strict semantics. Consider that in a C library, or any other strict language, the function loaded in the dynamic library is called by first evaluating all of the arguments, and then calling the function by value with the evaluated arguments. This is not how Haskell operates, where the arguments are, in effect, placed inside the function body directly and may, or may not be evaluated, as such a function can't be a black box that can arbitrarily change it's internal workings, so long as the outside a.b.i. remain stable — a program compiled against a dynamic library is fundamentally compiled against the inner workings of functions, rather than simply against the stable interface they proffer to the outside world.

egonschiele 6y ago

That is great news. Haskell toolchain could use some help. I wonder if they'll work on the documentation and error messaging too. I think people are scared off when they realize that some of the libraries have no docs and you just have to read the type signatures.

sfvisser 6y ago
Definitely agree with you that more documentation, especially with examples, would be great. However, reading just the types of a well written Haskell API probably gives me greater inside in its usage than most documentation I’ve read on npm packages. (To name an example I’m familiar with) There are so many very general patterns and idioms in Haskell being used all over the place that reading the signatures in the API docs can be extremely valuable.
- tome 6y ago
  I program roughly equal amounts in Python and Haskell and every time I read Python documentation I wish it were as good as Haskell's. Having type signatures goes a long, long way to understanding an API. Python replaces that with rambling, unclear, prose. The same goes for Python's standard library and popular libraries like flask, pandas and matplotlib. (Haskell's direct links to source are also good, but there's no fundamental reason Python couldn't have those. I've never seen it though.)

pbowyer 6y ago

> It appears that the intent is to focus on pain points in the Haskell toolchain and libraries. Good. I set myself the challenge of compiling a Haskell program [1] during the Christmas holidays. It was meant to be a "one mince pie" challenge, but after an hour I discovered the VM I used didn't have enough RAM (during compilation we were approaching 4GB), then I ran out of disk space as stack approaches 5GB & I had other stuff installed. Once a few hours had gone by (this program isn't fast to compile) I had a working program. I now have to figure out if I can distribute just the resulting binary to other servers, or if it needs other software like GHC installing. Having finished the pack of mince pies, that can wait to another day. I know when I first started compiling C/C++ software there was a learning curve and it took hours the first time, but I found it easier to get started. With Haskell, the way one version of GHC is installed first and then Stack installs a completely isolated version is confusing; plus the inscrutable error messages (haven't got it to hand, but one means OOM but doesn't say that - it takes a Google to find the GitHub issue to work that out). And this is before I try and experiment/decide to learn some Haskell. Apart from the error messages they're not issues with Haskell per se, but they contribute to the experience of it. 1. https://github.com/facebook/duckling https://github.com/facebook/duckling

smadge 6y ago
I upgraded my personal VPS from 1GB to 2GB RAM when I couldn't compile some Haskell standard library modules for a simple project.
mrkeen 6y ago
When I started, I took the command I used for compiling c programs and substituted a letter. gcc -Wall -O3 -o ./main main.c ghc -Wall -O3 -o ./main main.hs I still compile that way every now and then.
- pbowyer 6y ago
  Good to know, thanks. The README says to use stack, so I've stuck with that so far. I'll look for instructions for using ghc