3 ms·

Take their Haskell implementation. If I compile it unoptimized (with a slight change to hardwire the iteration count at one billion instead of reading it from a

by Hizonner 10mo ago

Take their Haskell implementation. If I compile it unoptimized (with a slight change to hardwire the iteration count at one billion instead of reading it from a file), I get bored waiting for it to finish after several minutes of clock time. If I compile it with "-O3", it runs in 4.75 seconds (it's an old machine). Suppose I remove the strictness annotations (3 exclamation points, in places that aren't obvious to a naive programmer coming from almost any other language). If I then compile it unoptimized, it gets up to over 30GB of resident memory before I get bored (it's an old machine with a lot of memory). It would probably die with an out of memory error if I tried to run it to completion. However, if I compile that same modified code optimized, the compiler infers the strictness and the program runs in exactly the same time as it does with the annotations there. BUT it's far from obvious to the casual observer when the compiler can make those inferences and when it can't. I had ChatGPT rewrite the Haskell code to use unboxed numeric types. It ran in 1.5 seconds (the C version takes 1.27). The rewrite mostly consists of sprinkling "#" liberally throughout the code, but also requires using a few specialized functions. I had ChatGPT do it because I have never used unboxed types, and you could argue that they're not common idiom. However, anybody who actually wrote that kind of numerical code in Haskell on a regular basis would use unboxed types as a matter of course. So which one is the right time?

ajross 10mo ago
> So which one is the right time? You're acting like this is a gotcha, but the answer is obviously "all of them" and that indeed, this tells you interesting things about the behavior of your compiler. There are lots of variant scores in the linked article that reflect different ways of expressing the problem. But also, it tells you something about the limitations of your language too. For example, the biggest single reason that C/C++ (and languages like Fortran/Zig/D and sometimes C# and Rust whose code generation is isomorphic to them) sit at the top of the list is that they autovectorize. SIMD code isn't a common idiom either, but the compiler figures it out anyway. And apparently Haskell isn't capable of doing enough static analysis to fall back to an unboxed implementation (though given the simplicity of this code, that should be possible I think). That's not a "flaw" and it doesn't mean Haskell "loses", but it absolutely is an important thing to note. And charts like this show us where those holes lie.
- Hizonner 10mo ago
  > You're acting like this is a gotcha, but the answer is obviously "all of them" and that indeed, this tells you interesting things about the behavior of your compiler. They tell me interesting things if I know enough about the language to know the difference. It tells me things if I'm getting into the weeds with Haskell specifically. That doesn't make the big comparison chart useful in any way. I still don't know anything that lets me compare anything with any other language unless I actually know that language nearly as well. And I definitely don't get much out of a long list of languages, most of which I know not at all or at most at a "hello world" level, with only a couple of the entries tagged with even minimal information about compilers or their configurations at all. Especially when, on top of that, I don't know how much the person writing the test code knew. At most I get "this language does a pretty good/poor job on this type of task when given code that may or may not be what a 'native expert' would write.". And that's not news. Nobody (with any sophistication) would write that code for real in Python, or probably in Haskell either, because most seasoned programmers know that if you want speed on a task like that, you write it in a more traditional compiled procedural language. It's also not a kind of code that most people write to begin with. If you want an arctangent (which is really what it's doing), you use the library function, and the underlying implementation of that is either handcrafted C, or, more likely, a single, crafted CPU instruction with some call handling code wrapped around it. So what is the overall chart giving me that I can use?
  - ajross 10mo ago
    > So what is the overall chart giving me that I can use? "If you write in C or an analog, your math will autovectorize nicely" "If you use a runtime with a managed heap, you're likely to take a penalty even on math stuff that doesn't look heap limited" "rust 1.92 is, surprisingly, well behind clang on autovectorizable code" I mean, I think that stuff is interesting. > If you want an arctangent (which is really what it's doing), you use the library function If you just want to call library functions, you're 100% at the mercy of whatever platform you picked[1]. So sure, don't look at benchmarks, they can only prove you wrong. [1] Picked without, I'll note, having looked carefully at cross-platform benchmarks before having made the choice! Because you inexplicably don't think they tell you anything useful.

ajross 10mo ago

> So which one is the right time? You're acting like this is a gotcha, but the answer is obviously "all of them" and that indeed, this tells you interesting things about the behavior of your compiler. There are lots of variant scores in the linked article that reflect different ways of expressing the problem. But also, it tells you something about the limitations of your language too. For example, the biggest single reason that C/C++ (and languages like Fortran/Zig/D and sometimes C# and Rust whose code generation is isomorphic to them) sit at the top of the list is that they autovectorize. SIMD code isn't a common idiom either, but the compiler figures it out anyway. And apparently Haskell isn't capable of doing enough static analysis to fall back to an unboxed implementation (though given the simplicity of this code, that should be possible I think). That's not a "flaw" and it doesn't mean Haskell "loses", but it absolutely is an important thing to note. And charts like this show us where those holes lie.

Hizonner 10mo ago
> You're acting like this is a gotcha, but the answer is obviously "all of them" and that indeed, this tells you interesting things about the behavior of your compiler. They tell me interesting things if I know enough about the language to know the difference. It tells me things if I'm getting into the weeds with Haskell specifically. That doesn't make the big comparison chart useful in any way. I still don't know anything that lets me compare anything with any other language unless I actually know that language nearly as well. And I definitely don't get much out of a long list of languages, most of which I know not at all or at most at a "hello world" level, with only a couple of the entries tagged with even minimal information about compilers or their configurations at all. Especially when, on top of that, I don't know how much the person writing the test code knew. At most I get "this language does a pretty good/poor job on this type of task when given code that may or may not be what a 'native expert' would write.". And that's not news. Nobody (with any sophistication) would write that code for real in Python, or probably in Haskell either, because most seasoned programmers know that if you want speed on a task like that, you write it in a more traditional compiled procedural language. It's also not a kind of code that most people write to begin with. If you want an arctangent (which is really what it's doing), you use the library function, and the underlying implementation of that is either handcrafted C, or, more likely, a single, crafted CPU instruction with some call handling code wrapped around it. So what is the overall chart giving me that I can use?
- ajross 10mo ago
  > So what is the overall chart giving me that I can use? "If you write in C or an analog, your math will autovectorize nicely" "If you use a runtime with a managed heap, you're likely to take a penalty even on math stuff that doesn't look heap limited" "rust 1.92 is, surprisingly, well behind clang on autovectorizable code" I mean, I think that stuff is interesting. > If you want an arctangent (which is really what it's doing), you use the library function If you just want to call library functions, you're 100% at the mercy of whatever platform you picked[1]. So sure, don't look at benchmarks, they can only prove you wrong. [1] Picked without, I'll note, having looked carefully at cross-platform benchmarks before having made the choice! Because you inexplicably don't think they tell you anything useful.