7 ms·

Having seen hundreds of these types of blog posts, touting faster-than-c (superluminal?) benchmarks in arbitrary languages, I'm extremely skeptical. Real-world

by duaneb 15y ago

Having seen hundreds of these types of blog posts, touting faster-than-c (superluminal?) benchmarks in arbitrary languages, I'm extremely skeptical. Real-world applications are rarely solely limited by small chunks of code that are simple enough to be optimized independently of the rest of the program. Accurate comparisons should be done on large, complex code bases that mirror an equivalent C program. Unfortunately, those don't exist, so in my mind the true performance potential of haskell is still unknown. I do have high hopes for the language, especially since whole-program optimization and aggressive inlining/code folding should yield very, very efficient code, but as of yet the only large programs in haskell remain GHC and darcs, and darcs is extremely slow. Still: a single benchmark showing a good result is better than one showing a bad result.

tel 15y ago
It's just part of the prescription though. The idea is to use GHC profiling code to find the real bottlenecks and then use techniques like stream fusion to cause those inner loops to compile to near C-optimal assembly. That said, it's a tough process.
Periodic 15y ago
The benchmark is pretty simple and artificial. I think what we should be looking at is that the performance can asymptotically approach C, where C is used because it's the closest to the theoretical optimum assembly that is still practical for writing large programs. If a language can get arbitrarily close to C in performance, it starts to become a much better design trade-off to use.
- Drbble 15y ago
  One of the touted benefits of pure strongly typed code in languages like Haskell or OCaml is that they can compile to assembler that is faster than C, because the compiler can guarantee certain behaviors and thereby increase sharing of computations and insert multicore parallelism .
sigil 15y ago
> Real-world applications are rarely solely limited by small chunks of code that are simple enough to be optimized independently of the rest of the program. Two reactions to this: 1. If they aren't, maybe they should be. Not sure the large applications of the past are a good guide to the future. Perhaps a large pile of functional code is a clear software engineering win for some value of "large," and we're still discovering this. 2. In my experience, large applications are often performance-limited by small chunks of code. The results of actually profiling never cease to amaze me -- my guesses are so often wrong. Of course, once you fix the bottleneck in one spot, the bottleneck moves to another, but hey that's the game of optimization in any language.
joeyh 15y ago
At 15 thousand lines, git-annex is only 5 thousand lines less of haskell than darcs. I happen to know, since I wrote git-annex. I'm not sure where you're coming from with your statement about there only being two large haskell programs. For that matter, I don't know if I'd consider darc's 20 kloc very large. Or that I'd consider another haskell program I wrote, github-backup, to be small -- that 2 kloc program sits at the apex of a lot of libraries, and probably combined they have more lines of code than darcs. Your whole premise about lines of code feels thoroughly flawed to me. Anyway, As a git extension, git-annex is expected to run quite fast. I've never had any difficulty, in writing git-annex, with the speed of haskell code. I'm sure darcs is slow due to its patch theory thing, not due to its implementation language.
- duaneb 15y ago
  > I'm not sure where you're coming from with your statement about there only being two large haskell programs. I concede the point, but I'd rather say that there's only one large haskell program. Let's consider large C/C++ programs: 1. Chromium clocks in around 4 million lines of code. 2. LLVM/clang together has a (fuzzy) estimate of ~1165539 LOC on my system. 3. GHC has unknown lines of code, and I am unsure as how to count it, whether to include the standard library, etc, but it has at least 100k, so we'll just call that sufficiently large. Now, I know haskell can be fast—I've spent a lot of time myself hand-tuning ghc processed code. But I've noticed that unless I can trace the bottleneck to a single function or small set of functions, optimization to near-c levels is extremely difficult. Now, even 2-3x slower than c—far slower than ghc-generated code—is still very fast, so I'm not calling haskell slow. Far from it.
  - joeyh 15y ago
    Actually, there are packages in hackage with > 100 kloc: CHXHtml KiCS-debugger HaRe
maxcan 15y ago
A big reason for this is that a big chunk of Haskell adoption is in software you will never know about: financial risk systems for major investment banks (one of which has over 400 KLoC in Haskell), crypto work for the NSA, backends powering parts of large sites (facebook, bump, etc) and the full stack on some smaller ones (planely, yesodweb, and quite a few others I can't mention).
dons 15y ago
There are several million lines of code in Haskell on Hackage now - so any non-trivial Hackage-based app is going to pull in 100k+ lines. The current project I'm working on commercially has 600k lines of Haskell, though I expect we'll hit the 1M mark in the next year or two. It benefits massively from the compiler being able to see the forest from the trees. From my point of view, this stuff was all solved years ago. There's nothing much left to prove -- so get on with building great things in Haskell.

by duaneb 15y ago

tel 15y ago

It's just part of the prescription though. The idea is to use GHC profiling code to find the real bottlenecks and then use techniques like stream fusion to cause those inner loops to compile to near C-optimal assembly. That said, it's a tough process.

Periodic 15y ago

The benchmark is pretty simple and artificial. I think what we should be looking at is that the performance can asymptotically approach C, where C is used because it's the closest to the theoretical optimum assembly that is still practical for writing large programs. If a language can get arbitrarily close to C in performance, it starts to become a much better design trade-off to use.

Drbble 15y ago
One of the touted benefits of pure strongly typed code in languages like Haskell or OCaml is that they can compile to assembler that is faster than C, because the compiler can guarantee certain behaviors and thereby increase sharing of computations and insert multicore parallelism .

sigil 15y ago

> Real-world applications are rarely solely limited by small chunks of code that are simple enough to be optimized independently of the rest of the program. Two reactions to this: 1. If they aren't, maybe they should be. Not sure the large applications of the past are a good guide to the future. Perhaps a large pile of functional code is a clear software engineering win for some value of "large," and we're still discovering this. 2. In my experience, large applications are often performance-limited by small chunks of code. The results of actually profiling never cease to amaze me -- my guesses are so often wrong. Of course, once you fix the bottleneck in one spot, the bottleneck moves to another, but hey that's the game of optimization in any language.

joeyh 15y ago

At 15 thousand lines, git-annex is only 5 thousand lines less of haskell than darcs. I happen to know, since I wrote git-annex. I'm not sure where you're coming from with your statement about there only being two large haskell programs. For that matter, I don't know if I'd consider darc's 20 kloc very large. Or that I'd consider another haskell program I wrote, github-backup, to be small -- that 2 kloc program sits at the apex of a lot of libraries, and probably combined they have more lines of code than darcs. Your whole premise about lines of code feels thoroughly flawed to me. Anyway, As a git extension, git-annex is expected to run quite fast. I've never had any difficulty, in writing git-annex, with the speed of haskell code. I'm sure darcs is slow due to its patch theory thing, not due to its implementation language.

duaneb 15y ago
> I'm not sure where you're coming from with your statement about there only being two large haskell programs. I concede the point, but I'd rather say that there's only one large haskell program. Let's consider large C/C++ programs: 1. Chromium clocks in around 4 million lines of code. 2. LLVM/clang together has a (fuzzy) estimate of ~1165539 LOC on my system. 3. GHC has unknown lines of code, and I am unsure as how to count it, whether to include the standard library, etc, but it has at least 100k, so we'll just call that sufficiently large. Now, I know haskell can be fast—I've spent a lot of time myself hand-tuning ghc processed code. But I've noticed that unless I can trace the bottleneck to a single function or small set of functions, optimization to near-c levels is extremely difficult. Now, even 2-3x slower than c—far slower than ghc-generated code—is still very fast, so I'm not calling haskell slow. Far from it.
- joeyh 15y ago
  Actually, there are packages in hackage with > 100 kloc: CHXHtml KiCS-debugger HaRe

maxcan 15y ago

A big reason for this is that a big chunk of Haskell adoption is in software you will never know about: financial risk systems for major investment banks (one of which has over 400 KLoC in Haskell), crypto work for the NSA, backends powering parts of large sites (facebook, bump, etc) and the full stack on some smaller ones (planely, yesodweb, and quite a few others I can't mention).

dons 15y ago

There are several million lines of code in Haskell on Hackage now - so any non-trivial Hackage-based app is going to pull in 100k+ lines. The current project I'm working on commercially has 600k lines of Haskell, though I expect we'll hit the 1M mark in the next year or two. It benefits massively from the compiler being able to see the forest from the trees. From my point of view, this stuff was all solved years ago. There's nothing much left to prove -- so get on with building great things in Haskell.