5 ms·

Scala 2.9 vs 2.10 Performance

68 points by bertzzie 14y ago

tikhonj 14y ago
This isn't the most compelling of benchmarks. It's still infinitely better than what most people do--the author actually measured something in a reasonable way!--but I don't think it's enough to make any interesting conclusions. I suppose it shows that there is a performance boost, which is great, but I'd really like to know where the gains really are. Could somebody familiar with the changes to 2.10 explain where the performance boost came from and what code it will affect? Does this make Scala a more compelling choice for the discerning functional programmer?
- gtani 14y ago
  It's incomplete reporting without the JRE heap, inlining, GC and other options used and some kind of residency and median memory usage data. Also, warmups, hardware and OS (Windows seems to frequently produce interesting results. (I haven't seen a GC testbench for java like dons' gc-tune pkg for haskell, but that would be a great tool
jaytaylor 14y ago
Performance improvements are nice and all, but I'm already generally satisfied with the performance of Scala 2.8.x and 2.9.x. What I'd really like to see is faster compile times. I know it's a tall order given the complexity of implicits, but compared to other languages Scala compilation slowness can be a tough sell.
- eropple 14y ago
  Performance improvements are really important given the rise of poor JVMs like Dalvik, though. Scala is really painful on Dalvik due to the amount of garbage it generates.
  - seanmcdirmid 14y ago
    I wouldn't call Dalvik a poor JVM, just a JVM optimized for a certain mobile application mix (not so GC intensive) that Scala doesn't really support yet.
    - eropple 14y ago
      Can you name an X for which Dalvik performs competitively with HotSpot? Honest question, because I can't think of one.
      - laureny 14y ago
        This comparison makes no sense, the two JVM's were designed with fundamentally different goals. At any rate, if you really wanted to compare, you would have to run the benchmarks on the same platform, i.e. Dalvik on the desktop or HotSpot on Android. Which makes no sense either.
        eropple 14y ago
        This comparison makes no sense, the two JVM's were designed with fundamentally different goals. What they are designed for and what they are quite commonly used for are different things. Both are used for running my application code. Serious performance problems on one makes that worse, regardless of whether it is worse-as-designed or not.
      - seanmcdirmid 14y ago
        To get decent GC performance, you need a large space for copying, which doesn't work so well on mobile, so they probably went with a less efficient in-place collector. Its a trade off really: low memory operation or high performance even when a lot of garbage is generated (and memory isn't so expensive)? WinRT has built in support for reference counting now (both in run-time and in its libraries); that's a big deal for mobile! Sure, the collector is still present and you can use it in C#, but...the game is changing quickly. Apple, of course, supports reference counting already, Dalvik does not as there is no easy way to bolt that onto Java (though I wouldn't be surprised if Google eventually does what MSFT did with WinRT). Disclosure: MSFT employee and general programming languages hacker.
- jorgeortiz85 14y ago
  The Scala compiler itself is written in Scala, so improvements in performance will also benefit compile times. Or so I hope.
  - habosa 14y ago
    How is this managed? Is there a "backup compiler" written in some other language to compile the compiler?
    - joshhart 14y ago
      You use a previous version of Scala to compile the trunk version. It's turtles all the way down! Just kidding. The initial Scala compiler was written in Java. After that was working, they rewrote it in Scala using their initial Java compiler, and then it became the process I mentioned.
alexjarvis 14y ago
Nice post, I just used your benchmark to also compare running Scala in JDK 6 and 7 http://news.ycombinator.com/item?id=4862035 http://news.ycombinator.com/item?id=4862035

68 points by bertzzie 14y ago

This isn't the most compelling of benchmarks. It's still infinitely better than what most people do--the author actually measured something in a reasonable way!--but I don't think it's enough to make any interesting conclusions. I suppose it shows that there is a performance boost, which is great, but I'd really like to know where the gains really are. Could somebody familiar with the changes to 2.10 explain where the performance boost came from and what code it will affect? Does this make Scala a more compelling choice for the discerning functional programmer?

gtani 14y ago
It's incomplete reporting without the JRE heap, inlining, GC and other options used and some kind of residency and median memory usage data. Also, warmups, hardware and OS (Windows seems to frequently produce interesting results. (I haven't seen a GC testbench for java like dons' gc-tune pkg for haskell, but that would be a great tool

jaytaylor 14y ago

Performance improvements are nice and all, but I'm already generally satisfied with the performance of Scala 2.8.x and 2.9.x. What I'd really like to see is faster compile times. I know it's a tall order given the complexity of implicits, but compared to other languages Scala compilation slowness can be a tough sell.

eropple 14y ago
Performance improvements are really important given the rise of poor JVMs like Dalvik, though. Scala is really painful on Dalvik due to the amount of garbage it generates.
- seanmcdirmid 14y ago
  I wouldn't call Dalvik a poor JVM, just a JVM optimized for a certain mobile application mix (not so GC intensive) that Scala doesn't really support yet.
  - eropple 14y ago
    Can you name an X for which Dalvik performs competitively with HotSpot? Honest question, because I can't think of one.
    - laureny 14y ago
      This comparison makes no sense, the two JVM's were designed with fundamentally different goals. At any rate, if you really wanted to compare, you would have to run the benchmarks on the same platform, i.e. Dalvik on the desktop or HotSpot on Android. Which makes no sense either.
      - eropple 14y ago
        This comparison makes no sense, the two JVM's were designed with fundamentally different goals. What they are designed for and what they are quite commonly used for are different things. Both are used for running my application code. Serious performance problems on one makes that worse, regardless of whether it is worse-as-designed or not.
    - seanmcdirmid 14y ago
      To get decent GC performance, you need a large space for copying, which doesn't work so well on mobile, so they probably went with a less efficient in-place collector. Its a trade off really: low memory operation or high performance even when a lot of garbage is generated (and memory isn't so expensive)? WinRT has built in support for reference counting now (both in run-time and in its libraries); that's a big deal for mobile! Sure, the collector is still present and you can use it in C#, but...the game is changing quickly. Apple, of course, supports reference counting already, Dalvik does not as there is no easy way to bolt that onto Java (though I wouldn't be surprised if Google eventually does what MSFT did with WinRT). Disclosure: MSFT employee and general programming languages hacker.
jorgeortiz85 14y ago
The Scala compiler itself is written in Scala, so improvements in performance will also benefit compile times. Or so I hope.
- habosa 14y ago
  How is this managed? Is there a "backup compiler" written in some other language to compile the compiler?
  - joshhart 14y ago
    You use a previous version of Scala to compile the trunk version. It's turtles all the way down! Just kidding. The initial Scala compiler was written in Java. After that was working, they rewrote it in Scala using their initial Java compiler, and then it became the process I mentioned.

alexjarvis 14y ago

Nice post, I just used your benchmark to also compare running Scala in JDK 6 and 7 http://news.ycombinator.com/item?id=4862035 http://news.ycombinator.com/item?id=4862035