3 ms·

Can nobody on planet earth read? That Google paper specifically said that they significantly refactored the algorithm to be more "scala-like" and got a much ch

by jbooth 15y ago

Can nobody on planet earth read? That Google paper specifically said that they significantly refactored the algorithm to be more "scala-like" and got a much cheaper implementation than the canonical way it was implemented in other languages. So, no, Scala is not faster than Java and probably won't be for a long time. But yes, picking your algorithm is more important than language microbenchmarks.

loup-vaillant 15y ago
Your phrasing is slanted. Citing the abstract of the paper: > The implementations each use the languages’ idiomatic container classes, looping constructs, and memory/object allocation schemes. It does not attempt to exploit specific language and run-time features to achieve maximum performance. Meaning, they went out of their way to write idiomatic Scala and idiomatic Java. No performance tweak in Java nor in Scala. The way you phrased it make it sound like they paid more attention to their Scala implementation. If you really think that, say so. Beware though: that would be a direct accusation of intellectual dishonesty. To me, Google's paper looks like clear (though not iron strong) evidence that idiomatic Scala is is faster than idiomatic Java, with the chosen implementations. Shortening that into "Scala is currently faster than Java" doesn't seem such a stretch. Now if you want to compare idiomatic Java written in Java, and idiomatic Java written in Scala, fine, but I don't think it's a fair comparison.
- scott_s 15y ago
  Frankly, their discussion is a bit confusing because they did have optimized versions discussed in the text. And the optimized Scala version did make fundamental changes to the algorithm. They could have kept a better division between the two - idiomatic versus optimized - in both their results and their discussion. So, jbooth is not accusing them of intellectual dishonesty. And if I consider the following three suppositions: jbooth's understanding of the paper was not complete, your understanding of the paper was not complete, and the people who wrote the paper are smart and hard working; then I simply conclude that performing and writing about this kind of study in a clear manner that will confuse none is very, very hard.
  - loup-vaillant 15y ago
    > your understanding of the paper was not complete, Indeed: I have not read it. But now I plan to. (Edit: since when a pubic recognition of one's own fault is worth a down vote? My not having read the paper provides meaningful context, by temporarily undermining my authority. Down vote the grand parent, but leave this one at 1 point!)
    - scott_s 15y ago
      I agree with your edit, and I tried to correct for it.
- jbooth 15y ago
  Read their section specifically on Scala. The algorithm as implemented in Scala was a different bag of oranges than the apples in the C/Java implementations. If you want to say that idiomatic Scala is so great, fine, although I wouldn't be so absolutist about it (remember the article on HN 2 days ago about the scala for loop being 20x as expensive as the while loop because of something to do with closures?). Anyways the point is that the comparison was not between two things that were at all the same. All that says is that this particular problem was better expressed in Scala, and nothing about how one was faster than the other.
  - loup-vaillant 15y ago
    Looks like I should update. http://wiki.lesswrong.com/wiki/How_To_Actually_Change_Your_Mind http://wiki.lesswrong.com/wiki/How_To_Actually_Change_Your_M... Give me some time. http://news.ycombinator.com/item?id=2678994 http://news.ycombinator.com/item?id=2678994
  - scott_s 15y ago
    And you should probably take a closer look at Figure 8 - note that the non-pro version of Scala that did not change the fundamental algorithm is still faster than the non-pro version of Java. The version you're mentioning is not in Figure 8. Again, I agree the results are confusing.
dkarl 15y ago
Too lazy to read past the first page before breakfast, but: All four implementations stay very close to the formal specification of the algorithm and do not attempt any form of language specific optimization or adaption. So there appears to be some inconsistency in how they describe their methods.
speckledjim 15y ago
> So, no, Scala is not faster than Java and probably won't be for a long time. Maybe I'm missing something obvious here, but how can a language that compiles down to Java, be faster than Java? Does C code execute faster than machine code?
- kscaldef 15y ago
  Scala compiles down to JVM bytecode. Java also compiles down to JVM bytecode. Scala does not compile down to Java.
scott_s 15y ago
The "pro" versions - the implementations that were tuned for performance by experts in that language - were not part of the main figure that compared runtimes (the table in Figure 8). All of the optimizations and reports of performance improvements were in the "Tunings" section, and were considered apart from the findings in Figure 8. Well, except for the Go Pro version, which blurs the division. They could have used a sentence in the opening of the "Tunings" section to reiterate that the optimized versions are not represented in Figure 8. The different versions in Figure 8 are mostly tuning the GC, not tuning the implementation itself. And, strangely, they do include the Go Pro version in Figure 8. Certainly, though, the C++, Java and Scala Pro versions are not in Figure 8, which is where they drew most of their conclusions from.

by jbooth 15y ago

loup-vaillant 15y ago

Your phrasing is slanted. Citing the abstract of the paper: > The implementations each use the languages’ idiomatic container classes, looping constructs, and memory/object allocation schemes. It does not attempt to exploit specific language and run-time features to achieve maximum performance. Meaning, they went out of their way to write idiomatic Scala and idiomatic Java. No performance tweak in Java nor in Scala. The way you phrased it make it sound like they paid more attention to their Scala implementation. If you really think that, say so. Beware though: that would be a direct accusation of intellectual dishonesty. To me, Google's paper looks like clear (though not iron strong) evidence that idiomatic Scala is is faster than idiomatic Java, with the chosen implementations. Shortening that into "Scala is currently faster than Java" doesn't seem such a stretch. Now if you want to compare idiomatic Java written in Java, and idiomatic Java written in Scala, fine, but I don't think it's a fair comparison.

scott_s 15y ago
Frankly, their discussion is a bit confusing because they did have optimized versions discussed in the text. And the optimized Scala version did make fundamental changes to the algorithm. They could have kept a better division between the two - idiomatic versus optimized - in both their results and their discussion. So, jbooth is not accusing them of intellectual dishonesty. And if I consider the following three suppositions: jbooth's understanding of the paper was not complete, your understanding of the paper was not complete, and the people who wrote the paper are smart and hard working; then I simply conclude that performing and writing about this kind of study in a clear manner that will confuse none is very, very hard.
- loup-vaillant 15y ago
  > your understanding of the paper was not complete, Indeed: I have not read it. But now I plan to. (Edit: since when a pubic recognition of one's own fault is worth a down vote? My not having read the paper provides meaningful context, by temporarily undermining my authority. Down vote the grand parent, but leave this one at 1 point!)
  - scott_s 15y ago
    I agree with your edit, and I tried to correct for it.
jbooth 15y ago
Read their section specifically on Scala. The algorithm as implemented in Scala was a different bag of oranges than the apples in the C/Java implementations. If you want to say that idiomatic Scala is so great, fine, although I wouldn't be so absolutist about it (remember the article on HN 2 days ago about the scala for loop being 20x as expensive as the while loop because of something to do with closures?). Anyways the point is that the comparison was not between two things that were at all the same. All that says is that this particular problem was better expressed in Scala, and nothing about how one was faster than the other.
- loup-vaillant 15y ago
  Looks like I should update. http://wiki.lesswrong.com/wiki/How_To_Actually_Change_Your_Mind http://wiki.lesswrong.com/wiki/How_To_Actually_Change_Your_M... Give me some time. http://news.ycombinator.com/item?id=2678994 http://news.ycombinator.com/item?id=2678994
- scott_s 15y ago
  And you should probably take a closer look at Figure 8 - note that the non-pro version of Scala that did not change the fundamental algorithm is still faster than the non-pro version of Java. The version you're mentioning is not in Figure 8. Again, I agree the results are confusing.

dkarl 15y ago

Too lazy to read past the first page before breakfast, but: All four implementations stay very close to the formal specification of the algorithm and do not attempt any form of language specific optimization or adaption. So there appears to be some inconsistency in how they describe their methods.

speckledjim 15y ago

> So, no, Scala is not faster than Java and probably won't be for a long time. Maybe I'm missing something obvious here, but how can a language that compiles down to Java, be faster than Java? Does C code execute faster than machine code?

kscaldef 15y ago
Scala compiles down to JVM bytecode. Java also compiles down to JVM bytecode. Scala does not compile down to Java.

scott_s 15y ago

The "pro" versions - the implementations that were tuned for performance by experts in that language - were not part of the main figure that compared runtimes (the table in Figure 8). All of the optimizations and reports of performance improvements were in the "Tunings" section, and were considered apart from the findings in Figure 8. Well, except for the Go Pro version, which blurs the division. They could have used a sentence in the opening of the "Tunings" section to reiterate that the optimized versions are not represented in Figure 8. The different versions in Figure 8 are mostly tuning the GC, not tuning the implementation itself. And, strangely, they do include the Go Pro version in Figure 8. Certainly, though, the C++, Java and Scala Pro versions are not in Figure 8, which is where they drew most of their conclusions from.