7 ms·

Java vs. Go performance

55 points by obituary_latte 12y ago

obituary_latte 12y ago
There are also many options available to compare different languages. Very cool tool.
- igouy 12y ago
  Also you can dive-into the source code and see the different ways people have written programs in the same language -- http://benchmarksgame.alioth.debian.org/u64q/performance.php?test=fannkuchredux http://benchmarksgame.alioth.debian.org/u64q/performance.php... Also for an overview -- http://benchmarksgame.alioth.debian.org/u64/which-programs-are-fastest.html http://benchmarksgame.alioth.debian.org/u64/which-programs-a... Also for a different overview -- http://benchmarksgame.alioth.debian.org/u64/code-used-time-used-shapes.php http://benchmarksgame.alioth.debian.org/u64/code-used-time-u...
  - obituary_latte 12y ago
    That last link...very very cool. Much more interesting than the Java vs go link I posted. You should submit it and spark a discussion :)
    - igouy 12y ago
      By my guest. It's probably been submitted several times over the years, but there are always people who haven't seen the benchmarks game at all, or have only seen the direct comparisons. The interesting unique detail is "Shortest C++".
jackyb 12y ago
I wonder if there will be any improvement if the Go code is compiled with GCC.
InTheArena 12y ago
This is a pretty eye opening chart. I know the Go guys have a philosophical point about wanting to be able to reason the whole source code as the justification for the Plan 9 and later GCC only stance, but I do have to think that these numbers are poor enough that a LLVM port should be investigated. I also think that it's a good think for Go that Apple hasn't open source'd swift. I suspect it would thrash Go here.
- beering 12y ago
  Maybe I come from a different world, but these numbers seem pretty good. If you compare C++ and Go, C++ is 3x-1x faster than Go using more code. For most applications that you'd expect to use Go for, that's decent efficiency at a smaller memory footprint than Java. Enough, certainly, to make a high-performance server that uses less RAM than Java and less code than C++.
  - nulltype 12y ago
    Go seems to be heavily optimized for write-time efficiency instead of runtime efficiency.
- kristianp 12y ago
  There is only one where Go is 3 times slower, regex-dna. That would either be because of the speed of the regex implementation or the GC for the allocations. Apart from that Go is similar or twice as fast for reverse-component. Note that Go uses less memory for tasks that require memory to be allocated too. Go is doing better at the benchmarks game than it has in the past.
- bad_user 12y ago
  I don't get it - is this because you think Java has poor performance? I for one suspect that Swift does not exceed Java in throughput either and in fact I'd bet that the performance of Swift is worse.
bcg1 12y ago
Look, Java is my right hand language, and I've never written a single line of Go in my life. But this is kind of unfair I think... the OpenJDK JVM has 20 years of development behind it, and has probably had mountains of money sunk into tuning f* out of it by the biggest most enterprisy players in the game. I'm sure that there is a lot of low hanging fruit for Go to grab at to improve performance, and most likely will. I can't stand benchmarks like this. Oh yeah, and Java FTW ;)
- melling 12y ago
  My understanding is that Go has a very simple optimizer. They were waiting for all the C code in the compiler to be translated to Go, which I believe was done in the last release.
  - torvix 12y ago
    not an expert, but i think you are talking about Go's garbage collection, which i don't think is an optimizer.
    - pkroll 12y ago
      Some improvement to the garbage collector was done in the last release, and involved moving much of the runtime to Go. Making the compiler into Go, however, is actually only completed in the 1.5 (development) branch, to be released in... August, I think. That'll open up the work on the optimizer (and everything else in the compiler) to be done in Go.
- ori_b 12y ago
  As someone who has a fairly strong distaste for Java: It's not unfair at all. If I want to make the most of my CPU, and Java runs faster than Go, then that's the way it is. There may be historical reasons for this. There may be low hanging fruit for Go. But in the end, all that matters to the consumer of my project is "code go fast?".
- deleted 12y ago
  [deleted]
karmakaze 12y ago
Interesting that the 32-bit speed is skewed more in Java's favor. I suppose the JVM was first 32-bit optimized where Go needn't be.
- igouy 12y ago
  iirc The programs compiled with Go 8g for x86 have always shown slower than those compiled with Go 6g for x64.
hanief 12y ago
Interesting. All this time I have a perception that Java app is slow as hell. What a terribly wrong assumption. Color me impressed.
- threeseed 12y ago
  That comes from the fact that Java takes quite a while to startup. It is going to be hopefully fixed in Java 9 where the platform becomes more modular. But that issue is what caused Java to have such a terrible reputation when apps were run on the desktop.
  - hugi 12y ago
    Probably part of it, but for me, the somewhat uncanny valley-ish non-standard look and behaviour of Swing UIs (and AWT before that) is the biggest turn off. Using java apps on the desktop feels a little like using a keyboard with mittens.
    - saiya-jin 12y ago
      This is an issue of unskilled devs using ancient UI design approach, usually not even bothering with skinning it. Things like bitTorrent were done in Java (correct me if I'm wrong), and you have no clue which language they are done in when looking at UI. But this a bit of extra effort, and for intranet apps, nobody usually bothers. Another reason for oh-not-so-windows-look of UI is something microsoft finally begun discovering - that there are other platforms than Windows. Java UI works anywhere where you have working JRE, and if you've ever seen Linux's KDE/GNOME/XFCE etc. then you might realize it ain't easy to come up with a cross-section between them. You need to hack things with Wine or similar to have it +-++ work the other way around (ie MFC app -> anywhere but MS world, and even there it might be tricky with all dll hell).
      - lmm 12y ago
        Standard bittorrent was in Python. Azureus was in Java and it felt significantly slower (than Python!) - it took so long to start it needed a splash screen, and you'd click a menu and wait for it to appear. Kind of an unfair comparison - the Python one just calls native Qt or Tk whereas the Java UI is actually written in Java - but it does colour the perception.
chvid 12y ago
I find these benchmarks slightly dubious because everytime I look into the actual code I find that is written some obscure manner. For example - here is the code of the Java "knucleotide" benchmark: http://benchmarksgame.alioth.debian.org/u64q/program.php?test=knucleotide&lang=java&id=7 http://benchmarksgame.alioth.debian.org/u64q/program.php?tes... (If I understand the website correctly.) Look at the way data is read (scanPastHeader, readRestIntoByteArray, fullSequence). Or how the result of calculateHash is put into a HashMap where it immediately would be hashed again. I think this is neither the most natural or performant way of solving the problem in Java.
- rockdoe 12y ago
  So submit an improved version?
- igouy 12y ago
  Currently 7 Java k-nucleotide programs are shown. > the most natural or performant way of solving the problem in Java "How to contribute programs" http://benchmarksgame.alioth.debian.org/play.html#contribute http://benchmarksgame.alioth.debian.org/play.html#contribute
unoti 12y ago
Go has a significantly smaller memory usage in all of the benchmarks, often by a factor of 10x. For most workloads, this can have a huge impact on when you need to spend more money to support your business.
- Skinney 12y ago
  It also has a large impact on how often the GC is run. Less memory used, less GC performed.
  - sacado2 12y ago
    It also produces less garbage since, more often than not, small values are copied, not created on the heap then referenced.
- realusername 12y ago
  I was thinking the same thing. It depends of your needs, if you need raw performance and you have at your disposition a lot of RAM on a powerful server , Java might be a good bet. If you are building things for a normal computer, a Raspberry PI or a background service, the memory usage of Java is going to hit you badly.
- torvix 12y ago
  well, you don't need to run a virtual machine, so...
- igouy 12y ago
  "Huge differences in default memory allocation don't necessarily mean there'll be huge differences in Memory-used for tasks that require memory to be allocated." http://benchmarksgame.alioth.debian.org/play.html#whymemory http://benchmarksgame.alioth.debian.org/play.html#whymemory
craftit 12y ago
As I understand it, Go also give some guarantees about the maximum amount of time the garbage collection will stall the program. An issue where ever you are trying to achieve consistent performance. As I understand it most of the java benchmarks postpone clearing up memory until it can be done trivially at exit, something you just can't do in a whole class of programs.
- Skinney 12y ago
  Not yet, but that is planned for the next release.
logicchains 12y ago
Since we're posting benchmarks, here's Java vs Haskell: http://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=ghc&lang2=java http://benchmarksgame.alioth.debian.org/u64q/compare.php?lan.... The Haskell implementations generally use much less memory than the Java ones and perform competitively. In spite of criticism that some of the Haskell implementations are not idiomatic, they're no more verbose than the Java ones in terms of LOC.
nextw33k 12y ago
This benchmark is out of date almost immediately because of the patch level of the Java runtime and the Golang compilers change frequently. I was half thinking of creating a VB6 module for LLVM just as a tongue in cheek joke of taking a language with a slow runtime and effectively strapping a V8 engine to it, but I saw someone already did most of the work: https://github.com/microcai/llvm-qbasic https://github.com/microcai/llvm-qbasic
- rockdoe 12y ago
  For that to be an argument two things would be required: 1) Changing patchlevels causing huge performance differences. 2) Those "patchlevels" being immediately deployable to production in the real world. Neither is true, so you don't have an argument.
- igouy 12y ago
  > This benchmark is out of date almost immediately Why do you think that matters?
mmf 12y ago
The way I read the results Java is basically faster only on the regex benchmark and it is known fact that today go has a slow regex library. Everything else the results show Java to be slower with larger code and larger memory footprint. Why are all the Java fans psyched by these results?
- igouy 12y ago
  > faster only on the regex benchmark fwiw Also binary-trees, pidigits, n-body. http://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=java&lang2=go#faster-programs-measurements http://benchmarksgame.alioth.debian.org/u64q/compare.php?lan...
TheLoneWolfling 12y ago
Now if only the benchmarks game would actually support, say, PyPy. Or LuaJIT. But nope, one implementation per language, and "language" defined rather arbitrarily at that.
- igouy 12y ago
  Please take the program source code and the measurement scripts and publish your own measurements. afaict we all feel the same way about this, we all feel that we should sit on our hands and wait for someone else to do the chores we don't wish to do. http://benchmarksgame.alioth.debian.org/play.html#languagex http://benchmarksgame.alioth.debian.org/play.html#languagex
  - TheLoneWolfling 12y ago
    I would, if I had access to a machine that was somewhat stable w.r.t. performance. Alas, my only machine currently is my laptop, and it overheats often enough on stress-tests that it's... less than accurate, shall I say?

obituary_latte 12y ago

There are also many options available to compare different languages. Very cool tool.

igouy 12y ago
Also you can dive-into the source code and see the different ways people have written programs in the same language -- http://benchmarksgame.alioth.debian.org/u64q/performance.php?test=fannkuchredux http://benchmarksgame.alioth.debian.org/u64q/performance.php... Also for an overview -- http://benchmarksgame.alioth.debian.org/u64/which-programs-are-fastest.html http://benchmarksgame.alioth.debian.org/u64/which-programs-a... Also for a different overview -- http://benchmarksgame.alioth.debian.org/u64/code-used-time-used-shapes.php http://benchmarksgame.alioth.debian.org/u64/code-used-time-u...
- obituary_latte 12y ago
  That last link...very very cool. Much more interesting than the Java vs go link I posted. You should submit it and spark a discussion :)
  - igouy 12y ago
    By my guest. It's probably been submitted several times over the years, but there are always people who haven't seen the benchmarks game at all, or have only seen the direct comparisons. The interesting unique detail is "Shortest C++".

jackyb 12y ago

I wonder if there will be any improvement if the Go code is compiled with GCC.

InTheArena 12y ago

This is a pretty eye opening chart. I know the Go guys have a philosophical point about wanting to be able to reason the whole source code as the justification for the Plan 9 and later GCC only stance, but I do have to think that these numbers are poor enough that a LLVM port should be investigated. I also think that it's a good think for Go that Apple hasn't open source'd swift. I suspect it would thrash Go here.

beering 12y ago
Maybe I come from a different world, but these numbers seem pretty good. If you compare C++ and Go, C++ is 3x-1x faster than Go using more code. For most applications that you'd expect to use Go for, that's decent efficiency at a smaller memory footprint than Java. Enough, certainly, to make a high-performance server that uses less RAM than Java and less code than C++.
- nulltype 12y ago
  Go seems to be heavily optimized for write-time efficiency instead of runtime efficiency.
kristianp 12y ago
There is only one where Go is 3 times slower, regex-dna. That would either be because of the speed of the regex implementation or the GC for the allocations. Apart from that Go is similar or twice as fast for reverse-component. Note that Go uses less memory for tasks that require memory to be allocated too. Go is doing better at the benchmarks game than it has in the past.
bad_user 12y ago
I don't get it - is this because you think Java has poor performance? I for one suspect that Swift does not exceed Java in throughput either and in fact I'd bet that the performance of Swift is worse.

bcg1 12y ago

Look, Java is my right hand language, and I've never written a single line of Go in my life. But this is kind of unfair I think... the OpenJDK JVM has 20 years of development behind it, and has probably had mountains of money sunk into tuning f* out of it by the biggest most enterprisy players in the game. I'm sure that there is a lot of low hanging fruit for Go to grab at to improve performance, and most likely will. I can't stand benchmarks like this. Oh yeah, and Java FTW ;)

melling 12y ago
My understanding is that Go has a very simple optimizer. They were waiting for all the C code in the compiler to be translated to Go, which I believe was done in the last release.
- torvix 12y ago
  not an expert, but i think you are talking about Go's garbage collection, which i don't think is an optimizer.
  - pkroll 12y ago
    Some improvement to the garbage collector was done in the last release, and involved moving much of the runtime to Go. Making the compiler into Go, however, is actually only completed in the 1.5 (development) branch, to be released in... August, I think. That'll open up the work on the optimizer (and everything else in the compiler) to be done in Go.
ori_b 12y ago
As someone who has a fairly strong distaste for Java: It's not unfair at all. If I want to make the most of my CPU, and Java runs faster than Go, then that's the way it is. There may be historical reasons for this. There may be low hanging fruit for Go. But in the end, all that matters to the consumer of my project is "code go fast?".
deleted 12y ago
[deleted]

karmakaze 12y ago

Interesting that the 32-bit speed is skewed more in Java's favor. I suppose the JVM was first 32-bit optimized where Go needn't be.

igouy 12y ago
iirc The programs compiled with Go 8g for x86 have always shown slower than those compiled with Go 6g for x64.

hanief 12y ago

Interesting. All this time I have a perception that Java app is slow as hell. What a terribly wrong assumption. Color me impressed.

threeseed 12y ago
That comes from the fact that Java takes quite a while to startup. It is going to be hopefully fixed in Java 9 where the platform becomes more modular. But that issue is what caused Java to have such a terrible reputation when apps were run on the desktop.
- hugi 12y ago
  Probably part of it, but for me, the somewhat uncanny valley-ish non-standard look and behaviour of Swing UIs (and AWT before that) is the biggest turn off. Using java apps on the desktop feels a little like using a keyboard with mittens.
  - saiya-jin 12y ago
    This is an issue of unskilled devs using ancient UI design approach, usually not even bothering with skinning it. Things like bitTorrent were done in Java (correct me if I'm wrong), and you have no clue which language they are done in when looking at UI. But this a bit of extra effort, and for intranet apps, nobody usually bothers. Another reason for oh-not-so-windows-look of UI is something microsoft finally begun discovering - that there are other platforms than Windows. Java UI works anywhere where you have working JRE, and if you've ever seen Linux's KDE/GNOME/XFCE etc. then you might realize it ain't easy to come up with a cross-section between them. You need to hack things with Wine or similar to have it +-++ work the other way around (ie MFC app -> anywhere but MS world, and even there it might be tricky with all dll hell).
    - lmm 12y ago
      Standard bittorrent was in Python. Azureus was in Java and it felt significantly slower (than Python!) - it took so long to start it needed a splash screen, and you'd click a menu and wait for it to appear. Kind of an unfair comparison - the Python one just calls native Qt or Tk whereas the Java UI is actually written in Java - but it does colour the perception.

chvid 12y ago

I find these benchmarks slightly dubious because everytime I look into the actual code I find that is written some obscure manner. For example - here is the code of the Java "knucleotide" benchmark: http://benchmarksgame.alioth.debian.org/u64q/program.php?test=knucleotide&lang=java&id=7 http://benchmarksgame.alioth.debian.org/u64q/program.php?tes... (If I understand the website correctly.) Look at the way data is read (scanPastHeader, readRestIntoByteArray, fullSequence). Or how the result of calculateHash is put into a HashMap where it immediately would be hashed again. I think this is neither the most natural or performant way of solving the problem in Java.

rockdoe 12y ago
So submit an improved version?
igouy 12y ago
Currently 7 Java k-nucleotide programs are shown. > the most natural or performant way of solving the problem in Java "How to contribute programs" http://benchmarksgame.alioth.debian.org/play.html#contribute http://benchmarksgame.alioth.debian.org/play.html#contribute

unoti 12y ago

Go has a significantly smaller memory usage in all of the benchmarks, often by a factor of 10x. For most workloads, this can have a huge impact on when you need to spend more money to support your business.

Skinney 12y ago
It also has a large impact on how often the GC is run. Less memory used, less GC performed.
- sacado2 12y ago
  It also produces less garbage since, more often than not, small values are copied, not created on the heap then referenced.
realusername 12y ago
I was thinking the same thing. It depends of your needs, if you need raw performance and you have at your disposition a lot of RAM on a powerful server , Java might be a good bet. If you are building things for a normal computer, a Raspberry PI or a background service, the memory usage of Java is going to hit you badly.
torvix 12y ago
well, you don't need to run a virtual machine, so...
igouy 12y ago
"Huge differences in default memory allocation don't necessarily mean there'll be huge differences in Memory-used for tasks that require memory to be allocated." http://benchmarksgame.alioth.debian.org/play.html#whymemory http://benchmarksgame.alioth.debian.org/play.html#whymemory

craftit 12y ago

As I understand it, Go also give some guarantees about the maximum amount of time the garbage collection will stall the program. An issue where ever you are trying to achieve consistent performance. As I understand it most of the java benchmarks postpone clearing up memory until it can be done trivially at exit, something you just can't do in a whole class of programs.

Skinney 12y ago
Not yet, but that is planned for the next release.

logicchains 12y ago

Since we're posting benchmarks, here's Java vs Haskell: http://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=ghc&lang2=java http://benchmarksgame.alioth.debian.org/u64q/compare.php?lan.... The Haskell implementations generally use much less memory than the Java ones and perform competitively. In spite of criticism that some of the Haskell implementations are not idiomatic, they're no more verbose than the Java ones in terms of LOC.

nextw33k 12y ago

This benchmark is out of date almost immediately because of the patch level of the Java runtime and the Golang compilers change frequently. I was half thinking of creating a VB6 module for LLVM just as a tongue in cheek joke of taking a language with a slow runtime and effectively strapping a V8 engine to it, but I saw someone already did most of the work: https://github.com/microcai/llvm-qbasic https://github.com/microcai/llvm-qbasic

rockdoe 12y ago
For that to be an argument two things would be required: 1) Changing patchlevels causing huge performance differences. 2) Those "patchlevels" being immediately deployable to production in the real world. Neither is true, so you don't have an argument.
igouy 12y ago
> This benchmark is out of date almost immediately Why do you think that matters?

mmf 12y ago

The way I read the results Java is basically faster only on the regex benchmark and it is known fact that today go has a slow regex library. Everything else the results show Java to be slower with larger code and larger memory footprint. Why are all the Java fans psyched by these results?

igouy 12y ago
> faster only on the regex benchmark fwiw Also binary-trees, pidigits, n-body. http://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=java&lang2=go#faster-programs-measurements http://benchmarksgame.alioth.debian.org/u64q/compare.php?lan...

TheLoneWolfling 12y ago

Now if only the benchmarks game would actually support, say, PyPy. Or LuaJIT. But nope, one implementation per language, and "language" defined rather arbitrarily at that.

igouy 12y ago
Please take the program source code and the measurement scripts and publish your own measurements. afaict we all feel the same way about this, we all feel that we should sit on our hands and wait for someone else to do the chores we don't wish to do. http://benchmarksgame.alioth.debian.org/play.html#languagex http://benchmarksgame.alioth.debian.org/play.html#languagex
- TheLoneWolfling 12y ago
  I would, if I had access to a machine that was somewhat stable w.r.t. performance. Alas, my only machine currently is my laptop, and it overheats often enough on stress-tests that it's... less than accurate, shall I say?