4 ms·

Go is unlikely to be faster than Java (JVM is very optimized) in the near future. So rerun your benchmarks!

by anuraj 14y ago

Tuna-Fish 14y ago
In a sufficiently short test (where the jit cannot show it's chops) I'd expect go to win. The tests in the benchmark game are totally useless anyway. When given a load that does not stress the L1i, modern CPUs perform pretty much completely differently than they do on real loads with more than 32kB of code. Given such a small code snippet, the CPU will never fail a (theoretically predictable) branch, will never spill L1, and, on SNB, will in fact never even have to decode an instruction after the uOP cache is primed.
- batista 14y ago
  >When given a load that does not stress the L1i, modern CPUs perform pretty much completely differently than they do on real loads with more than 32kB of code. Even if that's the case, one would assume that would hold for ALL languages, so you get a fair comparison at that...
  - lmm 14y ago
    In the same way that a bicycle race is a fair way to compare sprinters as long as you have all of them ride bicycles. It's a fair benchmark, but it's not benchmarking the case that's actually important.
    - batista 14y ago
      No, it's like comparing sprinters by having them run, and someone telling you: "hey, the problem is not that sprinter X is ten times slower. The problem is what you want to do in the real world. Do you want to go from A to B? Why not take the bus, etc". [I'm not saying this about the L1i cache case specially, which is a good point, but rather for the prevalent response to any benchmark in Go-land, a defensive attitude which I have not witnessed in any other language community. Usually Python, JS, Ruby, Rust etc guys get on to fixing such microbenchmark behavior or explain why it's as it is. Go guys just propose you forget about it and rewrite your code in another way]. I don't care about a specific real world case of getting from A to B, or how it can be done faster in another way. I care about measuring sprinters. The case that's important here is benchmarking itself. How fast each and every instruction of a similar type executes in a language. E.g if I do: for i in range(10): print i and for i:=0; i<10; i++ { fmt.Print(i); } I don't care if this kind of code is not representative of an actual program, I don't care if the code inside the loop might take more time in most cases, I don't care if I can write some particular program using some other structure. I only want to know why this takes X time in Python and c*X time in Go, and if the c factor can be improved. Suggestions about "real world programs" and "write this another way and then measure" in this regard are counter-productive, because they focus not on raw benchmarking the language but on specific cases.
      - Tuna-Fish 14y ago
        I'd like to point out that I'm not one from the goland, and in fact I find the language quite disappointing. However, the fact that the benchmarks are rather meaningless is a real problem, because languages use resources differently, and thus resource exhaustion affects them differently. A notable example is C++-style templates (typically every codepath is expanded independently) vs java-style generics (one codepath with checks for types). C++-style absolutely dominates in small benchmarks, because it provides the best possible code. However, if you use a lot of them on different types, it's very easy for them to totally ruin your performance due to resource exhaustion. Some of the very largest performance gains I've ever gotten from a small optimization (on the order of 10x real improvement for a spot change to a few dozen lines, with no change to algorithms) involved removing C++ templates, storing tags, and adding a few case statements based on them. > I'm not saying this about the L1i cache case specially, which is a good point Note that L1i is not the only, or even the biggest offender. BTB slots are another very critical one. Predicted branches are essentially free on modern OoO cores, and business logic often contains inhuman amounts of what's essentially if trees. Have enough branches in your main loop that the BTB overflows, and thanks to LRU replacement, all those ifs turn from costing half a cycle to costing ~10 cycles or so. Properly optimizing for that case is very different from optimizing for some microbenchmark with a 100 bytes of code. And again, this is not praise for on an indictment against some language. I'm not saying that C++ is slow, it was just an easy example of a case where things change when the codebase gets bigger. What I am saying is that the performance shootout is essentially useless, and promotes the kind of optimizations that either don't really help, or even hurt speed in the large. If performance actually matters to you, you are much better off measuring things like competing mature XML parsers or the like. While there are a lot of things wrong with using that to measure the speed of your favorite language, it's still less wrong than using a five-line microbenchmark.
        igouy 14y ago
        >>If performance actually matters to you, you are much better off measuring things like competing mature XML parsers or the like.<< Please tell us where we can see that comparison! >>less wrong than using a five-line microbenchmark<< The trouble with that hyperbole is that someone's already mentioned their five-line microbenchmark in this discussion -- http://news.ycombinator.com/item?id=4544096 http://news.ycombinator.com/item?id=4544096 The alternatives to the benchmarks game that people come up with are usually worse, not better.
- igouy 14y ago
  >>In a sufficiently short test (where the jit cannot show it's chops) I'd expect go to win.<< meteor-contest Go -- 0.16s meteor-contest Java 7 #2 -- 0.22s One example of use ;-)
nobleach 14y ago
A 'properly tuned' JVM can run circles around many things. A default out of the box JVM with some compiled Java? No stinkin way. Configuring the JVM takes someone with deep intimate understanding of JVM theory. Now,the default Go compiled program is going to fire right up and get going while the JVM is still warming up. (Yes, I know there are countless things one can do to make Hotspot do amazing things) But the average default user? No... they're still going to say "the JVM is slow" and from their perspective, they're not wrong... just not knowledgeable enough to configure a ton of settings.

Go is unlikely to be faster than Java (JVM is very optimized) in the near future. So rerun your benchmarks!

by anuraj 14y ago

Go is unlikely to be faster than Java (JVM is very optimized) in the near future. So rerun your benchmarks!

In a sufficiently short test (where the jit cannot show it's chops) I'd expect go to win. The tests in the benchmark game are totally useless anyway. When given a load that does not stress the L1i, modern CPUs perform pretty much completely differently than they do on real loads with more than 32kB of code. Given such a small code snippet, the CPU will never fail a (theoretically predictable) branch, will never spill L1, and, on SNB, will in fact never even have to decode an instruction after the uOP cache is primed.

batista 14y ago
>When given a load that does not stress the L1i, modern CPUs perform pretty much completely differently than they do on real loads with more than 32kB of code. Even if that's the case, one would assume that would hold for ALL languages, so you get a fair comparison at that...
- lmm 14y ago
  In the same way that a bicycle race is a fair way to compare sprinters as long as you have all of them ride bicycles. It's a fair benchmark, but it's not benchmarking the case that's actually important.
  - batista 14y ago
    No, it's like comparing sprinters by having them run, and someone telling you: "hey, the problem is not that sprinter X is ten times slower. The problem is what you want to do in the real world. Do you want to go from A to B? Why not take the bus, etc". [I'm not saying this about the L1i cache case specially, which is a good point, but rather for the prevalent response to any benchmark in Go-land, a defensive attitude which I have not witnessed in any other language community. Usually Python, JS, Ruby, Rust etc guys get on to fixing such microbenchmark behavior or explain why it's as it is. Go guys just propose you forget about it and rewrite your code in another way]. I don't care about a specific real world case of getting from A to B, or how it can be done faster in another way. I care about measuring sprinters. The case that's important here is benchmarking itself. How fast each and every instruction of a similar type executes in a language. E.g if I do: for i in range(10): print i and for i:=0; i<10; i++ { fmt.Print(i); } I don't care if this kind of code is not representative of an actual program, I don't care if the code inside the loop might take more time in most cases, I don't care if I can write some particular program using some other structure. I only want to know why this takes X time in Python and c*X time in Go, and if the c factor can be improved. Suggestions about "real world programs" and "write this another way and then measure" in this regard are counter-productive, because they focus not on raw benchmarking the language but on specific cases.
    - Tuna-Fish 14y ago
      I'd like to point out that I'm not one from the goland, and in fact I find the language quite disappointing. However, the fact that the benchmarks are rather meaningless is a real problem, because languages use resources differently, and thus resource exhaustion affects them differently. A notable example is C++-style templates (typically every codepath is expanded independently) vs java-style generics (one codepath with checks for types). C++-style absolutely dominates in small benchmarks, because it provides the best possible code. However, if you use a lot of them on different types, it's very easy for them to totally ruin your performance due to resource exhaustion. Some of the very largest performance gains I've ever gotten from a small optimization (on the order of 10x real improvement for a spot change to a few dozen lines, with no change to algorithms) involved removing C++ templates, storing tags, and adding a few case statements based on them. > I'm not saying this about the L1i cache case specially, which is a good point Note that L1i is not the only, or even the biggest offender. BTB slots are another very critical one. Predicted branches are essentially free on modern OoO cores, and business logic often contains inhuman amounts of what's essentially if trees. Have enough branches in your main loop that the BTB overflows, and thanks to LRU replacement, all those ifs turn from costing half a cycle to costing ~10 cycles or so. Properly optimizing for that case is very different from optimizing for some microbenchmark with a 100 bytes of code. And again, this is not praise for on an indictment against some language. I'm not saying that C++ is slow, it was just an easy example of a case where things change when the codebase gets bigger. What I am saying is that the performance shootout is essentially useless, and promotes the kind of optimizations that either don't really help, or even hurt speed in the large. If performance actually matters to you, you are much better off measuring things like competing mature XML parsers or the like. While there are a lot of things wrong with using that to measure the speed of your favorite language, it's still less wrong than using a five-line microbenchmark.
      - igouy 14y ago
        >>If performance actually matters to you, you are much better off measuring things like competing mature XML parsers or the like.<< Please tell us where we can see that comparison! >>less wrong than using a five-line microbenchmark<< The trouble with that hyperbole is that someone's already mentioned their five-line microbenchmark in this discussion -- http://news.ycombinator.com/item?id=4544096 http://news.ycombinator.com/item?id=4544096 The alternatives to the benchmarks game that people come up with are usually worse, not better.
igouy 14y ago
>>In a sufficiently short test (where the jit cannot show it's chops) I'd expect go to win.<< meteor-contest Go -- 0.16s meteor-contest Java 7 #2 -- 0.22s One example of use ;-)

nobleach 14y ago

A 'properly tuned' JVM can run circles around many things. A default out of the box JVM with some compiled Java? No stinkin way. Configuring the JVM takes someone with deep intimate understanding of JVM theory. Now,the default Go compiled program is going to fire right up and get going while the JVM is still warming up. (Yes, I know there are countless things one can do to make Hotspot do amazing things) But the average default user? No... they're still going to say "the JVM is slow" and from their perspective, they're not wrong... just not knowledgeable enough to configure a ton of settings.