8 ms·
Comparing Go and Java, Part 2 – Performance
- spiffworks 14y agoIs this right? Java really has better latency than Go right now? And with 2 processors, Go's latency is 25ms for 50 concurrent requests. Anybody with better experience care to comment on this?
- georgemcbay 14y agoI have no idea if it is "right" since I'm not going to duplicate this exact benchmark, but it seems likely. Raw performance hasn't been a huge focus for Go, though this has been changing. The tip version produces much faster code than the last official release that seems to have been used here. Also you often get more efficient Go from the gccgo frontend than from the Google compiler since it can take advantage of a lot of front-end-neutral optimizations that gcc already has (though this gap is closing as the Google compiler gets improved). For the runtime, work continues on the goroutine scheduling, gc, etc. Go is getting a lot faster and pretty quickly.
- eternalban 14y ago1 - There has been a lot of man years put into the JVM (threading and garbage collection), and Java memory model. I'm somewhat surprised by the magnitude of the difference. Java and JVM are very mature technologies. 2 - Go's CPM paradigm is not necessarily more performant than the preemptive threading model of Java, and I don't believe that has ever been claimed for CPM. It is claimed that it is "easier" to write concurrent code using the (CPM/Go) message passing paradigm, but in my experience, you either get concurrency (and can do both variants ok) or you don't, in which case any real world concurrent system will hardly be "easy". That said, for the concurrency novice, Go is far less intimidating experience given the language level support for goroutines (fibers), channels, selectors, etc. (think Java NIO ...)
- cgh 14y agoThere's still this lingering belief that Java is slow, maybe related to the terrible start-up times of the JVM. But once it's going, Java is fast - really fast. And since its niche is basically enterprise webapps, a lot of attention has been paid to scalability and concurrency.
- pjmlp 14y agoA little known secret is that most mainstream languages have nice libraries to handle multi-core programming that go beyond basic thread handling, similar to what Go offers. You just have to know where to look for, plus you don't need to throw away mature languages and tooling.
- erichocean 14y agoFor the JVM, Akka is pretty sweet. It comes with both Java and Scala bindings. http://akka.io http://akka.io
- kaffiene 14y agoI don't get why you're surprised. The JVM is extremely fast and highly optimised - especially for networked and concurrent operations.
- sitharus 14y agoThat seems reasonable to me - Java's had decades of optimisation, but Go's a fairly new language and runtime. I've just started playing around with Go, I'm quite liking it. It's a different approach to things, which is always fun.
- leothekim 14y agoAgree. I've been playing with Go a bit, and though I still lean towards python and scala, Go definitely shows promise. The interfaces model and channel synchronization primitives are compelling enough to keep me interested in its progress.
- koenigdavidmj 14y agoIt seems like it might be a fun project to compile Go to Java bytecode. GCC might actually have all the relevant bits---it includes gcj, a Java compiler including bytecode generator, and a Go frontend. Knowing this, someone's already done it---anyone aware of such a thing?
- durin42 14y agohttp://code.google.com/p/jgo/ http://code.google.com/p/jgo/ but I have no idea how complete it is, I only know it exists.
- strlen 14y agoThe big pull of go for me is AOT compilation to native code (which translates to fast start time for command line utilities), close integration with the OS,support for value types, and stack allocation. These are not currently available on the JVM (although this may change as of JDK 8).
- koenigdavidmj 14y agoI think Java supports escape analysis on loops and will actually allocate stuff on the stack if it's in a tight loop. Your other points are valid; this is more a curiosity than anything else.
- mritun 14y agoI'd make a guess in absence of code. Since none of the implementations were IO bound (unsaturated link), I'd bet on memory management as the deciding factor (it's hard to assume that an authentication service will be CPU bound). Then it's not difficult to see why Java won. When it comes to garbage collectors, JVM has the best (production) implementation of GC bar none. Go runtime has a lot of catching up to do.
- stcredzero 14y agoI would like to see Ulterior Reference Counting, or age-based hybrid GC for both. http://www.powershow.com/view/14655a-MGM5M/Ulterior_Reference_Counting_Fast_Garbage_Collection_without_a_Long_Wait_flash_ppt_presentation http://www.powershow.com/view/14655a-MGM5M/Ulterior_Referenc...
- bdcravens 14y agoFirst paragraph: "In Part 1, we looked at the code of two web services that implement an authentication web service. One written in Java, and one written in Go. It’s time to beat up on them a little bit." http://boundary.com/blog/2012/09/13/comparing-go-and-java/ http://boundary.com/blog/2012/09/13/comparing-go-and-java/
- masklinn 14y agoAnd the code repo: https://github.com/collinvandyck/go-and-java https://github.com/collinvandyck/go-and-java
- luriel 14y agoYour guess is probably quite accurate, is also probably mostly benchmarking the quality of the PostgreSQL drivers, the Go ones are good, but I'm sure there has been much more work in optimizing the Java ones.
- dchest 14y agoSome notes from looking at the Go code. I don't know how important are those for performance, more like style nits. Java version doesn't do type conversion: https://github.com/collinvandyck/go-and-java/blob/master/java/src/main/java/auth/UserDAO.java#L22 https://github.com/collinvandyck/go-and-java/blob/master/jav... rs.getString("id") rs.getBoolean("admin") Go version does conversion at runtime, as row.Scan() accepts interface values. Also, the usage of sql interface is not optimal (at least with regards to code length) -- since the query is for one row, why not use QueryRow instead of Query? Authorization header decoding is strange -- it creates new base64 decoder, then string reader, then decodes. Why not just use base64.StdEncoding.DecodeString() and get rid of a few lines of code and a few allocations? https://github.com/collinvandyck/go-and-java/blob/master/go/auth_resource.go#L57 https://github.com/collinvandyck/go-and-java/blob/master/go/... Similarly, JSON marshals into a newly allocated slice instead of creating a decoder and then marshaling directly into ResponseWriter. https://github.com/collinvandyck/go-and-java/blob/master/go/auth_resource.go#L35 https://github.com/collinvandyck/go-and-java/blob/master/go/... Constants for HTTP error codes, that are declared in net/http, for some reason are redeclared https://github.com/collinvandyck/go-and-java/blob/master/go/auth_resource.go#L12 https://github.com/collinvandyck/go-and-java/blob/master/go/... - - - I sometimes wonder why various benchmarks include colorful charts, but fail to include a few lines from a simple run of profiler. It's so easy to do, and yet nobody bothers to learn where the cycles are actually spent!
- reddit_clone 14y agoDoes anyone else find 4 lines of error checking (log and panic) boiler plate code for every line of functional code a bit tedious?
- eta_carinae 14y agoAnnoying as hell, that's what you get when you don't support exceptions.
- eternalban 14y agohttp://play.golang.org/p/WUxODxK_4r http://play.golang.org/p/WUxODxK_4r
- masklinn 14y ago> that's what you get when you don't support exceptions. That's what you get when you refuse to use them anyway. Go has exceptions, there's just a dogma about never using them.
- skelterjohn 14y agoYou can't really use panics as exceptions. If you do your code will be strange and hard to work with. This strangeness is fine, because they aren't meant to be used like that. A good rule of thumb is to use panics when it's a programmer error indicating a bug.
- luriel 14y agoNot dogma, just common sense based on experience. Panic() is for truly irrecoverable exceptional situations where you do not expect the caller to catch it.
- Evbn 14y agoYou can ignore errors if you want to crash, just like Java.
- 14y ago
- KevinEldon 14y agoThe article mentioned DropWizard (covered more in part 1). DropWizard is a very nice (the best?) way to write RESTful web services in Java. Coda Hale, that author, has glued together some of the best Java libraries with very sensible configurations, almost to the point of saying "look stupid, this is how you're supposed to do it (in Java)". The application configuration patterns (ugh... I said "patterns"), clear separation of resources, built-in metrics, and health checks is worth studying (at least for journeymen like me). Check it out: http://dropwizard.codahale.com/ http://dropwizard.codahale.com/
- rudiger 14y agoHow does this compare to using Spring application development framework (with Maven, of course)? My understanding is that the Spring framework is how you're "supposed to do it" in Java, and it certainly is very popular. http://www.springsource.org/ http://www.springsource.org/
- eckyptang 14y agoI've done a fair bit with Spring in Java. Unfortunately the promises seem to fall short of delivering as your application scales up in complexity. I've experienced a number of problems with annoying little bugs and some odd brick walls with transaction management (between JMS/Hibernate). I'd rather build something with Java EE6 if I had the choice now, or ASP.Net MVC+WCF+NHibernate if I had the choice of platform.
- shanemhansen 14y agoNo surprises here. Google did a benchmark on go, java, scala, and c++ that's worth reading. http://www.readwriteweb.com/hack/2011/06/cpp-go-java-scala-performance-benchmark.php http://www.readwriteweb.com/hack/2011/06/cpp-go-java-scala-p... Here's my personal experience. Simple go code is actually comparable to c, even for non-io bound tasks. I was very surprised when OpenSSL's AES implementation and go's AES implementations performed similarly in my microbenchmarks. The jvm actually performs very well (go figure that over a decade of optmization in running enterprise workloads would result in a fast runtime). I've inspected the generated assembly in a go code and compared it with that from the equivalent c code and there's no doubt go isn't the most efficient language ever created. Use go if you want something that's (pretty darn) fast, productive, and has a standard library written by some of the most respected names in the field. Don't use go if you need the most mature and performant runtime and libraries. I have no doubt that they will get there eventually.
- luriel 14y ago> No surprises here. Google did a benchmark on go, java, scala, and c++ that's worth reading. No, is not worth reading, is misleading at best and has been throughly debunked: http://blog.golang.org/2011/06/profiling-go-programs.html http://blog.golang.org/2011/06/profiling-go-programs.html Not to mention it used an ancient version of Go, even Go 1 is dramatically faster than that, and since Go 1 there have been even more dramatic performance improvements, but the main issue is that the guy that wrote the benchmarks really had no idea what he was doing (there were similar criticisms from outside the Go communities about the quality of the benchmark).
- klrr 14y agoWow, thanks for sharing.
- shanemhansen 14y agoHonestly I'm happy to hear that. I'm one of those rare individuals who gets to write go at work. I'm really happy with go performance and memory usage. Like really really happy. The different between the cpu usage of a go program using protocol buffers and a python program using protocol buffers is pretty dramatic (go is the clear winner). Most folks would assume a paper coming out of google involving go has some go experts involved. Clearly this is not the case.
- davidw 14y agoIt'd be fun to throw Node.js and Erlang into the mix.
- dgv 14y agoYou can see vs Erlang in http://shootout.alioth.debian.org/u64q/benchmark.php?test=all&lang=go&lang2=erlang http://shootout.alioth.debian.org/u64q/benchmark.php?test=al... and have a idea, Go is based on Newsqueak (lauched later of Erlang, other perspective to implement CSP).
- davidw 14y agoSure, but this benchmark was more about concurrency, whereas those benchmarks are more generic. Erlang is not the fastest language out there, but it's supposed to be good at this concurrency stuff...
- igouy 14y agoErlang is supposed to be good at distributed, reliable, soft real-time concurrent systems. http://www.erlang.org/faq/introduction.html#id49480 http://www.erlang.org/faq/introduction.html#id49480
- davidw 14y agoSomehow, it seems that when Erlang was getting a bit of hype, they abandoned the field for an even smaller niche, leaving the "real time web" type things to other languages. Granted, the language was created for that smaller niche, but if that's the only bit of terrain they want to defend... I see other languages squeezing them out over time. I don't know if that makes sense, but languages need somewhat large communities to really thrive, in my opinion, and defining yourself into a small niche isn't a good way to get them.
- igouy 14y agoFor a short time some people hyped Erlang to be the solution for all things concurrent and chatterers didn't read the FAQ.
- jamwt 14y agoPlease, if you're writing req/resp benchmarks, please include 98/99/99.9% latencies. These are essentially the only numbers that actually matter. At the mean, at concurrency 50, we have things like 37.5ms vs. 12ms. To a user, that is "instant" vs. "instant". Sure, if we stack a bunch up on a page, maybe we'll start to care, but.. Much more directly, in real-world scaling scenarios, it matters far more if the 98% number is 2s or 248ms than the mean is 37ms. A 2s latency means 2 out of every 100 requests (and likely > 2% of page views), the user experience will suck. And, all it takes is 6 requests to ensure that 25% of users experience this (your homepage alone probably requires 6 requests). I'm not just raising this to be pedantic--I've seen plenty of systems that have been tuned to do very well on things like req/s or even mean latency but do very poorly for 1 in 100 or 1 in 1000 (vs. being a very fair scheduler at the cost of overall throughput). We reward these systems by measuring and praising the wrong thing. (e.g. mongodb vs. riak) edit: (btw, not intended as a specific defense of either go or java wrt the article, just a general statement about benchmarking these systems)
- kristianp 14y agoHere's an insightful comment from the blog page, about cpu and memory usage: I downloaded patricks fork and got 4300r/s with maxprocs 1 with the go version, out the box. Recompiling and running with gomaxprocs=100, i got 4400r/s. mvn deploy and running auth-1.0.jar on JDK 1.7 on my box similarly peaked out at 3100r/s. It's worth noting though, that I was using patricks httperf bench.sh modification, and it appears that httperf was cpu bound in both cases, with the kernel and postres taking about half a core between them. Using wrk, by contrast, spun main (the go program) up to 2.5 cores, and 6000r/s. Java under wrk lit all cores for a time, then hit `Exception in thread "async-log-appender-0" java.lang.OutOfMemoryError: Java heap space` `Caused by: ! org.postgresql.util.PSQLException: FATAL: sorry, too many clients already` A bit more investigation and tuning later, using 10 clients `wrk -c 10 -r 100000 -t 4 -H 'Authorization: basic apikey_value' http://localhost:8080/authenticate` http://localhost:8080/authenticate` Java was spinning out 10kr/s. Go was limited to 6kr/s. It's worth noting some more details however: Go only ever spun up two postgres forks, whereas java spun up 10. There's scope for optimization there. Go used 8mb of ram, whereas java was sitting on 130mb after a few runs. The Java version, cranking out at 10kr/s was maxing the kernel out on one core, so that's probably approaching the practical limit for single machine tests. I suspect putting a load balancer in front of a couple of instances of the go program would allow you to totally smash the java performance, given that it lit 2 cores at 6kr/s, and java lit 8 at 10kr/s. The memory usage tradeoff is significant - JVM sitting at 130mb and Go sitting at 8mb. Clearly everyone needs to draw their own conclusions on this, for their own purposes. The JVM solution is carrying a stats server, a ton of other tooling and so on. The Go system has some - pprof was included, but it's limited by comparison. Arguably gdb and so on can actually be of real use in the Go case, but that's also an exercise for the reader. Interesting any which way you look at it. There are a lot of other interesting side effects (that need working out) in both programs as evidenced by this simple testing. Postgres also needs some tuning if you really want to slam this with anything remotely resembling a real world scale test. My tests were done on OSX 10.8 12B19 on a 2.3GHz i7 wiht 8GB of DDR3 at 1333MHz, an intel 510 SSD, totally untuned postgres. The machine is a Macbook from whatever year that is. Here's wrk, for anyone looking for it: https://github.com/wg/wrk https://github.com/wg/wrk