4 ms·

I'm rather surprised by the benchmark; I would expect the Go benchmark to be faster than Java (and the fact that it isn't may indicate some improvements that ca

I'm rather surprised by the benchmark; I would expect the Go benchmark to be faster than Java (and the fact that it isn't may indicate some improvements that can be done to fasthttp by learning from rapidoid or minihttp). Then again, the difference isn't that much, so it just could be implementation details that would require a total refactor to fix.

trishume 10y ago
Why would you expect that? AFAIK the Go compiler has a worse optimizer, a worse GC (at least throughput-wise) and possibly also a worse threading runtime. Since the benchmark measures steady-state performance and not memory or latency, and with the best Java server (probably optimized to almost never allocate) then Java having better code generation and threading could give it that much of an edge.
- Manishearth 10y ago
  Go GCs fewer things, though. At least in my experience -- this isn't a hard fact :) To me, Go feels like C-but-some-things-are-GCd, which Java is not. The explicit sharing annotations in Go means that it's super easy for it to not put most of the things on the GC heap, and it's super easy for me, the programmer, to see what's not being put on the GC heap. Whenever I've debugged Go code this been validated. But yes, it's not as clear cut. I ignored the threading runtime being a factor, and ignored that Java has a better GC (though it GCs more things). I also ignored JIT as steve pointed out to me in irc. So yeah, there could be language reasons behind the benchmark order too. Worth investigating though, I'm sure the folks using fasthttp would love a perf boost :)
- zzzcpan 10y ago
  Actually, optimizer, GC and a threading runtime have almost nothing to do with Go's net/http performing that badly. The reasons are mostly in very poor design choices in the library itself that forced them to do a lot of implicit synchronization, unnecessary memory copying, unnecessary system calls, etc. It's just a huge mess, avoid it whenever possible.
  - trishume 10y ago
    Yah I know, both me and the parent comment were talking about fasthttp vs the fastest Java web server. Both of which are likely optimized like crazy to avoid synchronization and allocation as much as possible.
aturon 10y ago
Yep, same here -- at least for fasthttp. This is pretty far into microbenchmarking territory, so the differences among the top libraries likely don't matter much in practice. But I think the main point is pretty clear -- even with an extreme microbenchmark, futures don't seem to be imposing notable cost.
gribbly 10y ago
Well, Go has a 1.7 release coming up very soon which has a nice performance boost due to the new SSA backend, I assume this test was done using Go 1.6 .
jerf 10y ago
You may find this makes somewhere more sense to think of it as ~5.3 microseconds per request for fasthttp vs. ~4.8 microseconds for Java vs. ~4.3 for Rust. It's 40 microseconds or so for the standard lib Go. I'm just eyeballing the graph but this should be close enough (dominated by local CPU variances and such). Just as some people point out that "gallons per mile" is a more intuitively useful way of thinking, I think that at this scale "overhead per request" is a better way of thinking about it. I'm not generally a big fan of measuring the "ping time" response for web servers, but in this case I believe it is justified since we really are trying to establish that this future library is very fast. I fear, based on experience, that some developers well be looking at this and will sit there trying to choose fasthttp vs. rapidoid vs. minihttp based on this one graph, without considering what they really mean. Overhead-per-request I think makes it more clear that for the vast, vast majority of purposes, all of these, including Go and Node, are "way way faster than your code", and unless you know you're building a server where you seriously need to answer an API call at several hundred thousand responses per second, all of these are "fast enough" and the real criteria for choosing should be "everything else". (One last edit... remember, it's "milli/micro/nano". Micro seems to be forgotten since it seems like many things that we care about fit into nano- or milli-. It's quite a challenge in the web world to get your request out in under a millisecond usually, so .040 milliseconds added as HTTP overhead is rarely the problem.)
- aturon 10y ago
  Thanks for this comment! I actually totally agree with this perspective, and wish I'd used your suggested scale in the post.
- im_down_w_otp 10y ago
  >> "I fear, based on experience, that some developers well be looking at this and will sit there trying to choose..." Sure, but it's not the responsibility of this blog post to protect bad/lazy engineers from themselves.
- twotwotwo 10y ago
  In the case of valaya/fasthttp vs. net/http, a bonus thing is that with fasthttp you (currently+AFAIK) lose HTTP/2 support! Needs vary a ton with the app, environment, etc., of course, but it'd be sad to change your server to save 40µs CPU per req. then wind up making users wait extra RTTs--tens to hundreds of ms--as a result.
pella 10y ago
other test: techempower benchmarks https://www.techempower.com/benchmarks/#section=data-r12&hw=peak&test=update https://www.techempower.com/benchmarks/#section=data-r12&hw=...
lossolo 10y ago
Add around 10-30% (but are cases of even 150%) of speed up in Go 1.7 that will be released next week.

trishume 10y ago

Why would you expect that? AFAIK the Go compiler has a worse optimizer, a worse GC (at least throughput-wise) and possibly also a worse threading runtime. Since the benchmark measures steady-state performance and not memory or latency, and with the best Java server (probably optimized to almost never allocate) then Java having better code generation and threading could give it that much of an edge.

Manishearth 10y ago
Go GCs fewer things, though. At least in my experience -- this isn't a hard fact :) To me, Go feels like C-but-some-things-are-GCd, which Java is not. The explicit sharing annotations in Go means that it's super easy for it to not put most of the things on the GC heap, and it's super easy for me, the programmer, to see what's not being put on the GC heap. Whenever I've debugged Go code this been validated. But yes, it's not as clear cut. I ignored the threading runtime being a factor, and ignored that Java has a better GC (though it GCs more things). I also ignored JIT as steve pointed out to me in irc. So yeah, there could be language reasons behind the benchmark order too. Worth investigating though, I'm sure the folks using fasthttp would love a perf boost :)
zzzcpan 10y ago
Actually, optimizer, GC and a threading runtime have almost nothing to do with Go's net/http performing that badly. The reasons are mostly in very poor design choices in the library itself that forced them to do a lot of implicit synchronization, unnecessary memory copying, unnecessary system calls, etc. It's just a huge mess, avoid it whenever possible.
- trishume 10y ago
  Yah I know, both me and the parent comment were talking about fasthttp vs the fastest Java web server. Both of which are likely optimized like crazy to avoid synchronization and allocation as much as possible.

aturon 10y ago

Yep, same here -- at least for fasthttp. This is pretty far into microbenchmarking territory, so the differences among the top libraries likely don't matter much in practice. But I think the main point is pretty clear -- even with an extreme microbenchmark, futures don't seem to be imposing notable cost.

gribbly 10y ago

Well, Go has a 1.7 release coming up very soon which has a nice performance boost due to the new SSA backend, I assume this test was done using Go 1.6 .

jerf 10y ago

You may find this makes somewhere more sense to think of it as ~5.3 microseconds per request for fasthttp vs. ~4.8 microseconds for Java vs. ~4.3 for Rust. It's 40 microseconds or so for the standard lib Go. I'm just eyeballing the graph but this should be close enough (dominated by local CPU variances and such). Just as some people point out that "gallons per mile" is a more intuitively useful way of thinking, I think that at this scale "overhead per request" is a better way of thinking about it. I'm not generally a big fan of measuring the "ping time" response for web servers, but in this case I believe it is justified since we really are trying to establish that this future library is very fast. I fear, based on experience, that some developers well be looking at this and will sit there trying to choose fasthttp vs. rapidoid vs. minihttp based on this one graph, without considering what they really mean. Overhead-per-request I think makes it more clear that for the vast, vast majority of purposes, all of these, including Go and Node, are "way way faster than your code", and unless you know you're building a server where you seriously need to answer an API call at several hundred thousand responses per second, all of these are "fast enough" and the real criteria for choosing should be "everything else". (One last edit... remember, it's "milli/micro/nano". Micro seems to be forgotten since it seems like many things that we care about fit into nano- or milli-. It's quite a challenge in the web world to get your request out in under a millisecond usually, so .040 milliseconds added as HTTP overhead is rarely the problem.)

aturon 10y ago
Thanks for this comment! I actually totally agree with this perspective, and wish I'd used your suggested scale in the post.
im_down_w_otp 10y ago
>> "I fear, based on experience, that some developers well be looking at this and will sit there trying to choose..." Sure, but it's not the responsibility of this blog post to protect bad/lazy engineers from themselves.
twotwotwo 10y ago
In the case of valaya/fasthttp vs. net/http, a bonus thing is that with fasthttp you (currently+AFAIK) lose HTTP/2 support! Needs vary a ton with the app, environment, etc., of course, but it'd be sad to change your server to save 40µs CPU per req. then wind up making users wait extra RTTs--tens to hundreds of ms--as a result.

pella 10y ago

other test: techempower benchmarks https://www.techempower.com/benchmarks/#section=data-r12&hw=peak&test=update https://www.techempower.com/benchmarks/#section=data-r12&hw=...

lossolo 10y ago

Add around 10-30% (but are cases of even 150%) of speed up in Go 1.7 that will be released next week.