6 ms·

Interesting results. While it's not surprising to me that the pure java implementation was far and ahead better since: 1.) Java's much faster than Javascript,

Interesting results. While it's not surprising to me that the pure java implementation was far and ahead better since: 1.) Java's much faster than Javascript, and 2.) Netty (which Vert.x is based on), has been doing high performance async IO for years before node even existed in extremely high performance environments. What is surprising however, is that javascript on the JVM w/ vert.x is faster than V8 with node. In both cases I would assume not much JS is executed, since in Vert.x most of the executing code is still Java, with a tiny piece of JS scripting on top, and in the Node.js case, I was under the impression that most of the executing code would be the C code that powers its HTTP server. Does anyone with knowledge of the node internals know what's going on here? Is Node's HTTP parser just slower? Is its reactor not as efficient?

rapala 14y ago
Ah, but is Vert.x web scale?
- mikeheath 14y ago
  I realize you're probably joking (at least, I hope so) but just to talk to the scalability point, I've had much better success scaling Netty (which Vert.x is built on) than Node. I've been able to scale Netty to more concurrent connections and Netty performs much better under load than Node. Just my experience.
- wickedchicken 14y ago
  > Ah, but is Vert.x web scale? Better write a Mongo-backed fibonacci generator to check.
  - purplefox 14y ago
    If you read the docs we specifically mention the "Fibonacci" farce. Vert.x (unlike node) does not force you to do everything on the event loop. It has a hybrid model. For things like long running calculations (e.g. Fibonacci) or calling blocking APIs, we support running them on a background thread pool so you don't end up doing stupid things on an event loop which are not appropriate for it.
tferris 14y ago
Very good point. V8 does heavy code optimization and Node.js http sever code should be well optimized as well. And maybe the load balancer for the multicore Node test isn't optimized enough. Thus, these results feel a little shady. Anyway, Vert.x was the trigger that I am finally downloading the JVM (JDK) to try Vert.x (and maybe later Clojure). But there's still one major drawback—the non-existant ecosystem. I know the hint to look for libs from the Java world but I need a concrete and precise guide how to do this. Let's say I want to plugin some Java lib for image manipulation. How? And who will guarantee that these libs will be concurrent and/or non-blocking as well? The lib developer or Vert.x? At the moment there is—except few out-of-the-box modules—nothing. No 3rd party libs, no module manager a la npm and no guide or documentation how to glue Java libs to this Vert.x thing. Nothing. Correct me if I'm wrong.
- baudehlo 14y ago
  We know the java libraries are there for lots of things, but Node is all about async and handling thousands of connections. It does this by forcing the entire ecosystem to be async too (including things like database drivers). By using a Java JDBC database driver you're completely losing any async support. Same presumably goes for redis or Mongo drivers. You can do some of the work with threads and pooling, but it's still not the same, and makes this another useless micro benchmark.
  - tferris 14y ago
    > It does this by forcing the entire ecosystem to be async too (including things like database drivers). Yes, that's exactly Node's main selling proposition everybody forgets when presenting their next Node.js
    - soc88 14y ago
      I would call it "node.js's largest implementation issue". It is not that JavaScript gives you another choice, while you make it sound like it was a principled decision. Other platforms/languages have real concurrency constructs and don't suffer node's limitations.
wh-uws 14y ago
But be mindful though you always have to look very carefully at benchmarks. I don't think these are speed tests. Just number of connections that can be handled
- mikeheath 14y ago
  I think you need to go back to the benchmarks and take a close look at the "Requests/Second" part. These benchmarks are nothing but speed tests.
  - wh-uws 14y ago
    Ah missed that
sehugg 14y ago
I would also be interested in the internals. I'd expect JS + a small C event loop to outperform compiled JVM code + lots of async Java I/O libs (for a micro benchmark, at least). But there's no question Netty is fast even on a non-benchmark. I'm using it for a long-polling server with upwards of 100K connections, and my CPU doesn't go much over 5%. Besides memory usage, the main limitation has been proper kernel configuration.
- joncooper 14y ago
  I was surprised. We built 'hellod' servers in numerous languages and with various runtime stacks[1]. The results are here and might surprise you: https://github.com/carbonfive/hellod/blob/master/results.md https://github.com/carbonfive/hellod/blob/master/results.md [1] C/libev, Clojure/Aleph, Clojure/Jetty, Erlang, Go, Java/Netty, Java/NIO, node.js, JRuby, MRI Ruby
cbsmith 14y ago
The polling loop and I/O interfaces with Vert.x is integrated in to the VM, whereas with Node.js, you'll calling out to libuv/libev/etc. That gives the JVM an advantage. Really, how much actual JavaScript code is V8 JIT'ing at all? Really what you are comparing here is the efficiency of the paths between the two JavaScript engines and their polling I/O libraries, more than any JIT work (and let's face it, even if we were comparing JIT work, the Sun JVM has had a lot more time to tune and tweak than V8 has). In Node's case, it's talking through libuv, which means it has to transition in-and out of the V8 engine's runtime and in to a C runtime. Even if the C code is super zippy, that's costly. For Vert.x though, all the I/O and polling is integrated in to the runtime/VM/JIT. That's kind of nice. An interesting way to test what I'm suggesting would be to do the same test with unix domain sockets/named pipes as the transport instead of TCP. The JVM doesn't have a native implementation, so it'd have to call out through JNI. I'd wager even odds it ends up slower than TCP on localhost.
courtneycouch 14y ago
Using this benchmark: https://gist.github.com/2652991 https://gist.github.com/2652991 nodejs beats Vert.x for example. People shouldn't be so quick to accept these half thought out microbenchmarks.

rapala 14y ago

Ah, but is Vert.x web scale?

mikeheath 14y ago
I realize you're probably joking (at least, I hope so) but just to talk to the scalability point, I've had much better success scaling Netty (which Vert.x is built on) than Node. I've been able to scale Netty to more concurrent connections and Netty performs much better under load than Node. Just my experience.
wickedchicken 14y ago
> Ah, but is Vert.x web scale? Better write a Mongo-backed fibonacci generator to check.
- purplefox 14y ago
  If you read the docs we specifically mention the "Fibonacci" farce. Vert.x (unlike node) does not force you to do everything on the event loop. It has a hybrid model. For things like long running calculations (e.g. Fibonacci) or calling blocking APIs, we support running them on a background thread pool so you don't end up doing stupid things on an event loop which are not appropriate for it.

tferris 14y ago

Very good point. V8 does heavy code optimization and Node.js http sever code should be well optimized as well. And maybe the load balancer for the multicore Node test isn't optimized enough. Thus, these results feel a little shady. Anyway, Vert.x was the trigger that I am finally downloading the JVM (JDK) to try Vert.x (and maybe later Clojure). But there's still one major drawback—the non-existant ecosystem. I know the hint to look for libs from the Java world but I need a concrete and precise guide how to do this. Let's say I want to plugin some Java lib for image manipulation. How? And who will guarantee that these libs will be concurrent and/or non-blocking as well? The lib developer or Vert.x? At the moment there is—except few out-of-the-box modules—nothing. No 3rd party libs, no module manager a la npm and no guide or documentation how to glue Java libs to this Vert.x thing. Nothing. Correct me if I'm wrong.

baudehlo 14y ago
We know the java libraries are there for lots of things, but Node is all about async and handling thousands of connections. It does this by forcing the entire ecosystem to be async too (including things like database drivers). By using a Java JDBC database driver you're completely losing any async support. Same presumably goes for redis or Mongo drivers. You can do some of the work with threads and pooling, but it's still not the same, and makes this another useless micro benchmark.
- tferris 14y ago
  > It does this by forcing the entire ecosystem to be async too (including things like database drivers). Yes, that's exactly Node's main selling proposition everybody forgets when presenting their next Node.js
  - soc88 14y ago
    I would call it "node.js's largest implementation issue". It is not that JavaScript gives you another choice, while you make it sound like it was a principled decision. Other platforms/languages have real concurrency constructs and don't suffer node's limitations.

wh-uws 14y ago

But be mindful though you always have to look very carefully at benchmarks. I don't think these are speed tests. Just number of connections that can be handled

mikeheath 14y ago
I think you need to go back to the benchmarks and take a close look at the "Requests/Second" part. These benchmarks are nothing but speed tests.
- wh-uws 14y ago
  Ah missed that

sehugg 14y ago

I would also be interested in the internals. I'd expect JS + a small C event loop to outperform compiled JVM code + lots of async Java I/O libs (for a micro benchmark, at least). But there's no question Netty is fast even on a non-benchmark. I'm using it for a long-polling server with upwards of 100K connections, and my CPU doesn't go much over 5%. Besides memory usage, the main limitation has been proper kernel configuration.

joncooper 14y ago
I was surprised. We built 'hellod' servers in numerous languages and with various runtime stacks[1]. The results are here and might surprise you: https://github.com/carbonfive/hellod/blob/master/results.md https://github.com/carbonfive/hellod/blob/master/results.md [1] C/libev, Clojure/Aleph, Clojure/Jetty, Erlang, Go, Java/Netty, Java/NIO, node.js, JRuby, MRI Ruby

cbsmith 14y ago

The polling loop and I/O interfaces with Vert.x is integrated in to the VM, whereas with Node.js, you'll calling out to libuv/libev/etc. That gives the JVM an advantage. Really, how much actual JavaScript code is V8 JIT'ing at all? Really what you are comparing here is the efficiency of the paths between the two JavaScript engines and their polling I/O libraries, more than any JIT work (and let's face it, even if we were comparing JIT work, the Sun JVM has had a lot more time to tune and tweak than V8 has). In Node's case, it's talking through libuv, which means it has to transition in-and out of the V8 engine's runtime and in to a C runtime. Even if the C code is super zippy, that's costly. For Vert.x though, all the I/O and polling is integrated in to the runtime/VM/JIT. That's kind of nice. An interesting way to test what I'm suggesting would be to do the same test with unix domain sockets/named pipes as the transport instead of TCP. The JVM doesn't have a native implementation, so it'd have to call out through JNI. I'd wager even odds it ends up slower than TCP on localhost.

courtneycouch 14y ago

Using this benchmark: https://gist.github.com/2652991 https://gist.github.com/2652991 nodejs beats Vert.x for example. People shouldn't be so quick to accept these half thought out microbenchmarks.