9 ms·

Node.js and io.js – Very different in performance

171 points by mschoebel 12y ago

richmarr 12y ago
> Depending on what your Node application does, my > findings may or may not apply to your use-case. I'm going to go out on a limb and say that the proportion of real world Node apps that will be noticably affected by this is less than 1%
Kiro 12y ago
> This can be extremely important if you have a project with heavy CPU-use Would you recommend using something different than JavaScript when writing CPU heavy apps? I was under the impression that it's better suited when dealing with high I/O.
- FooBarWidget 12y ago
  Technically you can offload CPU-heavy things to a cluster of extra Node processes that handle work in a serial fashion. Nothing wrong with this; if you're comfortable with Javascript then this is probably better than rewriting those parts in different languages.
  - lsiebert 12y ago
    You could, and that might get you the performance you need, but if not I would likely use https://github.com/node-ffi/node-ffi https://github.com/node-ffi/node-ffi or maybe http://www.swig.org/ http://www.swig.org/ to call out to C++ or C code. There's probably also a way to call Java, ( a quick search suggests https://github.com/joeferner/node-java https://github.com/joeferner/node-java perhaps) but I can't speak to that in particular.
- mschoebel 12y ago
  IMO if you really need to do heavy-duty processing, C/C++ will be fastest. On the other hand Node will make development easier. You have to judge for yourself which will be a better use of resources. Having more costs upfront for programming, or more costs in the long-run for servers.
- romanovcode 12y ago
  I would not recommend using Javascript at all when using CPU heavy apps. C++, Java, C# would be much better.
  - Kiro 12y ago
    Sorry, that's what I meant. Updated my question. Thanks.
- bananaoomarang 12y ago
  Generally speaking for CPU bound tasks you should use something like edge.js (http://tjanczuk.github.io/edge/#/ http://tjanczuk.github.io/edge/#/) if you're using Node for dev speed/ease's sake.
- askmike 12y ago
  If you decide to offload some CPU heavy parts I would personally use a messaging protocol like zeroMQ to keep a loose coupling.
- dagw 12y ago
  Sure if your project is almost exclusively CPU intensive then use something else. But sometimes you have a project that already is in JavaScript and where JavaScript mostly makes sense, but you have one or two task that are small but fairly CPU intensive. Then knowing how to write fast JavaScript is a pretty good idea since JavaScript can be pretty fast these days and the overhead (both in terms of runtime and developer time) of calling out to second language isn't always trivial and it's nice if you can avoid it. Secondly (and not entirely relevant to this) people are doing more and more 'clever' CPU-intensive things client side these days where JavaScript is your only choice. So understanding the performance characteristics of different JavaScript implementations can definitely come in handy there.
- richmarr 12y ago
  A few folks have replied with the usual "use C/C++/Java instead", but in the real world it's often either impractical (or rather commercially indefensible) to fork out a different environment with its own training, testing, environment, automation, documentation and maintenance overheads. A blanket rejection of Node for CPU-heavy tasks is naive. On the issue of performance, V8 lets Javascript run pretty quickly. Yes, there are languages that broadly offer faster execution, but that's far from the only factor in choosing a solution. The main issue from my perspective is that the event loop can easily get blocked by CPU-bound tasks, preventing it from doing other things, e.g. responding to HTTP requests. You hit a similar problem with a Java servlet runner, eg. if a couple of your threads are bogged down on CPU-heavy tasks then they can't be responding to requests. My personal preference would be to split CPU-heavy operations out so that they happen elsewhere, regardless of language, e.g having large PDFs generated by an internal microservice rather than by the webserver, or maybe via a queue in some cases. But that's just a personal preference.
  - k__ 12y ago
    Really? Companies I worked for always had two environments, one for new features and one for performance. Like PHP and C, new features where implemented in PHP and if they caught on, they got reimplemented in C if they needed better performance.
    - richmarr 12y ago
      Sure, if the companies you've worked for are in a field that needs incremental performance gains and are willing to pay for it then that's totally rational. Typically I see client-side performance concerns outweighing server-side performance in a ratio of 70/30 or so, with the remaining server-side performance biased towards I/O concerns like waiting for data, or file system reads with a ratio of 90/10 or more. That puts the actual saving available to language or algorithm changes in the app layer to be less than 3% for the kinds of apps I've worked on. I usually work at companies who are starting out, looking for Product Market Fit, where those marginal gains aren't worth the cost of reimplementing.
      - 12y ago
bnoordhuis 12y ago
Interesting results, thanks for sharing. I can perhaps shed some light on the performance differences. > Buffer 4.259 5.006 In v0.10, buffers are sliced off from big chunks of pre-allocated memory. It makes allocating buffers a little cheaper but because each buffer maintains a back pointer to the backing memory, that memory isn't reclaimed until the last buffer is garbage collected. Buffers in node.js v0.11 and io.js v1.x instead own their memory. It reduces peak memory (because memory is no longer allocated in big chunks) and removes a whole class of accidental memory leaks. That said, the fact that it's sometimes slower is definitely something to look into. > Typed-Array 4.944 11.555 Typed arrays in v0.10 are a homegrown and non-conforming implementation. Node.js v0.11 and io.js v1.x use V8's native typed arrays, which are indeed slower at this point. I know the V8 people are working on them, it's probably just a matter of time - although more eyeballs certainly won't hurt. > Regular Array 40.416 7.359 Full credit goes to the V8 team for that one. :-)
- fulafel 12y ago
  Does the dramatic speed difference between the "non-conforming" implementation and V8 mean that current Node typed arrays are not memory-safe and you may get C-style buffer overflow vulnerabilities when using them?
  - trevnorris 12y ago
    "Non-conforming" only means they didn't completely adhere to the ES specification. There should be no possibility of buffer overflow.
- mschoebel 12y ago
  FWIW I also did a test of Node:master and that performance was within 2% of what I measured for io.js. Interesting background about typed-arrays. I didn't know that. Thanks!
  - mikkom 12y ago
    > FWIW I also did a test of Node:master and that performance was within 2% of what I measured for io.js. It would have been a good thing to include that comment to the article as well.
SixSigma 12y ago
On a slight tangent, there's an article using the Sieve of Eratosthenes demonstrating the use of Communicating Sequential Threads (CSP) on Russ Cox' website (one of the developers of Go) http://swtch.com/~rsc/thread/ http://swtch.com/~rsc/thread/
wolframhempel 12y ago
These are very interesting findings. On a higher level though: Are there any significant performance differences between the APIs of node and io? E.g. tcp package processing, file system access etc? I know that a lot of them are effectively C, so independent of the V8 version.
explorigin 12y ago
There are two very good comments at the bottom of the article. Here for your consumption: Author: (Unknown) 2015-01-19 12:54 UTC io.js based on node v0.11, so you need compare - v0.10 (nodejs)- v0.11 (nodejs) - v0.11 (nodejs)- v1.0 (iojs) Author: Michael Schöbel 2015-01-19 13:01 UTC I also downloaded sources and compiled the latest master branch of Node yesterday evening. Performance was within 2% of io.js for all three tests. But most people won't compile themselves. Most will use the latest stable release.
- longlivegnu 12y ago
  >But most people won't compile themselves. Most will use the latest stable release I mean there is a good reason that
cdnsteve 12y ago
Reporting benchmark results on a single OS, on a single CPU type isn't really benchmarking. It's an isolated case of results. I'd recommend to perform an accurate suite of performance tests, use different OS (CoreOS, Ubuntu) that are actually used in server environments. Also different machine hardware will play a role. There's not enough data at this point to come to any conclusion at this point imo. This result set is like saying that 95% of the people on the web use the safari browser on the Apple website.
- morenoh149 12y ago
  ... in the apple store
daphneokeefe 12y ago
The competition between these teams is going to make both of them better. They will not only be competing on speed, but also on features. A huge win for developers.

171 points by mschoebel 12y ago

> Depending on what your Node application does, my > findings may or may not apply to your use-case. I'm going to go out on a limb and say that the proportion of real world Node apps that will be noticably affected by this is less than 1%

Kiro 12y ago

> This can be extremely important if you have a project with heavy CPU-use Would you recommend using something different than JavaScript when writing CPU heavy apps? I was under the impression that it's better suited when dealing with high I/O.

FooBarWidget 12y ago
Technically you can offload CPU-heavy things to a cluster of extra Node processes that handle work in a serial fashion. Nothing wrong with this; if you're comfortable with Javascript then this is probably better than rewriting those parts in different languages.
- lsiebert 12y ago
  You could, and that might get you the performance you need, but if not I would likely use https://github.com/node-ffi/node-ffi https://github.com/node-ffi/node-ffi or maybe http://www.swig.org/ http://www.swig.org/ to call out to C++ or C code. There's probably also a way to call Java, ( a quick search suggests https://github.com/joeferner/node-java https://github.com/joeferner/node-java perhaps) but I can't speak to that in particular.
mschoebel 12y ago
IMO if you really need to do heavy-duty processing, C/C++ will be fastest. On the other hand Node will make development easier. You have to judge for yourself which will be a better use of resources. Having more costs upfront for programming, or more costs in the long-run for servers.
romanovcode 12y ago
I would not recommend using Javascript at all when using CPU heavy apps. C++, Java, C# would be much better.
- Kiro 12y ago
  Sorry, that's what I meant. Updated my question. Thanks.
bananaoomarang 12y ago
Generally speaking for CPU bound tasks you should use something like edge.js (http://tjanczuk.github.io/edge/#/ http://tjanczuk.github.io/edge/#/) if you're using Node for dev speed/ease's sake.
askmike 12y ago
If you decide to offload some CPU heavy parts I would personally use a messaging protocol like zeroMQ to keep a loose coupling.
dagw 12y ago
Sure if your project is almost exclusively CPU intensive then use something else. But sometimes you have a project that already is in JavaScript and where JavaScript mostly makes sense, but you have one or two task that are small but fairly CPU intensive. Then knowing how to write fast JavaScript is a pretty good idea since JavaScript can be pretty fast these days and the overhead (both in terms of runtime and developer time) of calling out to second language isn't always trivial and it's nice if you can avoid it. Secondly (and not entirely relevant to this) people are doing more and more 'clever' CPU-intensive things client side these days where JavaScript is your only choice. So understanding the performance characteristics of different JavaScript implementations can definitely come in handy there.
richmarr 12y ago
A few folks have replied with the usual "use C/C++/Java instead", but in the real world it's often either impractical (or rather commercially indefensible) to fork out a different environment with its own training, testing, environment, automation, documentation and maintenance overheads. A blanket rejection of Node for CPU-heavy tasks is naive. On the issue of performance, V8 lets Javascript run pretty quickly. Yes, there are languages that broadly offer faster execution, but that's far from the only factor in choosing a solution. The main issue from my perspective is that the event loop can easily get blocked by CPU-bound tasks, preventing it from doing other things, e.g. responding to HTTP requests. You hit a similar problem with a Java servlet runner, eg. if a couple of your threads are bogged down on CPU-heavy tasks then they can't be responding to requests. My personal preference would be to split CPU-heavy operations out so that they happen elsewhere, regardless of language, e.g having large PDFs generated by an internal microservice rather than by the webserver, or maybe via a queue in some cases. But that's just a personal preference.
- k__ 12y ago
  Really? Companies I worked for always had two environments, one for new features and one for performance. Like PHP and C, new features where implemented in PHP and if they caught on, they got reimplemented in C if they needed better performance.
  - richmarr 12y ago
    Sure, if the companies you've worked for are in a field that needs incremental performance gains and are willing to pay for it then that's totally rational. Typically I see client-side performance concerns outweighing server-side performance in a ratio of 70/30 or so, with the remaining server-side performance biased towards I/O concerns like waiting for data, or file system reads with a ratio of 90/10 or more. That puts the actual saving available to language or algorithm changes in the app layer to be less than 3% for the kinds of apps I've worked on. I usually work at companies who are starting out, looking for Product Market Fit, where those marginal gains aren't worth the cost of reimplementing.
    - 12y ago

bnoordhuis 12y ago

Interesting results, thanks for sharing. I can perhaps shed some light on the performance differences. > Buffer 4.259 5.006 In v0.10, buffers are sliced off from big chunks of pre-allocated memory. It makes allocating buffers a little cheaper but because each buffer maintains a back pointer to the backing memory, that memory isn't reclaimed until the last buffer is garbage collected. Buffers in node.js v0.11 and io.js v1.x instead own their memory. It reduces peak memory (because memory is no longer allocated in big chunks) and removes a whole class of accidental memory leaks. That said, the fact that it's sometimes slower is definitely something to look into. > Typed-Array 4.944 11.555 Typed arrays in v0.10 are a homegrown and non-conforming implementation. Node.js v0.11 and io.js v1.x use V8's native typed arrays, which are indeed slower at this point. I know the V8 people are working on them, it's probably just a matter of time - although more eyeballs certainly won't hurt. > Regular Array 40.416 7.359 Full credit goes to the V8 team for that one. :-)

fulafel 12y ago
Does the dramatic speed difference between the "non-conforming" implementation and V8 mean that current Node typed arrays are not memory-safe and you may get C-style buffer overflow vulnerabilities when using them?
- trevnorris 12y ago
  "Non-conforming" only means they didn't completely adhere to the ES specification. There should be no possibility of buffer overflow.
mschoebel 12y ago
FWIW I also did a test of Node:master and that performance was within 2% of what I measured for io.js. Interesting background about typed-arrays. I didn't know that. Thanks!
- mikkom 12y ago
  > FWIW I also did a test of Node:master and that performance was within 2% of what I measured for io.js. It would have been a good thing to include that comment to the article as well.

SixSigma 12y ago

On a slight tangent, there's an article using the Sieve of Eratosthenes demonstrating the use of Communicating Sequential Threads (CSP) on Russ Cox' website (one of the developers of Go) http://swtch.com/~rsc/thread/ http://swtch.com/~rsc/thread/

wolframhempel 12y ago

These are very interesting findings. On a higher level though: Are there any significant performance differences between the APIs of node and io? E.g. tcp package processing, file system access etc? I know that a lot of them are effectively C, so independent of the V8 version.

explorigin 12y ago

There are two very good comments at the bottom of the article. Here for your consumption: Author: (Unknown) 2015-01-19 12:54 UTC io.js based on node v0.11, so you need compare - v0.10 (nodejs)- v0.11 (nodejs) - v0.11 (nodejs)- v1.0 (iojs) Author: Michael Schöbel 2015-01-19 13:01 UTC I also downloaded sources and compiled the latest master branch of Node yesterday evening. Performance was within 2% of io.js for all three tests. But most people won't compile themselves. Most will use the latest stable release.

longlivegnu 12y ago
>But most people won't compile themselves. Most will use the latest stable release I mean there is a good reason that

cdnsteve 12y ago

Reporting benchmark results on a single OS, on a single CPU type isn't really benchmarking. It's an isolated case of results. I'd recommend to perform an accurate suite of performance tests, use different OS (CoreOS, Ubuntu) that are actually used in server environments. Also different machine hardware will play a role. There's not enough data at this point to come to any conclusion at this point imo. This result set is like saying that 95% of the people on the web use the safari browser on the Apple website.

morenoh149 12y ago
... in the apple store

daphneokeefe 12y ago

The competition between these teams is going to make both of them better. They will not only be competing on speed, but also on features. A huge win for developers.