6 ms·

Node.js: details and best practices (a server side guy's analysis)

72 points by rpwilcox 15y ago

Androsynth 15y ago
One of the things I did was write a defer function (ie process.nextTick) and just threw it anywhere I thought some operation could potentially be blocking. This requires an analysis of latency vs bandwidth for your application (for me, extra latency was acceptable and bandwidth was at a premium). This also allows use of underscore libraries, albeit with extra defer calls everywhere. Unfortunately I could not convince my old company to use node.js in production because I was very curious to see how the extra function call overhead compared against the extra bandwidth. On another note, I was surprised that the eventEmitter callbacks are not executed asynchronously. I assumed each one was run on the next tick. That always struck me as inconsistent with the rest of the API.
twog 15y ago
Great information, horrible website. I dont expect every whitepaper, website, or blog to have world class design, but this is information is horribly displayed. There is no reason for a website in 2011 to look this way.
- rpwilcox 15y ago
  It was originally written in OmniOutliner, then exported to HTML. I'll see about tweaking the CSS...
davidhollander 15y ago
> Underscore.js for Iterations: each implicitly blocks ???? Blocking refers to IO system calls. underscore is not an IO library. Does a 15 year programmer with C++ experience not know what an O_NONBLOCK system call is? Blocking or non blocking is about how you are performing IO with the external world. It has nothing to do with the world internal to your program: callbacks are merely correlated with nonblocking io, continuation passing style does not constitute nonblocking in an of itself. You do not have to use continuation passing style and lambdas for code that does not perform IO for it to be "nonblocking".
- rpwilcox 15y ago
  The way I understood it is that if I have a section of Javascript that takes, say, 1/2 a second to run, and does not yield control back to the event loop, by definition blocks the event loop from firing. In a preemptive multithreaded environment it's different, but here you have one single thread and - at least to my understanding - any code that takes time to process blocks the event loop from being able to respond to incoming requests. Unless I'm missing something and node can (say) preempt your process at certain defined conditions. ("Call a function? Ok, let's check the event loop first and come right back"). I didn't see that anywhere (but I also didn't want to dig into libenv. I understand that IO is expensive, but so is performing string interpolation on a string in memory 50 times (for whatever reason).
  - bigiain 15y ago
    This is exactly what Zed was pointing out with his intentionally pessimized fibonacci generator. Sure node can make waiting on disk or network asyncronous - but it's no silver bullet, a poorly though out node.js server might scale just as badly as any other poorly thought out server in any other language. You still need to understand what's going on and not code yourself into the low performance corners.
    - stock_toaster 15y ago
      > This is exactly what Zed was pointing out with his intentionally pessimized fibonacci generator. You mean Ted Dziuba [1]? [1]: http://teddziuba.com/2011/10/node-js-is-cancer.html http://teddziuba.com/2011/10/node-js-is-cancer.html
    - rpwilcox 15y ago
      Exactly. The fibonacci generator (and subsequent async implementations) are one of the things I looked at when writing the article.
  - davidhollander 15y ago
    >I understand that IO is expensive, but so is performing string interpolation on a string in memory 50 times (for whatever reason). I think you are conflating waiting and working. Both cost time, but they are not the same. Waiting can be avoided (nonblocking), working cannot. Iterating over an array in most languages probably takes .000001 seconds. Reading from a bad connection could take 30+ seconds, or whatever your servers timeout is. Since the 30 seconds is spent _waiting_, it can be avoided within a single thread of execution. On the other hand, if you are spending 30 seconds _working_ to compute prime numbers, this cannot be avoided in a single thread of execution. Blocking vs nonblocking refers to dealing with waits. Does each IO operation do the waiting, or does each IO operation return immediately and we pool waits into a single poll() (etc.) call? Parallelization refers to dealing with work. In the rare instance one needs to generate huge fibonacci sequences, one can do so in a seperate fork\process\thread\cpu core\machine, and report results back to the event loop via pipe\socket\shared memory. [1] Multiplexing waits in a single thread makes sense, multiplexing work in a single thread such as array iteration does not: on the contrary it sounds like a good way to increase the overhead of iteration and generate more work rather than reduce it. It's confusing because some tools such as threads can be used to address both the problem of parallel work and the problem of parallel waits. [1] edit: quick example in lua: -- In fork, write to the blocking end of a pipe local r = formfork(function(w) local work = fibonacci(10^10) w:write(work) w:close() end) -- In event loop, read from the nonblocking end of a pipe readline(r , 1024, function(r, line) print('got work', line) close(r) end)
bilalhusain 15y ago
[disagreement with a select terminology eg. research, whitepaper; agreement with analysis, best practices]
jeromeparadis 15y ago
It is hinted in the article that node.js is better for some use cases than others. One thing node.js is good at is realtime apps and socket.io and the evented model make these easier. However, as a server developer guy too, I must point out that if scalability ( or premature optimization ) is dear to you, I would look further than cluster to run multiple node processes. A good solution I found for a few projects is to use Redis pub sub to encapsulate socket.io messages to connected clients to make it easy, non-blocking and performant to scale to multiple nodes. For a project where I have some potential CPU bound code, I further split these operations in another process (let's say a game state machine with calculations) with which I exchange pub sub messages. In such realtime apps, the worst that can happen is that some process intensive functions may take a bit more time to dispatch information to the clients but the core communications node processes won't be blocked and will be screaming fast and horizontally scalable. I've found that Redis pub sub is awesome for simple and fast interprocess communications.
blago 15y ago
"Useful Javascript stuff I can NOT use (because it implicitly blocks): Underscore.js" Huh? So what are we going to hear next: "Don't use prototype inheritance because chain traversal implicitly blocks"? Idiotic advise. How about: Don't hold the event loop for too long. The faster you return, the faster you can serve.
- saurik 15y ago
  Maybe one day someone will invent a mechanism that can assign some kind of time slice to each event being processed, and if it doesn't return control to the master event coordinator, it could preempt that task somehow, storing the context in which it is executing, and switching to a different one: that way we could write code without having to worry about this kind of stuff. Thinking about it more, while most people are using these event loops to process unrelated web requests (the data being shared between these different contexts likely being small), we'd still run into mild data corruption issues. :( I guess we'd also then have to figure out a way to protect these critical sections of our code... maybe some way to lock people out of your data while you are using it. (Silly pipe dream, I know.) (edit: I've been thinking a lot about this idea, and I realized that once we figure out all of those critical sections in the code, we might actually be able to run multiple event loops on the same machine at once, taking advantage of multiple CPUs or other types of execution core, without having to make any other serious changes to our code. I'm starting to think someone should actually work on this.)

72 points by rpwilcox 15y ago

One of the things I did was write a defer function (ie process.nextTick) and just threw it anywhere I thought some operation could potentially be blocking. This requires an analysis of latency vs bandwidth for your application (for me, extra latency was acceptable and bandwidth was at a premium). This also allows use of underscore libraries, albeit with extra defer calls everywhere. Unfortunately I could not convince my old company to use node.js in production because I was very curious to see how the extra function call overhead compared against the extra bandwidth. On another note, I was surprised that the eventEmitter callbacks are not executed asynchronously. I assumed each one was run on the next tick. That always struck me as inconsistent with the rest of the API.

twog 15y ago

Great information, horrible website. I dont expect every whitepaper, website, or blog to have world class design, but this is information is horribly displayed. There is no reason for a website in 2011 to look this way.

rpwilcox 15y ago
It was originally written in OmniOutliner, then exported to HTML. I'll see about tweaking the CSS...

davidhollander 15y ago

> Underscore.js for Iterations: each implicitly blocks ???? Blocking refers to IO system calls. underscore is not an IO library. Does a 15 year programmer with C++ experience not know what an O_NONBLOCK system call is? Blocking or non blocking is about how you are performing IO with the external world. It has nothing to do with the world internal to your program: callbacks are merely correlated with nonblocking io, continuation passing style does not constitute nonblocking in an of itself. You do not have to use continuation passing style and lambdas for code that does not perform IO for it to be "nonblocking".

rpwilcox 15y ago
The way I understood it is that if I have a section of Javascript that takes, say, 1/2 a second to run, and does not yield control back to the event loop, by definition blocks the event loop from firing. In a preemptive multithreaded environment it's different, but here you have one single thread and - at least to my understanding - any code that takes time to process blocks the event loop from being able to respond to incoming requests. Unless I'm missing something and node can (say) preempt your process at certain defined conditions. ("Call a function? Ok, let's check the event loop first and come right back"). I didn't see that anywhere (but I also didn't want to dig into libenv. I understand that IO is expensive, but so is performing string interpolation on a string in memory 50 times (for whatever reason).
- bigiain 15y ago
  This is exactly what Zed was pointing out with his intentionally pessimized fibonacci generator. Sure node can make waiting on disk or network asyncronous - but it's no silver bullet, a poorly though out node.js server might scale just as badly as any other poorly thought out server in any other language. You still need to understand what's going on and not code yourself into the low performance corners.
  - stock_toaster 15y ago
    > This is exactly what Zed was pointing out with his intentionally pessimized fibonacci generator. You mean Ted Dziuba [1]? [1]: http://teddziuba.com/2011/10/node-js-is-cancer.html http://teddziuba.com/2011/10/node-js-is-cancer.html
  - rpwilcox 15y ago
    Exactly. The fibonacci generator (and subsequent async implementations) are one of the things I looked at when writing the article.
- davidhollander 15y ago
  >I understand that IO is expensive, but so is performing string interpolation on a string in memory 50 times (for whatever reason). I think you are conflating waiting and working. Both cost time, but they are not the same. Waiting can be avoided (nonblocking), working cannot. Iterating over an array in most languages probably takes .000001 seconds. Reading from a bad connection could take 30+ seconds, or whatever your servers timeout is. Since the 30 seconds is spent _waiting_, it can be avoided within a single thread of execution. On the other hand, if you are spending 30 seconds _working_ to compute prime numbers, this cannot be avoided in a single thread of execution. Blocking vs nonblocking refers to dealing with waits. Does each IO operation do the waiting, or does each IO operation return immediately and we pool waits into a single poll() (etc.) call? Parallelization refers to dealing with work. In the rare instance one needs to generate huge fibonacci sequences, one can do so in a seperate fork\process\thread\cpu core\machine, and report results back to the event loop via pipe\socket\shared memory. [1] Multiplexing waits in a single thread makes sense, multiplexing work in a single thread such as array iteration does not: on the contrary it sounds like a good way to increase the overhead of iteration and generate more work rather than reduce it. It's confusing because some tools such as threads can be used to address both the problem of parallel work and the problem of parallel waits. [1] edit: quick example in lua: -- In fork, write to the blocking end of a pipe local r = formfork(function(w) local work = fibonacci(10^10) w:write(work) w:close() end) -- In event loop, read from the nonblocking end of a pipe readline(r , 1024, function(r, line) print('got work', line) close(r) end)

bilalhusain 15y ago

[disagreement with a select terminology eg. research, whitepaper; agreement with analysis, best practices]

jeromeparadis 15y ago

It is hinted in the article that node.js is better for some use cases than others. One thing node.js is good at is realtime apps and socket.io and the evented model make these easier. However, as a server developer guy too, I must point out that if scalability ( or premature optimization ) is dear to you, I would look further than cluster to run multiple node processes. A good solution I found for a few projects is to use Redis pub sub to encapsulate socket.io messages to connected clients to make it easy, non-blocking and performant to scale to multiple nodes. For a project where I have some potential CPU bound code, I further split these operations in another process (let's say a game state machine with calculations) with which I exchange pub sub messages. In such realtime apps, the worst that can happen is that some process intensive functions may take a bit more time to dispatch information to the clients but the core communications node processes won't be blocked and will be screaming fast and horizontally scalable. I've found that Redis pub sub is awesome for simple and fast interprocess communications.

blago 15y ago

"Useful Javascript stuff I can NOT use (because it implicitly blocks): Underscore.js" Huh? So what are we going to hear next: "Don't use prototype inheritance because chain traversal implicitly blocks"? Idiotic advise. How about: Don't hold the event loop for too long. The faster you return, the faster you can serve.

saurik 15y ago
Maybe one day someone will invent a mechanism that can assign some kind of time slice to each event being processed, and if it doesn't return control to the master event coordinator, it could preempt that task somehow, storing the context in which it is executing, and switching to a different one: that way we could write code without having to worry about this kind of stuff. Thinking about it more, while most people are using these event loops to process unrelated web requests (the data being shared between these different contexts likely being small), we'd still run into mild data corruption issues. :( I guess we'd also then have to figure out a way to protect these critical sections of our code... maybe some way to lock people out of your data while you are using it. (Silly pipe dream, I know.) (edit: I've been thinking a lot about this idea, and I realized that once we figure out all of those critical sections in the code, we might actually be able to run multiple event loops on the same machine at once, taking advantage of multiple CPUs or other types of execution core, without having to make any other serious changes to our code. I'm starting to think someone should actually work on this.)