5 ms·

Am I the only one that considers all of this ranting about Nodejs to be a little bit strange? I would have never expected a post that was obviously a troll to

by Spyro7 15y ago

Am I the only one that considers all of this ranting about Nodejs to be a little bit strange? I would have never expected a post that was obviously a troll to prompt this much of a reaction on both sides of an issue. That so many of these rants and counter rants made it to the front page of Hacker News is somewhat discouraging. I have been playing around with node for a few months, and I have tried to stay completely out of this "conversation". With that said, I would like to contribute just a few points: Bad programmers will be bad programmers regardless of the tools that they use. If they use node and fail to write code that is completely non-blocking, then that is what we call a teachable moment. There is no magic button, all technologies have downsides and tradeoffs. People keep talking about how nodejs is not good for computationally intensive tasks, but v8 is not a slow environment. Am I the only one that puts computationally intensive tasks into a queue to be taken care of by a pool of seperate processes? I am only just getting into web programming, and it seemed fairly obvious to me that you would not put something like that into your main event loop. Also, if you find that you must put something computationally intensive in your main event loop, then you should use something like node-proxy or nginx to proxy those requests to a number of "nodes". Over and over again, I have seen that people complain that node is not a good multithreaded environment. Well, yes... That is the tradeoff of using something that is closely tied to the concept of the event loop. If you are using node because you feel comfortable with threads and you need threads, then you are making a serious mistake. If you are a new programmer and you are using node because someone told you that it is cool, then you are making a serious mistake. If you are using node because you have a problem that can be solved or addressed with an event loop and you understand the tradeoffs inherent in this approach, then you are doing the right thing. To use nodejs effectively will often require rethinking your approach to fit the tool that you are using.

wyuenho 15y ago
I think this post is about ridiculing the childish behavior of the Node.js fandom. I have to agree that I've never seen anything quite like this before except the Church of St. Jobs. I always thought that we software engineers are reasonable people. But apparently a subset of us aren't. However reluctant I maybe to join in this time-wasting back and forth. I'm just glad that someone has taken the time to point out just how inexperienced the crowd and ridiculous the whole situation is. We have enough technology in the world already. We don't need another one that makes making mistakes so much easier than before. Lastly, I'm extremely disgusted by the misleading tag lines Ryan Dahl put on Node's front page. Programming isn't easy. It can only be easy for so long. Scaling is even harder because it actually requires deep knowledge and insight into how computers and networks work, something that I very much doubt that many of the current Node.js crowd understands. If your solution to your multiprogramming problem is to spawn more Node.js processes, maybe you've picked the wrong tech.
- deleted 15y ago
  [deleted]
- jrockway 15y ago
  I've seen it from the following communities: PHP, Ruby, Perl, Python, Java, C++, C, C#, Haskell, Emacs, Vim, Common Lisp, and Scheme. People are tribal. Some people are attracted to tribes, become attached without really knowing why, and start having the strong urge to fuck with the other tribes. Apparently this was good for the survival of the human race. Perhaps it's a bug now that we should consciously compensate for, like the desire to eat a box of doughnuts. mmm.... doughnuts... Good when you were a caveman. Bad when you sit in front of a desk for 16 hours a day and sleep for the other 8. What's the alternative to scaling an application by running multiple copies? Scaling an application by running multiple threads? Exact same thing. The only advancement over this model is when the runtime can automatically parallelize your code, and you actually scale linearly when it does that. In the mean time, if you use a database server and not an in-memory database, guess what, you're "using queues" and "scaling by starting processes". People do it because it's easy and it works.
  - wyuenho 15y ago
    This whole thread/process/event queue non-sense is really ruining my appetite. Mmm fried bacon sausage wrap on a stick… Guess what. When the packets come in from the network, they sit in an event queue. When Apache takes a request event, it takes it from a queue and hand off to a process. If you are proxying, the webapp server in the back takes the request events from a queue and hands off to a thread. When you make a DB call, the SQL goes to an event queue and the DB processes them 1 by 1. Real world web apps will always, is always and have always been done in a combination of event queues/threads and processes. Nothing to see here. Moving on… oh look! Takoyaki!
  - skeptical 15y ago
    __People are tribal. Some people are attracted to tribes, become attached without really knowing why, and start having the strong urge to fuck with the other tribes. Apparently this was good for the survival of the human race. Perhaps it's a bug now that we should consciously compensate for__ That this tribal behavior occurs among software engineering is a rather disappointing fact. Computers are pretty much the edge of technology in many aspects, technological achievements by Mankind should be the prove that we're able to use our brains in more advanced ways than basic instincts and produce great achievements like modern software. This flood of programmers that prefers to join a tribe instead of enjoying the good things from many different 'tribes' just makes me wonder how many of us are really devoted to do something useful/positive.
wonnage 15y ago
The whole target of his dissatisfaction is this quote on the Node home page: "Almost no function in Node directly performs I/O, so the process never blocks. Because nothing blocks, less-than-expert programmers are able to develop fast systems." That's bullshit. Evented programming isn't some magical fairy dust. If your request handler takes 500ms to run, you're not going to somehow serve more than two requests per second, node or no node. It's blocked on your request handling. And all that stuff Apache does for you? Well, you get to have fun setting that up in front of the nodejs server. Your sysadmin will love you. Basically if you're doing a lot of file/network IO that would normally block, node is great. You can do stuff while it's blocked, and callbacks are easier to pick up and handle than threads. But how often does that happen? Personally my Rails app spends about 10% of its time in the DB and the rest slowly generating views (and running GC, yay). AKA CPU-bound work. AKA stuff Node is just as slow at handling, with a silly deployment process to boot.
- exogen 15y ago
  Whoa, whoa. "Just as slow at handling?" It's most likely an order of magnitude faster at handling those. But you're right, deployment isn't a totally solved problem. Unless you just use Heroku: http://devcenter.heroku.com/articles/node-js http://devcenter.heroku.com/articles/node-js I mean, Node was created in 2009. I don't see anybody bragging about how easy it is to deploy yet; just that it's fast and easy to understand.
  - deleted 15y ago
    [deleted]
    - exogen 15y ago
      It's possible I'm misinterpreting the results (or the benchmark is flawed), but this seems to imply that it is often the case: http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=v8&lang2=yarv#faster-programs-measurements http://shootout.alioth.debian.org/u32/benchmark.php?test=all...
      - 15y ago
politician 15y ago
> Am I the only one that puts computationally intensive tasks into a queue to be taken care of by a pool of seperate processes? Thank you. I really find it hard to believe that the reaction to Ted's post was to mess around with Fib implementations rather than to just say, "use a background queue," and move on. If Ted has a point at all, it could be that he thinks that people using nodejs don't realize they're running in a loop. Maybe that's fair.
- jrockway 15y ago
  Ted hates queues, too. But I don't, so instead of playing tough-guy and picking on random open source communities that don't want me, I get to write cool computer programs. Maybe we should all do the same.
  - grandalf 15y ago
    What's his critique of queues?
    - deleted 15y ago
      [deleted]
    - BarkMore 15y ago
      http://teddziuba.com/2011/02/the-case-against-queues.html http://teddziuba.com/2011/02/the-case-against-queues.html
    - mhd 15y ago
      It's here: http://teddziuba.com/2011/02/the-case-against-queues.html http://teddziuba.com/2011/02/the-case-against-queues.html TL;DR - It's not so much about queues, but about stacks, i.e. new software stacks. His proposition is that quite often you're better off with existing systems. He specifically mentions syslog, so you're logging your tasks and then the consumers monitor this log. Prevents data loss and lets you potentially restart lost tasks. (That's his argument. I'd agree if you'd have to reimplement something like that in the pre-built *MQ solutions, but I don't know enough about all of them, maybe that – and more – is already in there.)
jbooth 15y ago
So, I'm not a node.js guy, and nothing against it, but being a server-side guy in general, I'll paraphrase a quote that Zed Shaw paraphrased from Chinese Kung Fu novels: "So the intermediate guy is doing all of these backflips and spinning roundhouse kicks and all that, it's very impressive, you couldn't imagine being in that good control of your body. He decides he's pretty good and spars with a master. The master barely even moves and puts the guy on his ass." I think infatuation with event-driven i/o is one of those intermediate programmer things. We all took an advanced systems course in college, saw the literature from single-CPU days about how it makes so much sense, and more importantly saw how damn clever it was. There are situations where it makes a lot of sense (static file server), and then there are situations where you say "Wait, so you want me to run 8 instances of this thing in order to utilize my 8-core server?".
- Tichy 15y ago
  Except that event-driven io is actually the simpler solution compared to threads, isn't it?
  - j_baker 15y ago
    It depends, but I don't think either one is inherently simpler than the other. As a general rule of thumb, I find that asynchronous solutions are generally better for I/O bound stuff while threads are better for computationally expensive stuff.
  - jbooth 15y ago
    Depends. For an application where you have significant CPU work, a simple worker pool (not coded by you), with an accept loop while (s=accept()) { dispatchToWorkerPool(s) } And an application-programmer method handleConnection(s) is pretty darned simple. Just do blocking I/O from your threads and rely on the machine to swap in and out appropriately. For the next level, you can have an evented i/o layer that passes sets of buffers back and forth to your worker pool. That's more complex but still spares application programmers from having to worry about either eventing or threads, they just worry about passing back the right bytes. Here's where things get interesting, though. It turns out someone studied this (in Java) and actually found that stupidly throwing 1k threads at the problem with blocking i/o performed better than a clever nonblocking server, due to linux 2.6's threading and context-switching improvements. http://www.mailinator.com/tymaPaulMultithreaded.pdf http://www.mailinator.com/tymaPaulMultithreaded.pdf Some of that might be specific to Java's integration with linux, but think about it.. the linux guys did a pretty good job at getting the right thread to wake up and assigning it to runnable state. Moving to evented i/o might be the right move for your workflow but it also might make no difference at the cost of some additional complexity. tl;dr context switching has gotten cheaper/better as linux has improved, 16 CPUs vs 1 CPU changes the math about context switching vs using select(), and worker pool libraries mean you shouldn't actually manage threads yourself.
sreque 15y ago
So I've worked on on an older open source project that does exactly what you suggest. The main loop accepts socket requests using non-blocking IO, figures out how to handle that request, and then forks a child process to handle it appropriately. This can work great and can scale very nicely, but it also has some major drawbacks compared to other solutions. The whole point of the original post was that single-threaded non-blocking event loops are very fragile. You can easily end up accidentally freezing the entire application. Other problems with forking event loops include: - Simple Event loops require you to manually CPS-transform your code. This isn't an issue for a simple request-response cycle, but it quickly turns into spaghetti code for anything more complex. For a college class I once wrote a toy non-blocking P2P client in Python using entirely non-blocking IO, while all the other students chose to use threads. My client was very performant, but the code ended up being FAR more complex than everyone else's solutions for the simple reason that I had to code in CPS the complex multi-step interactions that occur between peers. - If any of your tasks are not independent and must interact with each other, you have to start dealing with IPC, which can be very difficult to get right without a good message passing library. Even with one, it can still be more complicated than using threads. Clojure's STM, for instance, is useless without a shared memory model. - Your application is not portable to Windows if you rely on forking. And these are just some of the problems with using a simple non-blocking event loop to drive your entire application. Throw in the fact that you are using javascript, a very unscalable language never designed to do anything more than provide interactivity in the browser and that has incredibly sparse library support, and you've got lots more problems on top of that. Node.js just isn't as useful as its very enthusiastic proponents claim it to be.
InclinedPlane 15y ago
This post from Glenn Vanderburg a few years back is incredibly relevant: http://www.vanderburg.org/blog/Software/Development/sharp_and_blunt.rdoc http://www.vanderburg.org/blog/Software/Development/sharp_an... Especially this excerpt: "Weak developers will move heaven and earth to do the wrong thing. You can’t limit the damage they do by locking up the sharp tools. They’ll just swing the blunt tools harder."

by Spyro7 15y ago

wyuenho 15y ago

I think this post is about ridiculing the childish behavior of the Node.js fandom. I have to agree that I've never seen anything quite like this before except the Church of St. Jobs. I always thought that we software engineers are reasonable people. But apparently a subset of us aren't. However reluctant I maybe to join in this time-wasting back and forth. I'm just glad that someone has taken the time to point out just how inexperienced the crowd and ridiculous the whole situation is. We have enough technology in the world already. We don't need another one that makes making mistakes so much easier than before. Lastly, I'm extremely disgusted by the misleading tag lines Ryan Dahl put on Node's front page. Programming isn't easy. It can only be easy for so long. Scaling is even harder because it actually requires deep knowledge and insight into how computers and networks work, something that I very much doubt that many of the current Node.js crowd understands. If your solution to your multiprogramming problem is to spawn more Node.js processes, maybe you've picked the wrong tech.

deleted 15y ago
[deleted]
jrockway 15y ago
I've seen it from the following communities: PHP, Ruby, Perl, Python, Java, C++, C, C#, Haskell, Emacs, Vim, Common Lisp, and Scheme. People are tribal. Some people are attracted to tribes, become attached without really knowing why, and start having the strong urge to fuck with the other tribes. Apparently this was good for the survival of the human race. Perhaps it's a bug now that we should consciously compensate for, like the desire to eat a box of doughnuts. mmm.... doughnuts... Good when you were a caveman. Bad when you sit in front of a desk for 16 hours a day and sleep for the other 8. What's the alternative to scaling an application by running multiple copies? Scaling an application by running multiple threads? Exact same thing. The only advancement over this model is when the runtime can automatically parallelize your code, and you actually scale linearly when it does that. In the mean time, if you use a database server and not an in-memory database, guess what, you're "using queues" and "scaling by starting processes". People do it because it's easy and it works.
- wyuenho 15y ago
  This whole thread/process/event queue non-sense is really ruining my appetite. Mmm fried bacon sausage wrap on a stick… Guess what. When the packets come in from the network, they sit in an event queue. When Apache takes a request event, it takes it from a queue and hand off to a process. If you are proxying, the webapp server in the back takes the request events from a queue and hands off to a thread. When you make a DB call, the SQL goes to an event queue and the DB processes them 1 by 1. Real world web apps will always, is always and have always been done in a combination of event queues/threads and processes. Nothing to see here. Moving on… oh look! Takoyaki!
- skeptical 15y ago
  __People are tribal. Some people are attracted to tribes, become attached without really knowing why, and start having the strong urge to fuck with the other tribes. Apparently this was good for the survival of the human race. Perhaps it's a bug now that we should consciously compensate for__ That this tribal behavior occurs among software engineering is a rather disappointing fact. Computers are pretty much the edge of technology in many aspects, technological achievements by Mankind should be the prove that we're able to use our brains in more advanced ways than basic instincts and produce great achievements like modern software. This flood of programmers that prefers to join a tribe instead of enjoying the good things from many different 'tribes' just makes me wonder how many of us are really devoted to do something useful/positive.

wonnage 15y ago

The whole target of his dissatisfaction is this quote on the Node home page: "Almost no function in Node directly performs I/O, so the process never blocks. Because nothing blocks, less-than-expert programmers are able to develop fast systems." That's bullshit. Evented programming isn't some magical fairy dust. If your request handler takes 500ms to run, you're not going to somehow serve more than two requests per second, node or no node. It's blocked on your request handling. And all that stuff Apache does for you? Well, you get to have fun setting that up in front of the nodejs server. Your sysadmin will love you. Basically if you're doing a lot of file/network IO that would normally block, node is great. You can do stuff while it's blocked, and callbacks are easier to pick up and handle than threads. But how often does that happen? Personally my Rails app spends about 10% of its time in the DB and the rest slowly generating views (and running GC, yay). AKA CPU-bound work. AKA stuff Node is just as slow at handling, with a silly deployment process to boot.

exogen 15y ago
Whoa, whoa. "Just as slow at handling?" It's most likely an order of magnitude faster at handling those. But you're right, deployment isn't a totally solved problem. Unless you just use Heroku: http://devcenter.heroku.com/articles/node-js http://devcenter.heroku.com/articles/node-js I mean, Node was created in 2009. I don't see anybody bragging about how easy it is to deploy yet; just that it's fast and easy to understand.
- deleted 15y ago
  [deleted]
  - exogen 15y ago
    It's possible I'm misinterpreting the results (or the benchmark is flawed), but this seems to imply that it is often the case: http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=v8&lang2=yarv#faster-programs-measurements http://shootout.alioth.debian.org/u32/benchmark.php?test=all...
    - 15y ago

politician 15y ago

> Am I the only one that puts computationally intensive tasks into a queue to be taken care of by a pool of seperate processes? Thank you. I really find it hard to believe that the reaction to Ted's post was to mess around with Fib implementations rather than to just say, "use a background queue," and move on. If Ted has a point at all, it could be that he thinks that people using nodejs don't realize they're running in a loop. Maybe that's fair.

jrockway 15y ago
Ted hates queues, too. But I don't, so instead of playing tough-guy and picking on random open source communities that don't want me, I get to write cool computer programs. Maybe we should all do the same.
- grandalf 15y ago
  What's his critique of queues?
  - deleted 15y ago
    [deleted]
  - BarkMore 15y ago
    http://teddziuba.com/2011/02/the-case-against-queues.html http://teddziuba.com/2011/02/the-case-against-queues.html
  - mhd 15y ago
    It's here: http://teddziuba.com/2011/02/the-case-against-queues.html http://teddziuba.com/2011/02/the-case-against-queues.html TL;DR - It's not so much about queues, but about stacks, i.e. new software stacks. His proposition is that quite often you're better off with existing systems. He specifically mentions syslog, so you're logging your tasks and then the consumers monitor this log. Prevents data loss and lets you potentially restart lost tasks. (That's his argument. I'd agree if you'd have to reimplement something like that in the pre-built *MQ solutions, but I don't know enough about all of them, maybe that – and more – is already in there.)

jbooth 15y ago

So, I'm not a node.js guy, and nothing against it, but being a server-side guy in general, I'll paraphrase a quote that Zed Shaw paraphrased from Chinese Kung Fu novels: "So the intermediate guy is doing all of these backflips and spinning roundhouse kicks and all that, it's very impressive, you couldn't imagine being in that good control of your body. He decides he's pretty good and spars with a master. The master barely even moves and puts the guy on his ass." I think infatuation with event-driven i/o is one of those intermediate programmer things. We all took an advanced systems course in college, saw the literature from single-CPU days about how it makes so much sense, and more importantly saw how damn clever it was. There are situations where it makes a lot of sense (static file server), and then there are situations where you say "Wait, so you want me to run 8 instances of this thing in order to utilize my 8-core server?".

Tichy 15y ago
Except that event-driven io is actually the simpler solution compared to threads, isn't it?
- j_baker 15y ago
  It depends, but I don't think either one is inherently simpler than the other. As a general rule of thumb, I find that asynchronous solutions are generally better for I/O bound stuff while threads are better for computationally expensive stuff.
- jbooth 15y ago
  Depends. For an application where you have significant CPU work, a simple worker pool (not coded by you), with an accept loop while (s=accept()) { dispatchToWorkerPool(s) } And an application-programmer method handleConnection(s) is pretty darned simple. Just do blocking I/O from your threads and rely on the machine to swap in and out appropriately. For the next level, you can have an evented i/o layer that passes sets of buffers back and forth to your worker pool. That's more complex but still spares application programmers from having to worry about either eventing or threads, they just worry about passing back the right bytes. Here's where things get interesting, though. It turns out someone studied this (in Java) and actually found that stupidly throwing 1k threads at the problem with blocking i/o performed better than a clever nonblocking server, due to linux 2.6's threading and context-switching improvements. http://www.mailinator.com/tymaPaulMultithreaded.pdf http://www.mailinator.com/tymaPaulMultithreaded.pdf Some of that might be specific to Java's integration with linux, but think about it.. the linux guys did a pretty good job at getting the right thread to wake up and assigning it to runnable state. Moving to evented i/o might be the right move for your workflow but it also might make no difference at the cost of some additional complexity. tl;dr context switching has gotten cheaper/better as linux has improved, 16 CPUs vs 1 CPU changes the math about context switching vs using select(), and worker pool libraries mean you shouldn't actually manage threads yourself.

sreque 15y ago

So I've worked on on an older open source project that does exactly what you suggest. The main loop accepts socket requests using non-blocking IO, figures out how to handle that request, and then forks a child process to handle it appropriately. This can work great and can scale very nicely, but it also has some major drawbacks compared to other solutions. The whole point of the original post was that single-threaded non-blocking event loops are very fragile. You can easily end up accidentally freezing the entire application. Other problems with forking event loops include: - Simple Event loops require you to manually CPS-transform your code. This isn't an issue for a simple request-response cycle, but it quickly turns into spaghetti code for anything more complex. For a college class I once wrote a toy non-blocking P2P client in Python using entirely non-blocking IO, while all the other students chose to use threads. My client was very performant, but the code ended up being FAR more complex than everyone else's solutions for the simple reason that I had to code in CPS the complex multi-step interactions that occur between peers. - If any of your tasks are not independent and must interact with each other, you have to start dealing with IPC, which can be very difficult to get right without a good message passing library. Even with one, it can still be more complicated than using threads. Clojure's STM, for instance, is useless without a shared memory model. - Your application is not portable to Windows if you rely on forking. And these are just some of the problems with using a simple non-blocking event loop to drive your entire application. Throw in the fact that you are using javascript, a very unscalable language never designed to do anything more than provide interactivity in the browser and that has incredibly sparse library support, and you've got lots more problems on top of that. Node.js just isn't as useful as its very enthusiastic proponents claim it to be.

InclinedPlane 15y ago

This post from Glenn Vanderburg a few years back is incredibly relevant: http://www.vanderburg.org/blog/Software/Development/sharp_and_blunt.rdoc http://www.vanderburg.org/blog/Software/Development/sharp_an... Especially this excerpt: "Weak developers will move heaven and earth to do the wrong thing. You can’t limit the damage they do by locking up the sharp tools. They’ll just swing the blunt tools harder."