5 ms·

Is node.js best for Comet?

26 points by olegp 16y ago

admc 16y ago
+1
- silentbicycle 16y ago
  Hello. It looks like you haven't participated much here. The HN community usually shuns these sorts of comments, because they don't really add to the conversation -- an upvote is already a "+1".
ericflo 16y ago
"best" is a very subjective word. If we're talking about performance, I'd venture to guess it goes something like this: C > Java > Scala/Clojure> node.js > Python > Ruby But then there's all of the other tradeoffs, like how error-prone each environment is or how long it will take to code and debug in each language. Note that Urban Airship recently did some research and found that Java with direct NIO code worked the best for them, not Netty: http://blog.urbanairship.com/blog/2010/08/24/c500k-in-action-at-urban-airship/ http://blog.urbanairship.com/blog/2010/08/24/c500k-in-action...
- cmelbye 16y ago
  Why is Ruby last? EventMachine should work pretty well for most purposes I think.
  - moe 16y ago
    Eventmachine can juggle connections fairly well, but the common MRI problems kick in once you start actually doing something with these connections. Namely heap fragmentation followed by unbounded heap growth which results in stop&go operation during gc runs - until the host eventually swaps to death. Other ruby impls (REE, JRuby or rubinius) might be better in those regards. But I wouldn't trust MRI with a long running process that might get under anything but the lightest pressure.
- pmjordan 16y ago
  It also heavily depends on what kind of state you're maintaining, whether any of it needs to be persisted or shared across connections, and how much CPU load the processing outside of pure connection handling will produce. Having a suitable scalability mechanism may be worth more than reduction of per-connection-event CPU cycles. (scaling beyond more than one core, scaling beyond more than one server)
- amix 16y ago
  I think the ultimate perfomance is found in pure java.nio/C/C++ solutions, but I think having a bit slower perfomance and higher abstraction is better since it makes it much easier to maintain and debug the system. My general impression of java.nio is that it's very low-level and a generally hard to code against and it's the main reason why we didn't choose it. This said, I think node.js offers great usability while perfomance is pretty good. So if I was developing a new comet solution I would give node a go - you can always rewrite to something more low level once you begin to hit limits. IMO going after java.nio directly is a premature optimization and most projects won't hit limits with node.js.
nc 16y ago
What about Scala and Lift?
olegp 16y ago
http://github.com/sustainablecode/ringo-evented http://github.com/sustainablecode/ringo-evented might be worth a look to get the best of both worlds
deleted 16y ago
[deleted]
davidw 16y ago
Erlang deserves a mention too, although ultimately I'm a bit pessimistic about their chances to break out of the small niche language space.
- silentbicycle 16y ago
  Me too. Awesome language/platform, but probably a bit too weird for where the industry at large is at these days, unfortunately. Anybody else coming to ErlangCamp in Chicago this weekend, BTW?
  - davidw 16y ago
    I think it's beyond just being 'weird' - it's genuinely quirky for many things, and doesn't have good library support. IMO, this makes it sort of 'uneven' - really good at some things, and bad for others. The popular languages all seem to be those that are 'pretty good' for a lot of things, so once you learn the language, you can get many things done with it. Even if the language is perhaps not the best for any given task, it's good enough to at least crank out something that works ok.
    - silentbicycle 16y ago
      The "doesn't have good library support" factor is mitigated by its "ports" - Erlang is happy to run arbitrary processes in C, Python, Lua, whatever in a bubble and just be the fault-tolerant network glue tying them together. I seem to be fond of languages that make polarizing trade-offs but encourage multi-language programming: Lua, Prolog, Erlang, awk, etc. Use them when they're the best tool for the job, and when they aren't, they'll happily co-operate. Win win. (Requiring people to know multiple languages seems like it'd be a burden, but said languages can also be a lot smaller because they can avoid being responsible for what they're bad at. And, isn't that ultimately the point of DSLs?)
      - davidw 16y ago
        I thought you just liked languages that start arrays at 1... If you have to glue stuff together in C or with pipes, it's going to be just as easy to do it in Tcl, Python, Ruby or whatever though, which also have more libraries to start with, and are fairly "promiscuous" in their own right. I'm not sure I buy the 'glue language' thing: that's what both Tcl and PHP (and Perl to some degree) started out as, but it seems that people end up wanting to do everything in the higher level language that they can, because it's a lot less effort. Something that might be really cool is if one of the efforts like Reia took off and made something a little bit nicer looking that sits on top of the Erlang run time.
silentbicycle 16y ago
This would really be best handled as two separate questions -- "is a fully asynchronous approach best for Comet" and "is the node.js implementation best for Comet" are probably very different discussions. And again, "best" could mean either performance/scalability, usability, or some combination thereof.
datapimp 16y ago
socket.io with node.js is working out very well for me. If you want to throw redis and rails in the mix, check out github.com/maccman/juggernaut
nailer 16y ago
Looks like he was already using Python - why didn't he try Tornado, which is built for purpose?
- amix 16y ago
  We tried to use Twisted and it did not perform that well (too much CPU usage, too much memory usage). I think a similar limitation will be hit with Tornado, but I haven't tried it out. Most of Plurk is implemented in Python tho' and I would love a great Python solution.
  - nailer 16y ago
    I work with Tornado daily, and while I don't have the traffic you guys do, I can say that Tornado's a lot more pleasant to program in than Twisted. asyncronous, for the end user, is implemented as a decorator you add to the URL handler's get() method. You then have an 'on_new_messges' method on the URL that's the callback to run when you've got something to push. Best of all, there's about eight sample projects that are included in the source code. Ie, they're maintained, always work, and you always have a 'certified' example of how to do something. It's neat, and the community are friendly to boot.
    - equark 16y ago
      Beyond websockets, I've found that SocketTornado.IO works great out of the box and supports multiple transports with ease. http://github.com/SocketTornadIO/SocketTornad.IO http://github.com/SocketTornadIO/SocketTornad.IO
flipp 16y ago
Socket.io + node.js makes it super easy. It abstracts away all of the transport protocols so you can just focus on writing your application.
- equark 16y ago
  Same goes for TornadoSocketIO though, so this isn't unique to node.js.
jacksonh 16y ago
I've been working on my own framework called Manos de Mono. It runs on top of Mono and is styled after node.js. I am hoping that it will be really good for these scenarios and the Mono runtime will address some of these concerns. All that being said: node.js is impressively good. Its a beautiful piece of software.

26 points by olegp 16y ago

admc 16y ago

silentbicycle 16y ago
Hello. It looks like you haven't participated much here. The HN community usually shuns these sorts of comments, because they don't really add to the conversation -- an upvote is already a "+1".

"best" is a very subjective word. If we're talking about performance, I'd venture to guess it goes something like this: C > Java > Scala/Clojure> node.js > Python > Ruby But then there's all of the other tradeoffs, like how error-prone each environment is or how long it will take to code and debug in each language. Note that Urban Airship recently did some research and found that Java with direct NIO code worked the best for them, not Netty: http://blog.urbanairship.com/blog/2010/08/24/c500k-in-action-at-urban-airship/ http://blog.urbanairship.com/blog/2010/08/24/c500k-in-action...

cmelbye 16y ago
Why is Ruby last? EventMachine should work pretty well for most purposes I think.
- moe 16y ago
  Eventmachine can juggle connections fairly well, but the common MRI problems kick in once you start actually doing something with these connections. Namely heap fragmentation followed by unbounded heap growth which results in stop&go operation during gc runs - until the host eventually swaps to death. Other ruby impls (REE, JRuby or rubinius) might be better in those regards. But I wouldn't trust MRI with a long running process that might get under anything but the lightest pressure.
pmjordan 16y ago
It also heavily depends on what kind of state you're maintaining, whether any of it needs to be persisted or shared across connections, and how much CPU load the processing outside of pure connection handling will produce. Having a suitable scalability mechanism may be worth more than reduction of per-connection-event CPU cycles. (scaling beyond more than one core, scaling beyond more than one server)
amix 16y ago
I think the ultimate perfomance is found in pure java.nio/C/C++ solutions, but I think having a bit slower perfomance and higher abstraction is better since it makes it much easier to maintain and debug the system. My general impression of java.nio is that it's very low-level and a generally hard to code against and it's the main reason why we didn't choose it. This said, I think node.js offers great usability while perfomance is pretty good. So if I was developing a new comet solution I would give node a go - you can always rewrite to something more low level once you begin to hit limits. IMO going after java.nio directly is a premature optimization and most projects won't hit limits with node.js.

nc 16y ago

What about Scala and Lift?

olegp 16y ago

http://github.com/sustainablecode/ringo-evented http://github.com/sustainablecode/ringo-evented might be worth a look to get the best of both worlds

deleted 16y ago

[deleted]

davidw 16y ago

Erlang deserves a mention too, although ultimately I'm a bit pessimistic about their chances to break out of the small niche language space.

silentbicycle 16y ago
Me too. Awesome language/platform, but probably a bit too weird for where the industry at large is at these days, unfortunately. Anybody else coming to ErlangCamp in Chicago this weekend, BTW?
- davidw 16y ago
  I think it's beyond just being 'weird' - it's genuinely quirky for many things, and doesn't have good library support. IMO, this makes it sort of 'uneven' - really good at some things, and bad for others. The popular languages all seem to be those that are 'pretty good' for a lot of things, so once you learn the language, you can get many things done with it. Even if the language is perhaps not the best for any given task, it's good enough to at least crank out something that works ok.
  - silentbicycle 16y ago
    The "doesn't have good library support" factor is mitigated by its "ports" - Erlang is happy to run arbitrary processes in C, Python, Lua, whatever in a bubble and just be the fault-tolerant network glue tying them together. I seem to be fond of languages that make polarizing trade-offs but encourage multi-language programming: Lua, Prolog, Erlang, awk, etc. Use them when they're the best tool for the job, and when they aren't, they'll happily co-operate. Win win. (Requiring people to know multiple languages seems like it'd be a burden, but said languages can also be a lot smaller because they can avoid being responsible for what they're bad at. And, isn't that ultimately the point of DSLs?)
    - davidw 16y ago
      I thought you just liked languages that start arrays at 1... If you have to glue stuff together in C or with pipes, it's going to be just as easy to do it in Tcl, Python, Ruby or whatever though, which also have more libraries to start with, and are fairly "promiscuous" in their own right. I'm not sure I buy the 'glue language' thing: that's what both Tcl and PHP (and Perl to some degree) started out as, but it seems that people end up wanting to do everything in the higher level language that they can, because it's a lot less effort. Something that might be really cool is if one of the efforts like Reia took off and made something a little bit nicer looking that sits on top of the Erlang run time.

silentbicycle 16y ago

This would really be best handled as two separate questions -- "is a fully asynchronous approach best for Comet" and "is the node.js implementation best for Comet" are probably very different discussions. And again, "best" could mean either performance/scalability, usability, or some combination thereof.

datapimp 16y ago

socket.io with node.js is working out very well for me. If you want to throw redis and rails in the mix, check out github.com/maccman/juggernaut

nailer 16y ago

Looks like he was already using Python - why didn't he try Tornado, which is built for purpose?

amix 16y ago
We tried to use Twisted and it did not perform that well (too much CPU usage, too much memory usage). I think a similar limitation will be hit with Tornado, but I haven't tried it out. Most of Plurk is implemented in Python tho' and I would love a great Python solution.
- nailer 16y ago
  I work with Tornado daily, and while I don't have the traffic you guys do, I can say that Tornado's a lot more pleasant to program in than Twisted. asyncronous, for the end user, is implemented as a decorator you add to the URL handler's get() method. You then have an 'on_new_messges' method on the URL that's the callback to run when you've got something to push. Best of all, there's about eight sample projects that are included in the source code. Ie, they're maintained, always work, and you always have a 'certified' example of how to do something. It's neat, and the community are friendly to boot.
  - equark 16y ago
    Beyond websockets, I've found that SocketTornado.IO works great out of the box and supports multiple transports with ease. http://github.com/SocketTornadIO/SocketTornad.IO http://github.com/SocketTornadIO/SocketTornad.IO

flipp 16y ago

Socket.io + node.js makes it super easy. It abstracts away all of the transport protocols so you can just focus on writing your application.

equark 16y ago
Same goes for TornadoSocketIO though, so this isn't unique to node.js.

jacksonh 16y ago

I've been working on my own framework called Manos de Mono. It runs on top of Mono and is styled after node.js. I am hoping that it will be really good for these scenarios and the Mono runtime will address some of these concerns. All that being said: node.js is impressively good. Its a beautiful piece of software.