4 ms·

Plurk Comet: Handling of 100.000+ open connections

44 points by niels 17y ago

joel_feather 17y ago
Hmm, was twisted not the next big thing that everyone was rewriting their app in? What does he mean, it does not scale?
- mahmud 17y ago
  Not a python programmer here and I'm only guessing after a cursory look recently. Twisted developers are layering a higher-level "architecture" over the basic system calls they have wrapped. It's just a few calls down there (serial and scatter-gather I/O, multiplexing, etc.) but twisted adds higher level data structures, and design-patterny conceptual models to shield the pythonista from Unix. Some of that gotta add some fat; which might be OK for the great majority of the people, but the few who need raw performance might end up cutting some bacon off. That dude at amix.dk actually knows his stuff.
gertburger 17y ago
Being a python programmer I am a bit worried that one (In this case) needs to use java to overcome a scaling problem. From my experiences with twisted, which is on much smaller scale than the article, I have never found twisted to be anykind of bottleneck. It may be that since twisted(python) can only use one cpu effectively per interpreter(GIT et al) that it got left behind java which can easily use multiple CPUs for threads. A slightly different architecture might be required then where multiple python processes are used.
- amix 17y ago
  So far at Plurk we have struggled to overcome scalability issues - - and none of them are due to Python, even thought we use Python a lot. Most of the problems we have seen have been related to the database. It was and still is the biggest bottleneck. And all the people I know that drive big sites will confirm this. This all said, one should evaluate problems and not blindly use Python for everything. Python is a great language, but some other language is a better choice for some problems. And here Java and specially Java's NIO library is a much better choice for doing a server that should handle tons of open connections.
- majek 17y ago
  You don't have to use Java. That's exactly the reason I hacked together EvServer: http://code.google.com/p/evserver/wiki/Documentation http://code.google.com/p/evserver/wiki/Documentation Because of GIL, there is no point of using multiple threads in Python. Scaling must be done through adding more processes. Than an IPC communication starts to be an issue - that's what messaging middleware is for. The sooner you integrate messaging with your project - the better. Once you have a messaging platform - comet can be done in any technology you prefer, it really doesn't matter. That's because the scaling complexity is handled by the message broker.
- d0mine 17y ago
  s/GIT et al/GIL et al/
henryl 17y ago
Has anyone had any success with orbited?
- axod 17y ago
  afaik orbited is written using twisted, in python.
- sho 17y ago
  That Omegle site uses orbited, seems to work OK with reported load of a few thousand connections.
jasonkester 17y ago
This is a nice writeup that gives some good insight into what you're up against if you want to implement Comet today. It's a cool idea, but there's really nothing out there that you can use to do it off the shelf. It's great that he's gone through and actually tested out a bunch of these candidate technologies under load, so the next guy will at least know what not to try. I have faith that Comet will have its day eventually, but clearly it's not quite time. With the current crop of production quality web servers, handling 100k requests per second (polling) is a solved problem. Keeping 100k simultaneous connections alive (comet) is by no means solved. It'll be cool when it happens though.
- axod 17y ago
  >> "I have faith that Comet will have its day eventually, but clearly it's not quite time." ??? Many many big sites are already using comet, and have been for some years, so I'm not sure I understand your comments.
  - jasonkester 17y ago
    Correct. If you're Google, and can devote a team to build and maintain a webserver specifically designed to handle Comet, then you can use Comet. If you're small and need to use off-the-shelf tech to get your thing up and running quickly, you're still best off going with Polling today. A quick example would be to look at Thinkature vs. Twiddla. They had a team of two, one of which spent an entire year building a web server from the ground up so that they could use Comet. Twiddla only has me as a developer, and I'd rather spend my time improving the product, so we use Polling. One year later, Twiddla has a ~300ms lag between when you draw a line and when it appears on a remote screen, whereas Thinkature is out of business. I don't think it's that black and white of a tradeoff, but hopefully you see the thrust of what I'm saying. One day soon, there will be a mod_comet for Apache, or IIS 9 with Comet support built in. That will be the day it makes sense for small, lean teams to build a business around it.
    - davidw 17y ago
      axod is a team of one, and mibbit apparently uses "Comet".
est 17y ago
http://www.alexa.com/siteinfo/douban.com http://www.alexa.com/siteinfo/douban.com AFAIK douban.com seems to be the largest (pure) Python site.
- rams 17y ago
  Douban uses a lightweight python framework called Quixote. http://www.mems-exchange.org/software/quixote/ http://www.mems-exchange.org/software/quixote/ Here's the mail sent about two years back to the quixote users list: http://mail.mems-exchange.org/durusmail/quixote-users/5657/ http://mail.mems-exchange.org/durusmail/quixote-users/5657/
tezza 17y ago
SPEED :: In defence of Java, it is very quick when done right. See Kilim[1] for examples. SUPPORT :: On top of the speed Java has a supported hardware stack (Solaris). So if doing 100K connections is important to you, you can find engineers who can diagnose difficult problems on your whole stack. -------------- [1] http://www.youtube.com/watch?v=37NaHRE0Sqw http://www.youtube.com/watch?v=37NaHRE0Sqw at 23:23 . Java is quick without Kilim too, this is just an example with stats.
ars 17y ago
For the non UK people 100.000 = 100,000 or 100 000 if you want the ISO version.
- simonw 17y ago
  We use 100,000 in the UK too - 100.000 is used by some European countries (Spain is one, not sure about the others).
  - judofyr 17y ago
    FYI, in Norway we use 100,000 for 100 and 100 000 for 100000 (no dots).
- Raphael 17y ago
  No, the poster was just being very precise.
axod 17y ago
Number of open connections is a little bit meaningless until they're actually under load, sending and receiving packets in a real world situation. I'd agree, Java+NIO is a great system. Extremely fast+scalable.
- davidw 17y ago
  Not knowing much about Java's NIO, how do you handle 'long running calculations'? One of the great things about Erlang is that you can sit there doing some big huge long loop, and it won't block the rest of the system because it's got an internal scheduler. One way to do that kind of thing with C or Java is simply to manually divide your big long calculation into smaller chunks that yield control at regular, brief intervals. Does Java provide anything higher level?
  - axod 17y ago
    What sort of long calculations? If you're doing massive number crunching, you could just have a queue->thread->callback for it if it's not easy to break up into bite size chunks... I don't know of anything else built in.
    - davidw 17y ago
      Ok, fair enough. What's neat about Erlang is that you just write your code and don't have to worry about divying it up, or sending stuff of to a queue. I suppose one possible disadvantage of the Erlang way is if the scheduler is divying resources up equally, you could be 'thrashing' between too many "processes", rather than working your way through a FIFO queue that's going to only work on so many things at one time. But at that point you could just write a queue system in Erlang...
- sho 17y ago
  Well, it's not that meaningless. Comet is basically long polling; those connections aren't really doing much of anything until something turns up on the back end to be pushed down them, at which point the connection is usually closed and then re-requested by the client. Maybe with the occasional ping, I've seen some people do that too. How much memory and CPU is required to simply hold open the connection and sit there is definitely of interest to people. Of course, I'd like to see real world tests too, but that might be rather difficult to pull off without specially written, extremely efficient code on the test side.
sho 17y ago
Interesting. I'm writing a comet server right now, using Ruby's EventMachine. I might do some tests to see how many connections I can stretch it out to; unfortunately I have the feeling that local connections are not going to quite match up to real live outgoing ones. If I was doing it For Realz™ I reckon Erlang would be the go, still haven't gotten up to anywhere near that level of proficiency though. 37Signals recently rewrote their push server in Erlang and reported great results.
davidw 17y ago
This article seems relevant, BTW: http://news.ycombinator.com/item?id=334506 http://news.ycombinator.com/item?id=334506
iamelgringo 17y ago
There was a nice write up on Erlang based Comet apps here: http://cometdaily.com/2007/12/14/getting-started-with-comet-on-erlang/ http://cometdaily.com/2007/12/14/getting-started-with-comet-... Has anybody tried comet using Yaws?
c00p3r 17y ago
Is an actual c++ implementation of the java's nio is strictly better than nginx's one? Is it using epool() and sendfine()? Isn't all those java's abstraction leyers are overhead? It seems like yet another attempt to put some more air into java's bubble.

44 points by niels 17y ago

joel_feather 17y ago

Hmm, was twisted not the next big thing that everyone was rewriting their app in? What does he mean, it does not scale?

mahmud 17y ago
Not a python programmer here and I'm only guessing after a cursory look recently. Twisted developers are layering a higher-level "architecture" over the basic system calls they have wrapped. It's just a few calls down there (serial and scatter-gather I/O, multiplexing, etc.) but twisted adds higher level data structures, and design-patterny conceptual models to shield the pythonista from Unix. Some of that gotta add some fat; which might be OK for the great majority of the people, but the few who need raw performance might end up cutting some bacon off. That dude at amix.dk actually knows his stuff.

gertburger 17y ago

Being a python programmer I am a bit worried that one (In this case) needs to use java to overcome a scaling problem. From my experiences with twisted, which is on much smaller scale than the article, I have never found twisted to be anykind of bottleneck. It may be that since twisted(python) can only use one cpu effectively per interpreter(GIT et al) that it got left behind java which can easily use multiple CPUs for threads. A slightly different architecture might be required then where multiple python processes are used.

amix 17y ago
So far at Plurk we have struggled to overcome scalability issues - - and none of them are due to Python, even thought we use Python a lot. Most of the problems we have seen have been related to the database. It was and still is the biggest bottleneck. And all the people I know that drive big sites will confirm this. This all said, one should evaluate problems and not blindly use Python for everything. Python is a great language, but some other language is a better choice for some problems. And here Java and specially Java's NIO library is a much better choice for doing a server that should handle tons of open connections.
majek 17y ago
You don't have to use Java. That's exactly the reason I hacked together EvServer: http://code.google.com/p/evserver/wiki/Documentation http://code.google.com/p/evserver/wiki/Documentation Because of GIL, there is no point of using multiple threads in Python. Scaling must be done through adding more processes. Than an IPC communication starts to be an issue - that's what messaging middleware is for. The sooner you integrate messaging with your project - the better. Once you have a messaging platform - comet can be done in any technology you prefer, it really doesn't matter. That's because the scaling complexity is handled by the message broker.
d0mine 17y ago
s/GIT et al/GIL et al/

henryl 17y ago

Has anyone had any success with orbited?

axod 17y ago
afaik orbited is written using twisted, in python.
sho 17y ago
That Omegle site uses orbited, seems to work OK with reported load of a few thousand connections.

jasonkester 17y ago

This is a nice writeup that gives some good insight into what you're up against if you want to implement Comet today. It's a cool idea, but there's really nothing out there that you can use to do it off the shelf. It's great that he's gone through and actually tested out a bunch of these candidate technologies under load, so the next guy will at least know what not to try. I have faith that Comet will have its day eventually, but clearly it's not quite time. With the current crop of production quality web servers, handling 100k requests per second (polling) is a solved problem. Keeping 100k simultaneous connections alive (comet) is by no means solved. It'll be cool when it happens though.

axod 17y ago
>> "I have faith that Comet will have its day eventually, but clearly it's not quite time." ??? Many many big sites are already using comet, and have been for some years, so I'm not sure I understand your comments.
- jasonkester 17y ago
  Correct. If you're Google, and can devote a team to build and maintain a webserver specifically designed to handle Comet, then you can use Comet. If you're small and need to use off-the-shelf tech to get your thing up and running quickly, you're still best off going with Polling today. A quick example would be to look at Thinkature vs. Twiddla. They had a team of two, one of which spent an entire year building a web server from the ground up so that they could use Comet. Twiddla only has me as a developer, and I'd rather spend my time improving the product, so we use Polling. One year later, Twiddla has a ~300ms lag between when you draw a line and when it appears on a remote screen, whereas Thinkature is out of business. I don't think it's that black and white of a tradeoff, but hopefully you see the thrust of what I'm saying. One day soon, there will be a mod_comet for Apache, or IIS 9 with Comet support built in. That will be the day it makes sense for small, lean teams to build a business around it.
  - davidw 17y ago
    axod is a team of one, and mibbit apparently uses "Comet".

est 17y ago

http://www.alexa.com/siteinfo/douban.com http://www.alexa.com/siteinfo/douban.com AFAIK douban.com seems to be the largest (pure) Python site.

rams 17y ago
Douban uses a lightweight python framework called Quixote. http://www.mems-exchange.org/software/quixote/ http://www.mems-exchange.org/software/quixote/ Here's the mail sent about two years back to the quixote users list: http://mail.mems-exchange.org/durusmail/quixote-users/5657/ http://mail.mems-exchange.org/durusmail/quixote-users/5657/

tezza 17y ago

SPEED :: In defence of Java, it is very quick when done right. See Kilim[1] for examples. SUPPORT :: On top of the speed Java has a supported hardware stack (Solaris). So if doing 100K connections is important to you, you can find engineers who can diagnose difficult problems on your whole stack. -------------- [1] http://www.youtube.com/watch?v=37NaHRE0Sqw http://www.youtube.com/watch?v=37NaHRE0Sqw at 23:23 . Java is quick without Kilim too, this is just an example with stats.

ars 17y ago

For the non UK people 100.000 = 100,000 or 100 000 if you want the ISO version.

simonw 17y ago
We use 100,000 in the UK too - 100.000 is used by some European countries (Spain is one, not sure about the others).
- judofyr 17y ago
  FYI, in Norway we use 100,000 for 100 and 100 000 for 100000 (no dots).
Raphael 17y ago
No, the poster was just being very precise.

axod 17y ago

Number of open connections is a little bit meaningless until they're actually under load, sending and receiving packets in a real world situation. I'd agree, Java+NIO is a great system. Extremely fast+scalable.

davidw 17y ago
Not knowing much about Java's NIO, how do you handle 'long running calculations'? One of the great things about Erlang is that you can sit there doing some big huge long loop, and it won't block the rest of the system because it's got an internal scheduler. One way to do that kind of thing with C or Java is simply to manually divide your big long calculation into smaller chunks that yield control at regular, brief intervals. Does Java provide anything higher level?
- axod 17y ago
  What sort of long calculations? If you're doing massive number crunching, you could just have a queue->thread->callback for it if it's not easy to break up into bite size chunks... I don't know of anything else built in.
  - davidw 17y ago
    Ok, fair enough. What's neat about Erlang is that you just write your code and don't have to worry about divying it up, or sending stuff of to a queue. I suppose one possible disadvantage of the Erlang way is if the scheduler is divying resources up equally, you could be 'thrashing' between too many "processes", rather than working your way through a FIFO queue that's going to only work on so many things at one time. But at that point you could just write a queue system in Erlang...
sho 17y ago
Well, it's not that meaningless. Comet is basically long polling; those connections aren't really doing much of anything until something turns up on the back end to be pushed down them, at which point the connection is usually closed and then re-requested by the client. Maybe with the occasional ping, I've seen some people do that too. How much memory and CPU is required to simply hold open the connection and sit there is definitely of interest to people. Of course, I'd like to see real world tests too, but that might be rather difficult to pull off without specially written, extremely efficient code on the test side.

sho 17y ago

Interesting. I'm writing a comet server right now, using Ruby's EventMachine. I might do some tests to see how many connections I can stretch it out to; unfortunately I have the feeling that local connections are not going to quite match up to real live outgoing ones. If I was doing it For Realz™ I reckon Erlang would be the go, still haven't gotten up to anywhere near that level of proficiency though. 37Signals recently rewrote their push server in Erlang and reported great results.

davidw 17y ago

This article seems relevant, BTW: http://news.ycombinator.com/item?id=334506 http://news.ycombinator.com/item?id=334506

iamelgringo 17y ago

There was a nice write up on Erlang based Comet apps here: http://cometdaily.com/2007/12/14/getting-started-with-comet-on-erlang/ http://cometdaily.com/2007/12/14/getting-started-with-comet-... Has anybody tried comet using Yaws?

c00p3r 17y ago

Is an actual c++ implementation of the java's nio is strictly better than nginx's one? Is it using epool() and sendfine()? Isn't all those java's abstraction leyers are overhead? It seems like yet another attempt to put some more air into java's bubble.