5 ms·

Finally, Python may not be the best language for this real-time endpoint. Eventlet is a fantastic Python library and it allowed us to build something extremely

Finally, Python may not be the best language for this real-time endpoint. Eventlet is a fantastic Python library and it allowed us to build something extremely fast that has scaled to several thousand concurrent connections without breaking a sweat on launch day, but it has its limits. There is a large body of work out there on handling a large number of open connections, using Java's NIO framework, Erlang's mochiweb, or node.js. I wrote justin.tv's chat backend, in Python, using the Twisted network libraries. It has scaled to peaks of more than half a million concurrent chat connections, on 8 fairly modest commodity servers. Python is more than capable here, with the right networking approach. Feel free to ask me anything about it.

ericflo 16y ago
That's awesome, thanks for the offer, I'll definitely take you up on it and ping you if we run into problems scaling up that live endpoint using Python. As a side-note, this is why I love the software engineering community: everyone's always so willing to share knowledge and help each other out. Let's hope we never lose that as a community!
- SriniK 16y ago
  Thanks for sharing. Did you look at Tornado? I feel it is lightweight and simple.
  - ericflo 16y ago
    Yep, but we liked Eventlet better because of the simpler programming model and better support for things like psycopg2 (our database driver). So far it's worked out great--it's held up really well under our current amount of load. That said, Tornado is a really neat piece of software, and it worked very well for FriendFeed and others. It really just came down to personal preference.
    - BarkMore 16y ago
      In what way does Tornado not support psycopg2? Because the Tornado code includes some convenience functions for working with an unmodified MySQLdb driver, I assume that FriendFeed also used the MySQLdb driver with no modifications. If the MySQLdb driver works with Tornado, shouldn't psycopg2 also work?
      - ericflo 16y ago
        Not asynchronously.
        BarkMore 16y ago
        By using an unmodified MySQLdb driver, FriendFeed did not make asynchronous calls to the database. I searched around and found this comment w/ response from Paul Buchheit about the issue: http://news.ycombinator.com/item?id=1479301 http://news.ycombinator.com/item?id=1479301 Do you need to make async calls to the database because your queries are slow?
- axod 16y ago
  FWIW, I'd recommend Java NIO, it's served me really well on Mibbit, so feel free to ping me if you decide to go that route. Whatever language you choose though there's plenty of interesting scaling problems to work on ;)
powdahound 16y ago
I'd like to second this. Although Twisted's documentation and 5-year-old bugs will have you cursing the libraries name at times it really does perform well once you figure it all out. We're using Python + Twisted for our XMPP servers at HipChat (http://www.hipchat.com http://www.hipchat.com) and neither have let us down. Hopefully we can reach 500k+ connections on such a small number of servers as well. One specific note about Twisted/Python vs node.js: The combination of Python's yield and Twisted's defer.inlineCallbacks generator makes it really easy to write and maintain nonblocking code. In node I find it far too easy to get lost in a sea of callback functions. Here's what you get to do in Twisted: http://enthusiasm.cozy.org/archives/2009/03/python-twisteds-inlinecallbacks http://enthusiasm.cozy.org/archives/2009/03/python-twisteds-...
- hammerdr 16y ago
  With Node.js, you can use defers (promises), as well. As long as you feel that promises isn't too much mental overhead (obviously you don't), I would recommend using promises for any sizable Node.js application. https://github.com/kriskowal/q https://github.com/kriskowal/q
- deleted 16y ago
  [deleted]
huyegn 16y ago
We're also using Twisted a lot at my company, IQ Engines. It's powerful and has done a pretty good job for us so far. I just wish the documentation, which is crucial for getting new coworkers up to speed, was better. Do you guys continue to build new services at Justin.TV using it or have you migrated to other eventloops ie Tornado, gevent etc?
- abstractbill 16y ago
  We use it for new things - I think the last thing I wrote with it was a custom dynamic dns server we needed - that was just a few weeks ago I think. Yeah, the documentation is awful. Fortunately I've pretty much always just assumed that all documentation is going to be bad, so I pretty much never even bother trying - I just read the source.
deleted 16y ago
[deleted]

ericflo 16y ago

That's awesome, thanks for the offer, I'll definitely take you up on it and ping you if we run into problems scaling up that live endpoint using Python. As a side-note, this is why I love the software engineering community: everyone's always so willing to share knowledge and help each other out. Let's hope we never lose that as a community!

SriniK 16y ago
Thanks for sharing. Did you look at Tornado? I feel it is lightweight and simple.
- ericflo 16y ago
  Yep, but we liked Eventlet better because of the simpler programming model and better support for things like psycopg2 (our database driver). So far it's worked out great--it's held up really well under our current amount of load. That said, Tornado is a really neat piece of software, and it worked very well for FriendFeed and others. It really just came down to personal preference.
  - BarkMore 16y ago
    In what way does Tornado not support psycopg2? Because the Tornado code includes some convenience functions for working with an unmodified MySQLdb driver, I assume that FriendFeed also used the MySQLdb driver with no modifications. If the MySQLdb driver works with Tornado, shouldn't psycopg2 also work?
    - ericflo 16y ago
      Not asynchronously.
      - BarkMore 16y ago
        By using an unmodified MySQLdb driver, FriendFeed did not make asynchronous calls to the database. I searched around and found this comment w/ response from Paul Buchheit about the issue: http://news.ycombinator.com/item?id=1479301 http://news.ycombinator.com/item?id=1479301 Do you need to make async calls to the database because your queries are slow?
axod 16y ago
FWIW, I'd recommend Java NIO, it's served me really well on Mibbit, so feel free to ping me if you decide to go that route. Whatever language you choose though there's plenty of interesting scaling problems to work on ;)

powdahound 16y ago

I'd like to second this. Although Twisted's documentation and 5-year-old bugs will have you cursing the libraries name at times it really does perform well once you figure it all out. We're using Python + Twisted for our XMPP servers at HipChat (http://www.hipchat.com http://www.hipchat.com) and neither have let us down. Hopefully we can reach 500k+ connections on such a small number of servers as well. One specific note about Twisted/Python vs node.js: The combination of Python's yield and Twisted's defer.inlineCallbacks generator makes it really easy to write and maintain nonblocking code. In node I find it far too easy to get lost in a sea of callback functions. Here's what you get to do in Twisted: http://enthusiasm.cozy.org/archives/2009/03/python-twisteds-inlinecallbacks http://enthusiasm.cozy.org/archives/2009/03/python-twisteds-...

hammerdr 16y ago
With Node.js, you can use defers (promises), as well. As long as you feel that promises isn't too much mental overhead (obviously you don't), I would recommend using promises for any sizable Node.js application. https://github.com/kriskowal/q https://github.com/kriskowal/q
deleted 16y ago
[deleted]

huyegn 16y ago

We're also using Twisted a lot at my company, IQ Engines. It's powerful and has done a pretty good job for us so far. I just wish the documentation, which is crucial for getting new coworkers up to speed, was better. Do you guys continue to build new services at Justin.TV using it or have you migrated to other eventloops ie Tornado, gevent etc?

abstractbill 16y ago
We use it for new things - I think the last thing I wrote with it was a custom dynamic dns server we needed - that was just a few weeks ago I think. Yeah, the documentation is awful. Fortunately I've pretty much always just assumed that all documentation is going to be bad, so I pretty much never even bother trying - I just read the source.

deleted 16y ago

[deleted]