6 ms·

Decline of the Tornado Web Server - What happened?

64 points by Soupy 16y ago

Soupy 16y ago
This graph displays the average requests/second that the python tornado framework can handle graphed over time by commits. Very surprising and also a little concerning since I am relying on Tornado for my current projects. Just wanted to make sure that you all were made aware of this in case you are considering using Tornado for your own projects (although I have had nothing but positive experiences with it). Source - http://groups.google.com/group/python-tornado/browse_thread/thread/94b45e815ac992ac# http://groups.google.com/group/python-tornado/browse_thread/...
- crad 16y ago
  Thanks for posting this. I'm eager to see Ben and others reply to this on the list.
- viksit 16y ago
  Thats a great script you guys wrote! Any chance it's going to be open sourced? Would be really fun to play around with github projects!
- SriniK 16y ago
  thanks for sharing.. its pretty awesome how you automated the test.. i would love to hear more from bitly folks or hipmunk folks about their experience on the performance. I am big fan of tornado, ben and it's simplicity. I guess i will continue to use my old branch for now.
  - Soupy 16y ago
    Just wanted to quickly clarify that I was not responsible for the test, all credit goes to Jacob on the tornado google group (http://groups.google.com/group/python-tornado/browse_thread/thread/94b45e815ac992ac# http://groups.google.com/group/python-tornado/browse_thread/...). Sorry for the misunderstanding.
  - tomwans 16y ago
    The Tornado framework is AMAZING for its simplicity. It gives you ONLY the things you'll need and a just a little bit extra. It's really great.
  - kn0thing 16y ago
    Thanks for mentioning this - I'm talking Steve/Chris into doing a tornado talk at pycon right this moment. This is great ammo.
    - j2d2j2d2 16y ago
      I believe Steve was one of the bigger pushes for the stack context stuff too. Mailing list suggests so anyway.
tptacek 16y ago
What a cool visualization! Between Github and a continuous integration benchmarking tool, you could generate this for any project. I would pay to have it.
zzzeek 16y ago
Here we see the movement from "Only 200 lines of code! Does 8 jillion requests per second!" to "Does things correctly, works in the real world!". Welcome to life.
- BarkMore 16y ago
  Looking at the data, it appears that the performance drops are related to the StackContext feature. This feature is a convenience and not about doing things correctly to work in the real world. Because the very earliest versions of Tornado were used by real websites, it looks like the code that did 8 jillion requests per second worked well enough in the real world.
- deleted 16y ago
  [deleted]
- bdarnell 16y ago
  That's often true, but doesn't appear to be the case here. This was a simple case of making invasive changes without quantifying the performance effects.
ams6110 16y ago
The link takes me to a Google Docs login page
- micheljansen 16y ago
  In my case (already logged in) it just tells me I "do not have permission to access this spreadsheet".
  - axod 16y ago
    Click on the link twice. Google docs has some weird bug at the moment afaik which causes this. (Worked for me the second click, probably some cookie set the first time).
- Soupy 16y ago
  Sorry about that, it appears like the document is getting hammered and is throwing some users to a login page or to a simplified spreadsheet view.
jaekwon 16y ago
Very cool. I'm using Tornado for my projects and I gotta say, I'm really enjoying the minimalism and design. I can dive into the source and tweak it fairly easily. Recently I tweaked the templating engine to accept keyword arguments in a namespace for the {% include %} command. Anyone interested in my patch?
- jaekwon 16y ago
  ... i don't understand why people downvote sometimes. weird. let's play a game. why would anyone downvote the parent comment?
  - mey 16y ago
    I didn't down vote your original comment, but my reactions are 1) Off-topic, what does your template patch have to do with the decline in performance? 2) If your patch is useful, submit it back to Tornado, or setup a fork, then just post a link
deleted 16y ago
[deleted]
rschildmeijer 16y ago
Don't miss the graph/chart (in the 'Tornado Performance Trend' Google doc). Yet another Tornado (and node.js and Deft) benchmark available on http://www.deftserver.org http://www.deftserver.org
krosaen 16y ago
glad to see Ben Darnell is on top of it: http://groups.google.com/group/python-tornado/msg/681856dbb421bec7 http://groups.google.com/group/python-tornado/msg/681856dbb4...
- MatthewRayfield 16y ago
  The fact that Ben jumped on this, made changes, AND commited them in less than a day gives me a lot of confidence as a Tornado user.
bdarnell 16y ago
Wow, didn't expect this to make the front page of HN. There's more context in the groups thread (http://groups.google.com/group/python-tornado/browse_thread/thread/94b45e815ac992ac http://groups.google.com/group/python-tornado/browse_thread/...), but the short version is that a change to abstract away some error handling with the python 'with' statement had a surprising performance impact. When used with the @contextlib.contextmanager decorator, the with statement is more than 25 times as expensive as a try/except statement.
- tomwans 16y ago
  yeah, I didn't know HN still cared about Tornado. I thought I was the only one using it for production ...
  - BarkMore 16y ago
    You and FriendFeed, Hipmunk, Brizzly, Bitly (they recently built async mongodb driver for use with Tornado), Quora (for long polling), ...
- petdog 16y ago
  It looks like you forgot to call the function in 'simple_catch'. Interestingly, if you do, manual_with_catch will actually be faster in the failing case. (A try:fn() finally: return True beats everything else in the failing case)
  - bdarnell 16y ago
    Ah, that makes a big difference. Thanks for the correction.
dlsspy 16y ago
This was the feedback from the twisted community when it launched. There were lots of issues they found in a cursory glance. My complaint (and I heard the same complaint from a developer as recently as yesterday) was that they tossed away all compatibility with everything that existed in order to solve one specific problem. Then people tried to use it as a general solution. I spent an hour or two soon after tornado was released figuring out how to take the good parts of tornado and the good parts of twisted and put them together for this reason: http://dustin.github.com/2009/09/12/tornado.html http://dustin.github.com/2009/09/12/tornado.html The result (without any attempt to optimize it) was a reducing in raw performance, but the ability to actually do ridiculous numbers of things you can't do in tornado even today.
- BarkMore 16y ago
  Tornado solves the specific problem of building web applications. What compatibility was tossed in accomplishing this goal? As far as I can tell, Tornado is compatible with HTTP, HTML and other related web standards. What are some of the things that you can do with your version of Tornado that I cannot do with regular Tornado?
  - dlsspy 16y ago
    * Use relational databases (the mysql samples shipped with tornado blocked all network traffic while mysql was performing a query). * Use non-relational databases (I have a couchdb interface for twisted, and a blocking one for regular python, nothing for tornado) * Use memcached (it's fast, but it blocks) * Make long-lived streaming http requests (cited in my blog post) * Spawn and communicate with subprocesses that might be not be instant * Schedule periodic execution of events (including http requests) * Run efficiently on BSDs and OS X * Send email * Do DNS lookups (important part of making service requests) * Use work queues (beanstalkd, gearman, ampq, etc...) These are things I do with twisted today (though I've got at least one app I've moved to node.js for small kind of small scripty sorts of thing). Again, you can run arbitrary python code, but if it blocks, it stops all traffic from flowing and new connections from coming in.
    - 16y ago
Groxx 16y ago
Image of the rendered graph (because it was going to raw data view due to how many people were viewing): http://imgur.com/YhTdg.jpg http://imgur.com/YhTdg.jpg The gray line represents the 2009 year marker.

64 points by Soupy 16y ago

Soupy 16y ago

This graph displays the average requests/second that the python tornado framework can handle graphed over time by commits. Very surprising and also a little concerning since I am relying on Tornado for my current projects. Just wanted to make sure that you all were made aware of this in case you are considering using Tornado for your own projects (although I have had nothing but positive experiences with it). Source - http://groups.google.com/group/python-tornado/browse_thread/thread/94b45e815ac992ac# http://groups.google.com/group/python-tornado/browse_thread/...

crad 16y ago
Thanks for posting this. I'm eager to see Ben and others reply to this on the list.
viksit 16y ago
Thats a great script you guys wrote! Any chance it's going to be open sourced? Would be really fun to play around with github projects!
SriniK 16y ago
thanks for sharing.. its pretty awesome how you automated the test.. i would love to hear more from bitly folks or hipmunk folks about their experience on the performance. I am big fan of tornado, ben and it's simplicity. I guess i will continue to use my old branch for now.
- Soupy 16y ago
  Just wanted to quickly clarify that I was not responsible for the test, all credit goes to Jacob on the tornado google group (http://groups.google.com/group/python-tornado/browse_thread/thread/94b45e815ac992ac# http://groups.google.com/group/python-tornado/browse_thread/...). Sorry for the misunderstanding.
- tomwans 16y ago
  The Tornado framework is AMAZING for its simplicity. It gives you ONLY the things you'll need and a just a little bit extra. It's really great.
- kn0thing 16y ago
  Thanks for mentioning this - I'm talking Steve/Chris into doing a tornado talk at pycon right this moment. This is great ammo.
  - j2d2j2d2 16y ago
    I believe Steve was one of the bigger pushes for the stack context stuff too. Mailing list suggests so anyway.

tptacek 16y ago

What a cool visualization! Between Github and a continuous integration benchmarking tool, you could generate this for any project. I would pay to have it.

zzzeek 16y ago

Here we see the movement from "Only 200 lines of code! Does 8 jillion requests per second!" to "Does things correctly, works in the real world!". Welcome to life.

BarkMore 16y ago
Looking at the data, it appears that the performance drops are related to the StackContext feature. This feature is a convenience and not about doing things correctly to work in the real world. Because the very earliest versions of Tornado were used by real websites, it looks like the code that did 8 jillion requests per second worked well enough in the real world.
deleted 16y ago
[deleted]
bdarnell 16y ago
That's often true, but doesn't appear to be the case here. This was a simple case of making invasive changes without quantifying the performance effects.

ams6110 16y ago

The link takes me to a Google Docs login page

micheljansen 16y ago
In my case (already logged in) it just tells me I "do not have permission to access this spreadsheet".
- axod 16y ago
  Click on the link twice. Google docs has some weird bug at the moment afaik which causes this. (Worked for me the second click, probably some cookie set the first time).
Soupy 16y ago
Sorry about that, it appears like the document is getting hammered and is throwing some users to a login page or to a simplified spreadsheet view.

jaekwon 16y ago

Very cool. I'm using Tornado for my projects and I gotta say, I'm really enjoying the minimalism and design. I can dive into the source and tweak it fairly easily. Recently I tweaked the templating engine to accept keyword arguments in a namespace for the {% include %} command. Anyone interested in my patch?

jaekwon 16y ago
... i don't understand why people downvote sometimes. weird. let's play a game. why would anyone downvote the parent comment?
- mey 16y ago
  I didn't down vote your original comment, but my reactions are 1) Off-topic, what does your template patch have to do with the decline in performance? 2) If your patch is useful, submit it back to Tornado, or setup a fork, then just post a link

deleted 16y ago

[deleted]

rschildmeijer 16y ago

Don't miss the graph/chart (in the 'Tornado Performance Trend' Google doc). Yet another Tornado (and node.js and Deft) benchmark available on http://www.deftserver.org http://www.deftserver.org

krosaen 16y ago

glad to see Ben Darnell is on top of it: http://groups.google.com/group/python-tornado/msg/681856dbb421bec7 http://groups.google.com/group/python-tornado/msg/681856dbb4...

MatthewRayfield 16y ago
The fact that Ben jumped on this, made changes, AND commited them in less than a day gives me a lot of confidence as a Tornado user.

bdarnell 16y ago

Wow, didn't expect this to make the front page of HN. There's more context in the groups thread (http://groups.google.com/group/python-tornado/browse_thread/thread/94b45e815ac992ac http://groups.google.com/group/python-tornado/browse_thread/...), but the short version is that a change to abstract away some error handling with the python 'with' statement had a surprising performance impact. When used with the @contextlib.contextmanager decorator, the with statement is more than 25 times as expensive as a try/except statement.

tomwans 16y ago
yeah, I didn't know HN still cared about Tornado. I thought I was the only one using it for production ...
- BarkMore 16y ago
  You and FriendFeed, Hipmunk, Brizzly, Bitly (they recently built async mongodb driver for use with Tornado), Quora (for long polling), ...
petdog 16y ago
It looks like you forgot to call the function in 'simple_catch'. Interestingly, if you do, manual_with_catch will actually be faster in the failing case. (A try:fn() finally: return True beats everything else in the failing case)
- bdarnell 16y ago
  Ah, that makes a big difference. Thanks for the correction.

dlsspy 16y ago

This was the feedback from the twisted community when it launched. There were lots of issues they found in a cursory glance. My complaint (and I heard the same complaint from a developer as recently as yesterday) was that they tossed away all compatibility with everything that existed in order to solve one specific problem. Then people tried to use it as a general solution. I spent an hour or two soon after tornado was released figuring out how to take the good parts of tornado and the good parts of twisted and put them together for this reason: http://dustin.github.com/2009/09/12/tornado.html http://dustin.github.com/2009/09/12/tornado.html The result (without any attempt to optimize it) was a reducing in raw performance, but the ability to actually do ridiculous numbers of things you can't do in tornado even today.

BarkMore 16y ago
Tornado solves the specific problem of building web applications. What compatibility was tossed in accomplishing this goal? As far as I can tell, Tornado is compatible with HTTP, HTML and other related web standards. What are some of the things that you can do with your version of Tornado that I cannot do with regular Tornado?
- dlsspy 16y ago
  * Use relational databases (the mysql samples shipped with tornado blocked all network traffic while mysql was performing a query). * Use non-relational databases (I have a couchdb interface for twisted, and a blocking one for regular python, nothing for tornado) * Use memcached (it's fast, but it blocks) * Make long-lived streaming http requests (cited in my blog post) * Spawn and communicate with subprocesses that might be not be instant * Schedule periodic execution of events (including http requests) * Run efficiently on BSDs and OS X * Send email * Do DNS lookups (important part of making service requests) * Use work queues (beanstalkd, gearman, ampq, etc...) These are things I do with twisted today (though I've got at least one app I've moved to node.js for small kind of small scripty sorts of thing). Again, you can run arbitrary python code, but if it blocks, it stops all traffic from flowing and new connections from coming in.
  - 16y ago

Groxx 16y ago

Image of the rendered graph (because it was going to raw data view due to how many people were viewing): http://imgur.com/YhTdg.jpg http://imgur.com/YhTdg.jpg The gray line represents the 2009 year marker.