5 ms·

Python vs Go - Requests per Second

37 points by dustinrcollins 13y ago

programminggeek 13y ago
People might beat up the benchmark or the methods, but in all honesty, Go should be an order of magnitude faster. It's compiled and runs about as fast as java in many benchmarks. I've seen some companies go from ruby to the jvm and have basically said 1 JVM instance = 10 ruby thin instances. So, when you think in that context, it makes total sense that go would be 10x faster than python for the same reasons that a compiled JVM language is 10x faster than ruby. I'd say from the numbers I've seen this benchmark is not an unreasonable display of the performance difference between go and python.
kbar13 13y ago
What about not using a framework for python?
- MetaCosm 13y ago
  or for Go, seems kinda pointless for both.
616c 13y ago
I know you put effort this and do not want to give you a hard time, but a lot of people on HN will give you a hard time with benchmarks, especially as simple as this, gracious concluding paragraph or not. Some people here are very dismissive of benchmarking in general, but if you are interested in evaluating programming languages for use in web applications, you would be interested in TechEmpower benchmarks, which are very thorough. [0] You can even send pull requests with updated languages, frameworks, or better implementations of different tests to give more competitive/accurate results. Enjoy. [0] http://www.techempower.com/benchmarks/ http://www.techempower.com/benchmarks/ [1] https://github.com/TechEmpower/FrameworkBenchmarks https://github.com/TechEmpower/FrameworkBenchmarks UPDATE: Fixed link to the benchmarks.
- chmike 13y ago
  I regret there isn't any benchmarks for d, especially vibe.d.
- bhauer 13y ago
  Thanks! I just want to clarify that the URL for the web application framework benchmarks project is: http://www.techempower.com/benchmarks/ http://www.techempower.com/benchmarks/ OP: I recommend installing and using Wrk for web-server benchmarks. The Apache Bench tool (ab) is single-threaded and with today's higher-performance frameworks and platforms, Apache Bench itself can be a limiting factor in tests. See a reply I wrote elsewhere [1] for more information. [1] https://news.ycombinator.com/item?id=6211883 https://news.ycombinator.com/item?id=6211883
devonnulled 13y ago
What about using alternative frameworks for HTTP servers for Python? Pointing at only Bottle and going "hey! Python is slower!" is like pointing at WeBRICK and going "Hey! Ruby is slower!" when you haven't even touched Passenger or Unicorn. What about: * Flask? http://flask.pocoo.org/ http://flask.pocoo.org/ * Tornado? http://www.tornadoweb.org/en/stable/ http://www.tornadoweb.org/en/stable/ * Django? https://www.djangoproject.com/ https://www.djangoproject.com/ * CherryPy? http://www.cherrypy.org/ http://www.cherrypy.org/ What about using PyPy ( http://pypy.org/ http://pypy.org/ ), Jython ( http://www.jython.org/ http://www.jython.org/ ) or even IronPython ( http://ironpython.net/ http://ironpython.net/ ) instead of plain CPython?
- no_l0gic 13y ago
  All good points, except the last... Does anybody use PyPy, Jython or IronPython for a production web service? I'm seriously wondering. I've had the misfortune of having had to use Jython, but just to read Python pickles from a Java service (don't ask). Even that simple use case wasn't fun (the last stable release was also over a year ago). PyPy looks great for small or new projects, but there is this disclaimer: "PyPy has alpha/beta-level support for the CPython C API, however, as of 2.1 release this feature is not yet complete. Many libraries will require a bit of effort to work..." - this has caused PyPy to be incompatible with every large Python project I've ever worked on. And well, personal (dis)taste in .NET aside, IronPython hasn't had a release in over a year either... Where are these projects being used in production?
  - continuations 13y ago
    Quora runs on PyPy: http://www.quora.com/Alex-Gaynor/Posts/Quora-is-now-running-on-PyPy http://www.quora.com/Alex-Gaynor/Posts/Quora-is-now-running-...
    - no_l0gic 13y ago
      Nope, not any more: http://www.quora.com/PyPy/Is-Quora-still-running-on-PyPy http://www.quora.com/PyPy/Is-Quora-still-running-on-PyPy (The answer you linked is quite old) I'm even more skeptical now that I see a company tried moving to PyPy, and actually hired a core PyPy developer (Alex Gaynor - http://pypy.org/people.html http://pypy.org/people.html) to make the move, and it still didn't work out.
ominous_prime 13y ago
I don't see any point in comparing a production quality web server with python's stdlib web server that is really only to be used for testing. It's not even worth discussing the other issues with this "benchmark".
- hboon 13y ago
  Bottle (http://bottlepy.org http://bottlepy.org) isn't part of Python stdlib.
  - mace 13y ago
    Bottle uses `wsgiref` (https://github.com/defnull/bottle/blob/master/bottle.py#L2639 https://github.com/defnull/bottle/blob/master/bottle.py#L263...) by default as its server. Per the name, it is a reference implementation of a WSGI server. No one should really use `wsgiref` for an application they care about. This test is invalid for this and lots of other reasons.
human_error 13y ago
This isn't Python vs. Go. This is bottle.py vs Go.
- jaegerpicker 13y ago
  no it's bottle.py vs go-restful, but so what? Change the packages for nearly any you want and the results will be similar. Go is faster at runtime and python is faster at development time. Pick which trade off you want and go with it.
  - no_l0gic 13y ago
    This: "[X] is faster at runtime and python is faster at development time." I wish more people knew / recognized / accepted this understanding of Python's goals. There is the old but still quite accurate: http://www.python.org/doc/essays/comparisons/ http://www.python.org/doc/essays/comparisons/ I'm not sure if Guido is the original author, but he at least comments on it. There are also quite a few excerpts going back to the origins of Python where Guido plainly states that the goals of the language are ease, brevity and developer productivity - I've never seen performance as a stated goal.
Demiurge 13y ago
In related news, I just measured Apples to be 100 times redder than Oranges! What is the point of benchmarking Python vs Go, when by design we all know Python is slower? So, in this particular case it's 100 times, for completely random reasons. Speed is not why I use Python every day and absolutely have no need for Go, although I wish it would magically replace C++.
- chmike 13y ago
  I would say because of the rule "if it works, don't fix it". But a factor 10 in performance means potential server cost reduction by a factor 10. I think this is the purpose of this blog post.
  - Demiurge 13y ago
    Again, we already know any strongly typed language can be 100, 1000 times faster. By design. Raw request throughput is not why anyone should be using python. This is why we already have caching and nginx. No one who is using python going to go "aha, scrap this piece of system and rewrite it Go, we will reduce our hardware costs 10 times!" because it is just so much more complicated than that.
    - chmike 13y ago
      To me it's that. Go is compiled to native and the constrain of strongly typed language is another optimization. A ramen profitable startup should make the jump because it can make the difference. Of course there are some optimization strategies available for pythons, but you have infrastructure complexity depth and so many places where things can break. Beside the optimizations apply only to what can be optimized (static page cache). Python has a fast code/test/run loop cycle, but Go and D have that now as well with the additional benefit of being fast. These are the reasons why I decided to leave Python and go for the Go or D route. And I don't see why Go or D couldn't provide a web server as fast as nginx one day.
      - Demiurge 13y ago
njharman 13y ago
I challenge the assumption that being faster is reason enough to check out X.
- saosebastiao 13y ago
  I challenge the assumption that valid reasons to check out X can be summarized authoritatively by one person.
pjmlp 13y ago
What is the sense of all these Python vs Go runtime benchmarks? Unless one compares implementations of the same nature, it is obvious that the compiled one, in this case Go, it will always be faster. Why bother at all to find out the obvious?!
est 13y ago
Okay, this is yet another weekly HN "Hello world http benchmark on a Macbook Pro" series. I'd bite. The following Python snippet seems much faster than OP's Go code under the same ab testing. HTTP server: https://github.com/jonashaag/bjoern https://github.com/jonashaag/bjoern import bjoern, json def home_app(environ, start_response): start_response('200 OK', [('Content-Type', 'text/plain')]) s = {'name': 'A Royal Baby', 'body':'A slow news week'} yield json.dumps(s) bjoern.run(home_app, '127.0.0.1', 8086)
userulluipeste 13y ago
Google supported Python a while ago. Now, after knocking off its new "Go" toy, started to kick in the nuts the tagged-"lousy kid" Python. Bad attitudes aside, Google would better find another strategy to bolster its offspring. Making a reputation by mocking something else is not something that even a language like Go wants to rely on.
- mseepgood 13y ago
  What does this blog post have to do with Google?
  - userulluipeste 13y ago
    <sarcasm>Well, when I see Go-promoting pieces like this I somehow feel around Google's love ...you know?</sarcasm> If you don't agree with me then you may blame me for talking without strong and direct facts (without down-votes I hope, but I'll take them if they will), and I'll accept that and stand down for a while. After all, this Go thing got in my attention only recently and I didn't bothered to gather Go/Google "having to do with" material. If you are however, a Google-related person, notice that there is much too much Go around, at least here on HN. Maybe it's better to tune it down a little bit, otherwise it might induce a negative effect.
e12e 13y ago
While I'm not convinced this is "python" vs "go" (see other comments) what everyone seems to miss is that this is 9 lines vs 23, as well. Ok, so in this test, the performance/line is off by a factor of five (2 times the lines, ten times the performance) -- but it looks to me like the performance could be made up by sticking something like Varnish in front -- so there's really no "real world" performance difference, just more complex code. I may be wrong, but that's all I see from this. (Assuming architecture >> micro benchmarks)
handsomegui 13y ago
Check out this. Tested by GregBowyer: https://gist.github.com/GregBowyer/6265715 https://gist.github.com/GregBowyer/6265715

People might beat up the benchmark or the methods, but in all honesty, Go should be an order of magnitude faster. It's compiled and runs about as fast as java in many benchmarks. I've seen some companies go from ruby to the jvm and have basically said 1 JVM instance = 10 ruby thin instances. So, when you think in that context, it makes total sense that go would be 10x faster than python for the same reasons that a compiled JVM language is 10x faster than ruby. I'd say from the numbers I've seen this benchmark is not an unreasonable display of the performance difference between go and python.

kbar13 13y ago

What about not using a framework for python?

MetaCosm 13y ago
or for Go, seems kinda pointless for both.

616c 13y ago

I know you put effort this and do not want to give you a hard time, but a lot of people on HN will give you a hard time with benchmarks, especially as simple as this, gracious concluding paragraph or not. Some people here are very dismissive of benchmarking in general, but if you are interested in evaluating programming languages for use in web applications, you would be interested in TechEmpower benchmarks, which are very thorough. [0] You can even send pull requests with updated languages, frameworks, or better implementations of different tests to give more competitive/accurate results. Enjoy. [0] http://www.techempower.com/benchmarks/ http://www.techempower.com/benchmarks/ [1] https://github.com/TechEmpower/FrameworkBenchmarks https://github.com/TechEmpower/FrameworkBenchmarks UPDATE: Fixed link to the benchmarks.

chmike 13y ago
I regret there isn't any benchmarks for d, especially vibe.d.
bhauer 13y ago
Thanks! I just want to clarify that the URL for the web application framework benchmarks project is: http://www.techempower.com/benchmarks/ http://www.techempower.com/benchmarks/ OP: I recommend installing and using Wrk for web-server benchmarks. The Apache Bench tool (ab) is single-threaded and with today's higher-performance frameworks and platforms, Apache Bench itself can be a limiting factor in tests. See a reply I wrote elsewhere [1] for more information. [1] https://news.ycombinator.com/item?id=6211883 https://news.ycombinator.com/item?id=6211883

devonnulled 13y ago

What about using alternative frameworks for HTTP servers for Python? Pointing at only Bottle and going "hey! Python is slower!" is like pointing at WeBRICK and going "Hey! Ruby is slower!" when you haven't even touched Passenger or Unicorn. What about: * Flask? http://flask.pocoo.org/ http://flask.pocoo.org/ * Tornado? http://www.tornadoweb.org/en/stable/ http://www.tornadoweb.org/en/stable/ * Django? https://www.djangoproject.com/ https://www.djangoproject.com/ * CherryPy? http://www.cherrypy.org/ http://www.cherrypy.org/ What about using PyPy ( http://pypy.org/ http://pypy.org/ ), Jython ( http://www.jython.org/ http://www.jython.org/ ) or even IronPython ( http://ironpython.net/ http://ironpython.net/ ) instead of plain CPython?

no_l0gic 13y ago
All good points, except the last... Does anybody use PyPy, Jython or IronPython for a production web service? I'm seriously wondering. I've had the misfortune of having had to use Jython, but just to read Python pickles from a Java service (don't ask). Even that simple use case wasn't fun (the last stable release was also over a year ago). PyPy looks great for small or new projects, but there is this disclaimer: "PyPy has alpha/beta-level support for the CPython C API, however, as of 2.1 release this feature is not yet complete. Many libraries will require a bit of effort to work..." - this has caused PyPy to be incompatible with every large Python project I've ever worked on. And well, personal (dis)taste in .NET aside, IronPython hasn't had a release in over a year either... Where are these projects being used in production?
- continuations 13y ago
  Quora runs on PyPy: http://www.quora.com/Alex-Gaynor/Posts/Quora-is-now-running-on-PyPy http://www.quora.com/Alex-Gaynor/Posts/Quora-is-now-running-...
  - no_l0gic 13y ago
    Nope, not any more: http://www.quora.com/PyPy/Is-Quora-still-running-on-PyPy http://www.quora.com/PyPy/Is-Quora-still-running-on-PyPy (The answer you linked is quite old) I'm even more skeptical now that I see a company tried moving to PyPy, and actually hired a core PyPy developer (Alex Gaynor - http://pypy.org/people.html http://pypy.org/people.html) to make the move, and it still didn't work out.

ominous_prime 13y ago

I don't see any point in comparing a production quality web server with python's stdlib web server that is really only to be used for testing. It's not even worth discussing the other issues with this "benchmark".

hboon 13y ago
Bottle (http://bottlepy.org http://bottlepy.org) isn't part of Python stdlib.
- mace 13y ago
  Bottle uses `wsgiref` (https://github.com/defnull/bottle/blob/master/bottle.py#L2639 https://github.com/defnull/bottle/blob/master/bottle.py#L263...) by default as its server. Per the name, it is a reference implementation of a WSGI server. No one should really use `wsgiref` for an application they care about. This test is invalid for this and lots of other reasons.

human_error 13y ago

This isn't Python vs. Go. This is bottle.py vs Go.

jaegerpicker 13y ago
no it's bottle.py vs go-restful, but so what? Change the packages for nearly any you want and the results will be similar. Go is faster at runtime and python is faster at development time. Pick which trade off you want and go with it.
- no_l0gic 13y ago
  This: "[X] is faster at runtime and python is faster at development time." I wish more people knew / recognized / accepted this understanding of Python's goals. There is the old but still quite accurate: http://www.python.org/doc/essays/comparisons/ http://www.python.org/doc/essays/comparisons/ I'm not sure if Guido is the original author, but he at least comments on it. There are also quite a few excerpts going back to the origins of Python where Guido plainly states that the goals of the language are ease, brevity and developer productivity - I've never seen performance as a stated goal.

Demiurge 13y ago

In related news, I just measured Apples to be 100 times redder than Oranges! What is the point of benchmarking Python vs Go, when by design we all know Python is slower? So, in this particular case it's 100 times, for completely random reasons. Speed is not why I use Python every day and absolutely have no need for Go, although I wish it would magically replace C++.

chmike 13y ago
I would say because of the rule "if it works, don't fix it". But a factor 10 in performance means potential server cost reduction by a factor 10. I think this is the purpose of this blog post.
- Demiurge 13y ago
  Again, we already know any strongly typed language can be 100, 1000 times faster. By design. Raw request throughput is not why anyone should be using python. This is why we already have caching and nginx. No one who is using python going to go "aha, scrap this piece of system and rewrite it Go, we will reduce our hardware costs 10 times!" because it is just so much more complicated than that.
  - chmike 13y ago
    To me it's that. Go is compiled to native and the constrain of strongly typed language is another optimization. A ramen profitable startup should make the jump because it can make the difference. Of course there are some optimization strategies available for pythons, but you have infrastructure complexity depth and so many places where things can break. Beside the optimizations apply only to what can be optimized (static page cache). Python has a fast code/test/run loop cycle, but Go and D have that now as well with the additional benefit of being fast. These are the reasons why I decided to leave Python and go for the Go or D route. And I don't see why Go or D couldn't provide a web server as fast as nginx one day.
    - Demiurge 13y ago

njharman 13y ago

I challenge the assumption that being faster is reason enough to check out X.

saosebastiao 13y ago
I challenge the assumption that valid reasons to check out X can be summarized authoritatively by one person.

pjmlp 13y ago

What is the sense of all these Python vs Go runtime benchmarks? Unless one compares implementations of the same nature, it is obvious that the compiled one, in this case Go, it will always be faster. Why bother at all to find out the obvious?!

est 13y ago

Okay, this is yet another weekly HN "Hello world http benchmark on a Macbook Pro" series. I'd bite. The following Python snippet seems much faster than OP's Go code under the same ab testing. HTTP server: https://github.com/jonashaag/bjoern https://github.com/jonashaag/bjoern import bjoern, json def home_app(environ, start_response): start_response('200 OK', [('Content-Type', 'text/plain')]) s = {'name': 'A Royal Baby', 'body':'A slow news week'} yield json.dumps(s) bjoern.run(home_app, '127.0.0.1', 8086)

userulluipeste 13y ago

Google supported Python a while ago. Now, after knocking off its new "Go" toy, started to kick in the nuts the tagged-"lousy kid" Python. Bad attitudes aside, Google would better find another strategy to bolster its offspring. Making a reputation by mocking something else is not something that even a language like Go wants to rely on.

mseepgood 13y ago
What does this blog post have to do with Google?
- userulluipeste 13y ago
  <sarcasm>Well, when I see Go-promoting pieces like this I somehow feel around Google's love ...you know?</sarcasm> If you don't agree with me then you may blame me for talking without strong and direct facts (without down-votes I hope, but I'll take them if they will), and I'll accept that and stand down for a while. After all, this Go thing got in my attention only recently and I didn't bothered to gather Go/Google "having to do with" material. If you are however, a Google-related person, notice that there is much too much Go around, at least here on HN. Maybe it's better to tune it down a little bit, otherwise it might induce a negative effect.

e12e 13y ago

While I'm not convinced this is "python" vs "go" (see other comments) what everyone seems to miss is that this is 9 lines vs 23, as well. Ok, so in this test, the performance/line is off by a factor of five (2 times the lines, ten times the performance) -- but it looks to me like the performance could be made up by sticking something like Varnish in front -- so there's really no "real world" performance difference, just more complex code. I may be wrong, but that's all I see from this. (Assuming architecture >> micro benchmarks)

handsomegui 13y ago

Check out this. Tested by GregBowyer: https://gist.github.com/GregBowyer/6265715 https://gist.github.com/GregBowyer/6265715