15 ms·
Million requests per second with Python
- zepolen 10y agoCouldn't get this to work on osx, initially got an error from strip in src/picohttpparser/build - which was fixed by running `strip -x libpicohttpparser.so` which got `python setup.py develop` to work. But then running the demo app resulted in: ImportError: dlopen(/tmp/japronto/src/japronto/protocol/cprotocol.cpython-35m-darwin.so, 2): Library not loaded: libpicohttpparser.so Referenced from: /tmp/japronto/src/japronto/protocol/cprotocol.cpython-35m-darwin.so Reason: image not found Didn't bother after that
- Anderkent 10y ago>HTTP pipelining is crucial here since it’s one of the optimizations that Japronto takes into account when executing requests. https://en.wikipedia.org/wiki/HTTP_pipelining https://en.wikipedia.org/wiki/HTTP_pipelining > Of all the major browsers, only Opera based on Presto layout engine had a fully working implementation that was enabled by default. In all other browsers HTTP pipelining is disabled or not implemented.[3] >Internet Explorer 8 does not pipeline requests, due to concerns regarding buggy proxies and head-of-line blocking.[7] >Internet Explorer 11 does not support pipelining. [8] >Mozilla browsers (such as Mozilla Firefox, SeaMonkey and Camino) support pipelining; however, it is disabled by default.[9][10] Pipelining is disabled by default to avoid issues with misbehaving servers.[11] When pipelining is enabled, Mozilla browsers use some heuristics, especially to turn pipelining off for older IIS servers.[12] >Konqueror 2.0 supports pipelining, but it's disabled by default.[citation needed] >Google Chrome previously supported pipelining, but it has been disabled due to bugs and problems with poorly behaving servers.[13] That seems like optimizing for a benchmark.
- willvarfar 10y agoI wish there was a header a server could send to declare it supports pipelining.
- OJFord 10y ago"HTTP/1.1"
- burnbabyburn 10y agoheadshot
- sametmax 10y agoYeah. More like I wish implementors would follow standard.
- kevin_thibedeau 10y agoPipelining support is optional. It's up to the client to back off if its attempts fail. 8.1.2.2 Clients MUST also be prepared to resend their requests if the server closes the connection before sending all of the corresponding responses.
- OJFord 10y ago> HTTP/1.1 conforming servers are required to support pipelining. This does not mean that servers are required to pipeline responses, but that they are required to not fail if a client chooses to pipeline requests. http://www-archive.mozilla.org/projects/netlib/http/pipelining-faq.html http://www-archive.mozilla.org/projects/netlib/http/pipelini... 8.1.2.2 that you quoted [RFC2616] continues: > A server MUST send its responses to those requests in the same order that the requests were received.
- dorianm 10y agoWould it make make sense to optimize for HTTP/2 then?
- kid0m4n 10y agoI checked the benchmark code out. I have asked the author for more information. I am not sure if all the tests were limited to using only 1 core during the run. GOMAXPROCS=1 severely limits the Go version. 1 Go worker process can utilise all the cores in the server. I just want to get that clarified.
- dom96 10y agoFrom the article: > To be fair all the contestants (including Go) were running single worker process That seems to imply that they were all limited to a single core.
- jsd1982 10y agoA single worker process sounds like what it says it is. A single process containing possibly many OS threads. Just because it's a single process doesn't mean it won't use all cores. Where's the implication that process = core? GOMAXPROCS defaults to the number of cores in the system.
- floatboth 10y agoGOMAXPROCS is the number of cores used.
- knivets 10y agoI think, one of the main points of writing a framework in Python or another high-level language is the ease of extension in the future by the community.
- pekk 10y agoWhere are the "ease of extension" benchmarks? What kind of data do people look at when they want to prove their language is better, or choose the best language?
- throwawayish 10y agoEvidently there's at least one group of developers who mainly derive their decisions from irrelevant and often incorrectly conducted micro-benchmarks.
- tuyguntn 10y agoDjango needs this kind of performance boost, without depending on gunicorn/uwsgi servers
- Notre1 10y agoWhy would you prefer to depend on this (Japronto), instead of Gunicorn or uWSGI?
- burnbabyburn 10y agoI don't thing for now those are comparable, uWsgi is a LOT more than what a 0.1 release is.
- xyzzy123 10y agoNothing will save django for free because if you used that your most likely real problem is scaling postgres or mysql. Use as many appservers as your customer base will pay you for. Basically, cry in your beer because you're so rich and successful now that the framework won't handle your stuff.
- scriptkiddy 10y agoAgreed. Scaling Django isn't difficult really. Just spin up another instance if you need to. The major problem is scaling the DB. Making sure you're not bogging Postgres down by making it have to deal with possible race conditions is difficult.
- est 10y agouWSGI has very decent HTTP/1.1 performance http://uwsgi-docs.readthedocs.io/en/latest/HTTP.html http://uwsgi-docs.readthedocs.io/en/latest/HTTP.html
- danielovichdk 10y agoWhen did anyone start to believe that languages scale ?
- burnbabyburn 10y agowhat do you mean?
- fb03 10y agoI think he meant, in a slightly acid (but valid) way, that languages don't scale, but concepts and good programming techniques do. You can scale up whatever code you want in whatever language, you just need to structure it in a way that allows for that.
- GrayShade 10y agoIt might be a variant of the old argument that "languages aren't slow, implementations are", meaning that scalability is a property of the language, not the implementation. In any case, this is written in C so I'm not sure it proves anything about the Python language.
- pekk 10y agoIt says something about the performance that is available from the Python language. This is how Python has always been, starting from the interpret and builtin data structures in CPython which are written in C. That isn't cheating, that's normal, realistic usage of the tool. Why should benchmarks go out of their way to use only unrealistic Python programs just to prove that it cannot "x += 1" as fast as C? Who is surprised by that, and why would it matter? Node uses libuv and nobody blinks an eye.
- deleted 10y ago[deleted]
- mjolk 10y agoI like the cute expression that you're referencing, but for a majority of projects, to a major extent, language choice greatly matters. If you're getting 10x performance from Go over RoR, that's 10 fewer servers you need to deal with and a lot less thinking about network concurrency.
- ecopoesis 10y agoA million requests per second in Python, if you write all the code that does work in C.
- andy_ppp 10y agoAnd as little actual python code in the routes as possible...
- bpicolo 10y agoAnd then you get all the fun of debugging clang internals when things go wrong.
- botverse 10y agoYou use it in Python. OP writes the C code for you.
- stefantalpalaru 10y agoYes, but all the performance related praises should go to C.
- pg314 10y agoAs soon as you do anything else than return 'hello, world' in Python, your performance will drop dramatically. Just parsing a tiny JSON string a million times in python takes 7s (i5-2467M). $ time python -c "import json; for i in range(0, 1000000): json.loads('{\"data\": \"hello, world\"}')" real 0m7.055s user 0m6.951s sys 0m0.063s It is no use optimising one part of a system if other parts take 99% of the time. As it stands, the title of the post would be more accurately stated as 'million requests per second with C'.
- SEJeff 10y agoFWIW, you can use ujson, which is much faster than the builtin stdlib json parser as well. I've seen some larger webapps in python do just this.
- 10y ago
- xyzzy123 10y agoYeah the hard part is having customers. If you have that, PHP, 5 layers of proxies, worst case latency, who cares? The pig can be made to fly at that point. "optimised for loopback!" - which is nobody's actual problem. It's a cool thing though, just cynical I guess because it doesn't solve my actual problems which are all self inflicted and organisational. Thinking about deleting this comment because it would logically apply to every incremental technical advance :(
- VMG 10y agoI don't think it's cynical at all. These edge-case optimizations really are just entertainment. The author's claim that the other http servers are deficient is misleading.
- deleted 10y ago[deleted]
- gjvc 10y agothis is a ruse. all the code which does the work in this is in C.
- deleted 10y ago[deleted]
- Spiritus 10y agoSo? The API is still Python. How do think numpy got so popular?
- gjvc 10y agoas someone else further down this thread said "It matters because all the heavy lifting is done by C and has little if anything to do with Python itself." I'm getting slammed for my previous comment!
- CorvusCrypto 10y agoI think the difference here is that numpy is not flaunting its speed as a benefit of using python. They are very open that they were bringing the performance of BLAS and LAPACK to python.
- xorcist 10y agoThis looks like a contrived benchmark to me. If that's really your use case, use nginx instead. The http parser might be useful, but it's not very well documented and not super readable at first glance. I do prefer not to parse http at every layer if I can but other people seems to like it. Along those lines, what happened to uwsgi async mode? uwsgi has been rock solid for me, with a straightforward binary framing and hot loading, and it would be great to use it for the (admittedly, few) cases where async might be useful. It seems like one of those obvious good ideas that no one uses?
- deleted 10y ago[deleted]
- Mayzie 10y agoITT people complaining that the underlying library is written in C. Who cares? The author wrote the C code and Python bindings for his library so we can _all_ benefit from handling more requests per second just by writing Python code and using his framework. I appreciate the work and will be exploring its potential use. Thank you squeaky_pl for writing it.
- jsd1982 10y agoIt matters because all the heavy lifting is done by C and has little if anything to do with Python itself. You could give the same treatment to virtually any other host scripting language (Ruby, Lua, PHP) and then make outlandish claims that that host language is now capable of ridiculous things when it had nothing to do with the alleged success other than that it exposes a FFI that lets you create something like this.
- SEJeff 10y agoYour same argument can be applied to virtually any other language: This ruby benchmark doesn't matter because ruby is written in C! This nodejs benchmark doesn't matter because v8 is written in C/C++! etc.
- mattnewton 10y agoI think his argument is that ymmv as you add more python code to this native-heavy benchmark.
- throwawayish 10y agoITT: Web developers being surprised by the meaninglessness of microbenchmarks.
- jsd1982 10y agoNo, we're not surprised by microbenchmarks. We're surprised by the amount of investment in trying to improve the foundations of toppling-over buildings when that investment could be better spent elsewhere.
- floatboth 10y agoThe project is very interesting but the benchmark results must be completely wrong. On my machine, using https://github.com/rakyll/hey https://github.com/rakyll/hey to test: 1 concurrent request: node.js does ~8000 req/second, go does ~9000, japronto does ~10000 10 concurrent requests: node.js does ~24000 req/second, go does ~45000, japronto does ~55000 Nowhere near a million. That's without pipelining though. Can pipelining really add SO MUCH performance? UPDATE: tried wrk with pipelining. japronto does 700000, golang 150000, node 35000. Holy shit, pipelining is epic. But the benchmark is not fair when only one server supports pipelining.
- nhumrich 10y agoYes, because the real bottleneck is TCP. Pipelining reduces number of packets sent. However, very few clients actually support pipelining, so it's almost useless
- floatboth 10y agoYeah, HTTP/1.1 pipelining is useless. But the good news is, HTTP/2 is pipelined (and doesn't have the head-of-line blocking problem). And HTTP/2 is supported by modern browsers. But anyway… HTTP/2, or 1.1 pipelining for that matter, is usually terminated at the reverse proxy level. So it's really not necessary in a web framework! Just makes these unfair benchmark results possible.
- cuu508 10y agoAlso, it's measuring the number of static responses to a single client. I guess there probably is an use case for this, but a more typical situation is many clients making one or a few requests, and also doing the TLS handshake.
- jsd1982 10y agoI'd rather stick with Golang here because it's not hiding anything. Everything in the Go stack is written in Go, and all your http handler (business) logic is written in Go. With this mix of C and Python it's impossible to tell what will happen to the system's performance when you actually include significant amounts of business logic written in Python on top of the C framework here. The author even makes a point about how little actual Python code and/or data structures are in use. If use of the host language is discouraged in the framework, how can I trust the performance of the framework with my code written in the host language? I bet if you put anything non-trivial on top and try to connect with real-world http clients which mostly don't use pipelining, it will fall over hard. Enjoy debugging what you can't understand. The emphasis on http parsing using SSE intrinsics is odd, as http parsing is rarely a bottleneck. I'm not saying it can't be, but that'll only be when you've got the rest of your stack highly tuned, performance is predictable, and profiling has shown that http parsing is your actual bottleneck. Even so, http/2 in theory alleviates this bottleneck and you don't have to use processor-specific intrinsics, as well as providing a better solution to pipelining problems with its multiplexing of requests. EDITED: "how can I trust the performance of the framework with my code written in the host language?" Original text lacked emphasized addition.
- mmgutz 10y agoWhen something is too good to be true it probably is not. Even if it was written in assembly these are dubious results. The graph alone tells me the quality of the claims.
- 79d697i6fdif 10y agoHow does this get to the front page? Has nobody seen this? https://www.techempower.com/benchmarks/#section=data-r13 https://www.techempower.com/benchmarks/#section=data-r13 Much more thorough than this toy benchmark article, which reads like marketing release...
- edraferi 10y agoI certainly hadn't. Great site.
- pekk 10y agoIf you work out a way to model the performance of your own systems based on these benchmarks, then on the way you will discover what is wrong with them. Or switch to Urweb for all your future sites, I guess?
- 79d697i6fdif 10y agothe source to all the benchmark is available https://github.com/TechEmpower/FrameworkBenchmarks https://github.com/TechEmpower/FrameworkBenchmarks If you want to make your chosen platform fast you can see what they're doing in the src to get those results. The ones not marked "stripped" and with "full" for ORM are very realistic examples. Not sure where you're going with this? Nobody's real workload looks exactly like benchmarks.
- rlarroque 10y agoHave you tried load testing it with Gatling (http://gatling.io/ http://gatling.io/) which seems to provide more accurate results including potential hicups you can't see with other load testing framework?
- sandGorgon 10y agoCan this be retrofitted into flask (just the performance improvements) ? I would pay for that.. a lot (as much as I can afford). And I'm pretty sure so would a lot of others.
- brilliantcode 10y agoI'd also pay for this to happen.
- voidlogic 10y ago>wrk with 1 thread, 100 connections and 24 simultaneous Say what? When I do this kind of testing using wrk: 1. I use multiple clients 2. Each client uses a thread for every logical CPU 3. I usually use at least 256 connections What is actually being tested here? This smells like a test to get a desired result. No to mention in regards to Go/Others: 1. He is using an RC 2. By default it uses all all cores, why handicap it? 3. He didn't share his code for his contestants, how can I trust this?
- floatboth 10y agoThe contestants are in the repo. One core is used "to be fair" (the Python stuff only uses one core). But the test, yeah. What's being tested is the presence of pipelining :D
- voidlogic 10y ago>What's being tested is the presence of pipelining :D Which is silly considering a majority of wild browsers now support HTTP/2. Thanks for pointing our the repo, I missed it before: https://github.com/squeaky-pl/japronto https://github.com/squeaky-pl/japronto His Go isn't horrible, but it could be written like this: func hello(respWtr http.ResponseWriter, req *http.Request) { text, status := []byte("Hello world!"), http.StatusOK if request.URL.Path != "/" { text, status = []byte("Not Found"), http.StatusNotFound } respWtr.WriteHeader(status) response.Write(text) } Rather than: https://github.com/squeaky-pl/japronto/blob/master/benchmarks/golang/micro.go https://github.com/squeaky-pl/japronto/blob/master/benchmark... Also, limiting Go to use a single core is unfair because the Go GC is optimized for multi-core operation.
- myf01d 10y agoThe current situation of Python's performance is really pathetic. Thanks for the project, we need to make Python great again otherwise the future is for Go, Elixir and Rust.
- sametmax 10y agoPeople are obsessed with performances, while my experience is that for the enormous vast majority of projects you could run on something 10 times slower than Python. Actually, I have a friend of mine running a streaming website. He banks 15k€ a month for 400k daily users. Python perf has zero inpact on his project. Python ease of use and maintainability means he, as a terrible (really terrible) programmer have been able to keep the site up and running for 5 years. I worked in africa for a while. Needs for Python perf ? Zero. Needs for code that is understandable and easy to write. A LOT. Moved to geographes working on SIG. Needs for Python perf ? Hell the GIS engine does the heavy work, they don't care. But they don't undestand jack about programming and need things done. Python let them to do it cleanly. Or I moved to embeded programmers. Hardcore C coders that are using Python to test their boards. Need for Python perf ? They don't even know what you are talking about. For them perf is assembly. They want something that is fast to write, with 10000 libs available. As a trainer and dev I teach Python on a monthly basis. JS training. Jeez, where do I begin ? If you let them do something more than a one-file stuff, they are lost. Python ? They are project ready in no time. Data mongers ? They have numpy and databases. Now myself ? I code in Python all the time. I've seen many slow sites. It's always either because of the DB queries or the static assets. Never about the server side language, be it Python, JS, Ruby or PHP. Coding in Go, Elixir or Rust for task where my productivity is more important than the machine output makes no sense. Especially in a world where an unmetered VPS cost 3€/month (https://www.ovh.com/us/vps/ https://www.ovh.com/us/vps/) with 2 GB RAM and 10 GB SSD. I can just take 100 of those and build a freaking skynet without making my company accountant blink. We are not google or facebook (and they do use Python BTW. Fb just announced they are migrating to 3 massively). So yes, more perf is nice. I would literally pay to help having better perf because it's always a good thing. But stop pretending you need it so bad, because you would be magically the 0.1% that actually do on all the forum post where people are saying they are. It's a nice to have. If you want to switch to something else, just admit you are following a trend. Dev always are. They love what's new and shiny. Nothing wrong about it. But those "Python is slow I'm out" posts are ridiculous. Dropbox can say that. Bank of America can say that.
- nerdponx 10y agoSorta unrelated, but the author made an interesting claim that PyPy will reach 3.5 "conformance" in 2017 in the context of talking about NumPy. Does that mean PyPy is expected to work seamlessly with C extensions like those used in SciPy or generated by Cython?
- molind 10y agoThere is no pure python out there. It's python with bindings to the highly optimized C code. It would be fair to say it's C gives you almost million of RPS in some tweaked tests.
- jjawssd 10y agoIt's worth considering Python asyncio as well for this type of work https://magic.io/blog/asyncpg-1m-rows-from-postgres-to-python/ https://magic.io/blog/asyncpg-1m-rows-from-postgres-to-pytho...
- Xorlev 10y agoThis is a great project. However, it not a very useful result (1M RPS in Python). Once you start layering more work into your handlers (JSON deserialization, admittedly C-optimized, business logic, IO, etc.), you can be assured that your HTTP serving layer is not the bottleneck.
- vgy7ujm 10y agoNot so sure that I would want to use Python. For anything.
- vgy7ujm 10y agoSome very petulant pythonista downvoters here. Thinking that no one should have a different opinion than themselves. Actually that profile fits the language... only one way... sad people.
- NathanKP 10y ago> To be fair all the contestants (including Go) were running single worker process. This makes the benchmark results much less impressive. Sure its cool that this micro framework allows you to serve a million requests per second with a single process but no one is going to be running a single process per server in production if they value their uptime.
- red2awn 10y agoWho cares about Hello world benchmark?
- owaislone 10y agoI love Python and use it as the main tool at my day job but I have to call it out. > It lets you do synchronous and asynchronous programming with asyncio and it’s shamelessly fast. Even faster than NodeJS and Go. and then, > To be fair all the contestants (including Go) were running single worker process. What is the point of such a benchmark? It is completely useless. You are deliberately crippling other tech before comparing your tech with them. What am I missing here HN?
- dvirsky 10y agoIf I'm reading this correctly, and he set GOMAXPROC to 1, that means that Go uses one system thread, but it still serves many requests concurrently, as there is an asynchronous IO loop under the hood. Same goes for Node.
- throwawayish 10y agoWhile I certainly don't want to give this sort of "benchmarking" any credibility at all: allocating significantly different compute resources for different samples would be at least as laughable (eg. giving Go a dozen cores [per it's defaults] but constraining Python to one worker [per it's defaults]).
- tomc1985 10y agoI was impressed til he revealed most of Japronto is C... so much for learning of some new nifty Python technique :/
- znpy 10y agoA million requests per second with Python. Spoiler: technically, it's not "with Python".
- shanemhansen 10y agoI wonder how it would compare to the fasthttp libary https://github.com/valyala/fasthttp https://github.com/valyala/fasthttp I don't suppose the hand tuned microhttp parser supports h2? Pipelining is a dead technology that was never really alive. For those who aren't familiar with it, it's based on a http "hack" where after http/1.1 was released and response end could be signaled via something other than EOF on the tcp connection they figure you can send n requests in a row and receive n responses in a row. This was in the days where good manners required you make no more than 2 simultaneous requests to a server. Nobody ever really supported it, and the lack of support for out of order responses meant that any one slow request could slow down everything else. The h2 model allows at least hundreds of simultaneous requests and responses multiplexed over one TCP stream.
- mixmastamyk 10y agoGreat work. The name looks to be Brazilian PT for "already ready," aka now!
- edsiper2 10y agoHTTP Pipelining should not be the base for a performance benchmark. It's an unrealistic scenario. To make the benchmark a bit fair, add a graphic of memory usage, pipelining might have a huge impact on resources usage in the server side.
- jchook 10y agoJust want to say that this post was inspiring to me. Thanks for writing it in such detail! Sounds like you have a long and fun road ahead of you with this project.
- mvindahl 10y agoTrying to grasp the concepts here. But if I understand pipelining correctly, it's a great boost for a single browser making a million requests but won't really help you in dealing with a million browsers firing one request each. If that is correctly understood then the benchmark is highly artificial. But yeah, feel free to correct me if I got it wrong.