12 ms·
Uvloop: Fast Python networking
- 1st1 10y agoI'm the dev behind uvloop. AMA.
- plainOldText 10y agoWhat are some of the non-obvious limitations/peculiarities one should be aware before using this?
- 1st1 10y agoWherever you use asyncio, it should be safe to just start using uvloop (once we graduate it from beta). uvloop shouldn't behave any differently, I've paid special attention to make sure it works exactly the same way as asyncio (down to when its objects are garbage collected).
- ruffrey 10y ago> at least 2x faster than nodejs, gevent, as well as any other Python asynchronous framework I did not see any benchmarks in the repo to support this. How was this statistic determined?
- kevinastone 10y agoThe benchmark (simple echo server) results were right there in the blog post: http://magic.io/blog/uvloop-blazing-fast-python-networking/tcp-bench.html http://magic.io/blog/uvloop-blazing-fast-python-networking/t...
- 1st1 10y agoFor benchmarks we have a separate project on the GH: [1] -- you can run the benchmarks on a Linux box pretty easily. Here's an example output: [2] [1] https://github.com/MagicStack/vmbench https://github.com/MagicStack/vmbench [2] http://magic.io/blog/uvloop-blazing-fast-python-networking/tcp-bench.html http://magic.io/blog/uvloop-blazing-fast-python-networking/t...
- e12e 10y agoNot sure what you mean, the benchmark section lists ~40k packets/s for nodejs and asyncio, ~100k for uvloop (for 1KiB packages, similar difference for 10 and 100 KiB) - and ~20k req/s for nodejs, and ~37k for uvloop w/httptools. Interestingly, uvloop pulls ahead for 100KiB request size for the http case.
- danappelxx 10y agoBuilding on to this, how does it compare to raw libuv in c?Personally, I'm not surprised that python (especially cython) is faster than node in this case, but I still need to see how much less overhead there is to node.
- 1st1 10y ago> Building on to this, how does it compare to raw libuv in c? Building something in C is very hard. uvloop wraps all libuv primitives in Python objects which know how to manage the memory safely (i.e. not to "free" something before libuv is done with it). So development time wise, uvloop is much better. As for the performance, I guess you'd be able to squeeze another 5-15% if you write the echo server in C. > I'm not surprised that python (especially cython) is faster than node in this case Cython is a statically typed compiled language, it can be anywhere from 2x to 100x faster than CPython.
- danappelxx 10y ago>So development time wise, uvloop is much better. Yeah, I definitely get that. I'm just trying to see the smaller picture here. >Cython is a statically typed compiled language, it can be anywhere from 2x to 100x faster than Python. Ah, my bad for not knowing the difference between Cython and CPython. It seems to me, then, that this isn't really a fair comparison to node, is it? Naturally a statically typed language is going to be faster than a dynamic one. Good on you for including a comparison with Go, though.
- Sean1708 10y ago> It seems to me, then, that this isn't really a fair comparison to node, is it? Isn't node written in C?
- danappelxx 10y agoYes, but JavaScript is still a dynamic, garbage collected language.
- breathesalt 10y agoI'm working on a Python CLI that uses asyncio/aiohttp to make and process requests to a 3rd party API. Anyways, I ran into the 10,000 socket problem today and ended up using a semaphore, that actually boosted the overall performance. Why is that? Is it just because the CPU is overwhelmed otherwise?
- 1st1 10y agoIt depends. Maybe you aren't closing the sockets properly (shutdown + close). Or maybe, because of how TCP works, the sockets are stuck in timeouts and don't really close for a long period of time. If its something like that, then your old connections aren't really closing, and new ones can't be created. Or maybe it's a simple problem of aiohttp performance -- as shown in the blog post, its HTTP parser is a bit slow. In general, I'd recommend to use a fewer number of sockets and implement some pipelining of API requests.
- takeda 10y agoAre you making all connections within a single session? Not long ago I saw example here that someone was creating a new session for every single connection. This is not very optimal way of using it. If you use it within same session, aiohttp will make use of keep-alive, which in turn will reuse existing connections and reduce overhead. You also won't need to use a semaphore, since you can define limit in TCPConnector. Why you had performance issues? As other said, you were making thousands of connections, each socket need to be in TIME_WAIT state for 2 minutes after closing (limitation of TCP, SCTP does not have this problem). So if you use all connections within short time, you'll essentially run out of them. Some people use tcp_tw_reuse/recycle, and that solves this issue, but that makes your connections no longer follow RFC and you might encounter strange issues later on. The advice above should resolve your problem without any hacks.
- zoombini29 10y agodo you have perf benchmarks under heavier concurrency? (e.g., how does it do with 100 concurrent clients? 1000?)
- 1st1 10y agoThe HTTP benchmarks were actually run under concurrency level of 300. I've just updated the post with extra details. See also the full report of HTTP benchmark [1] [1] http://magic.io/blog/uvloop-blazing-fast-python-networking/http-bench.html http://magic.io/blog/uvloop-blazing-fast-python-networking/h...
- continuations 10y ago1) What makes uvloop which is based on libuv 2x faster than node.js which is also based on libuv? 2) Can uvloop be used with frameworks like flask or django? 3) gevent uses monkey patching to turn blocking libraries such as DB drivers non-blocking, does uvloop do anything similar? If not how does it work with blocking libraries?
- 1st1 10y ago1) I don't know :( I've answered a similar question in this thread with a couple of guesses. 2) No, they have a different architecture. Although I heard that there is a project to integrate asyncio into django to get websockets and http/2. 3) asyncio/uvloop require you to use explicit async/await. So, unfortunately, the existing networking code that isn't built for asyncio can't be reused. On the bright side, there are so many asyncio DB drivers and other modules now!
- sandGorgon 10y ago> 2) No, they have a different architecture Having a web framework (a next generation Flask if you will) built ground up with concurrency is the missing key. I have used Flask for many years, but this is the right time to introduce a new framework - because of the internal restructuring and slowdown of Flask's maintainers (and I say this with the utmost respect). If you are keen, this has the potential to be the killer application for Python 3.
- 1st1 10y agoI'm actually thinking about writing such a framework :) I'll call it "spin".
- happytrails 10y agosounds good! Go forth!
- sandGorgon 10y agoYou totally should. I think people are hungry for a next gen async web framework... And if it leverages Python 3, then so much the better. Flask cannot go here even if it wants because its core philosophy is to be WSGI compliant. You don't necessarily have to adhere to that. Just one point, please make sure you have designed DB access as a core part of your framework (e.g. [1]). Too many frameworks discount database interaction until it's too late. Oh and please please choose your name so that it doesn't conflict on Google search. http://www.spinframework.org http://www.spinframework.org [1] http://initd.org/psycopg/docs/advanced.html#async-support http://initd.org/psycopg/docs/advanced.html#async-support vs https://github.com/chtd/psycopg2cffi https://github.com/chtd/psycopg2cffi
- jamesmishra 10y agoIt looks like uvloop requires Python 3.5. How practical is it to create a fork for companies stuck on Python 2.7?
- 1st1 10y agoNot really practical. uvloop is designed to work in tandem with asyncio, which is a Python 3-only module. asyncio, in turn, requires 'yield from' support, something that Python 2 doesn't have.
- jquast 10y agoThe 'Trollius' module backported asyncio to Python 2, requiring syntax changes, though. For example "yield from" to "yield From()". Work on Trollius was stopped a few weeks ago, there wasn't enough interest/use. Call for interested maintainers, http://trollius.readthedocs.io/deprecated.html#deprecated http://trollius.readthedocs.io/deprecated.html#deprecated
- takeda 10y agoThe problem with Trollius was that packages that asyncio packages needed explicitly to add support for it to work (because Python 2 does not have yield from) Several packages (I remember aiohttp was one of them) did not want to do that.
- sharpshadow 10y agoToday it should be good practice to use containers with Python 2.7 and 3.5 or any other versions you like. With that you can solve most szenarios.
- takeda 10y agoPython is actually designed in such way that you can install multiple major versions and they can coexist perfectly fine together. For example you can install python 2.6, 2.7, 3.3, 3.4, & 3.5 all on one host without any conflicts. The limitation is that many distributions prefer to not maintain different versions of supposedly the same language. If you use RedHat or CentOS you can just use https://ius.io/ https://ius.io/ and get access to the other python versions. This is one of few repos that makes sure the packages don't conflict with system ones.
- viraptor 10y agoNo questions, just a thank you for including relevant information about the tests content, concurrency, adding percentile boxes to graphs, etc. (and the tested environment itself!) It's refreshing to see a benchmark taken seriously rather than "we tested some stuff, here are 3 numbers, victory!".
- 1st1 10y agoThanks!
- felixweis 10y agoPretty awesome stuff. Do you think aiohttp's http parsing would benefit much on a hypothetical pypy3 interpreter?
- 1st1 10y agoMaybe. But I think its parser is better to be replaced with the httptools one.
- rattray 10y agoThis is awesome! How does it compare to PyPy? Once PyPy3 is available, will this work with it?
- 1st1 10y ago> Once PyPy3 is available, will this work with it? We'll find a way!
- patrickg_zill 10y agoAre you using this code in production?
- 1st1 10y agoI'd say it's not yet ready for production. The test coverage is fairly decent, though, so I hope we'll make a stable release soon. That said, uvloop should be fully compatible with asyncio. All APIs should be ready, so you can start testing it in your projects.
- cjbprime 10y agoWow. What's the intuition for why python on top of libuv is 2x faster than node on top of libuv?
- 1st1 10y agoTwo things I'd check first: 1. The benchmarks make servers generate a huge number of objects, so maybe, the GC is under too much pressure. 2. Another possibility is that the v8 JIT can't optimize some JS code in nodejs, or does a poor job. That said, only careful profiling can answer your question :)
- geekuillaume 10y agoI would be interested in seeing the performance difference in NodeJS TCP echo benchmark by using piping instead of reading/writing manually: socket.pipe(socket);
- fzaninotto 10y agoHow about clustering the Node server? By default, node uses only one process. Making the server use all available processes requires a bit more work (see https://nodejs.org/docs/latest/api/cluster.html https://nodejs.org/docs/latest/api/cluster.html or https://github.com/Unitech/pm2 https://github.com/Unitech/pm2) but will probably do more justice to Node.js vs Python.
- 1st1 10y ago> but will probably do more justice to Node.js vs Python. How so? Clustering for node just makes it run in several OS processes. You can (and should) do the same for Python code, it's easy.
- occasionalreedr 10y agoC Python uses ref counting, whereas Node/V8 uses GC. Ref counting schemes generally use much less memory and GC scans can be expensive when there's a lot of new data, as is the case for a web server.
- windlep 10y agoThis is quite interesting, but I don't find req/sec very interesting at all. These should be about matters of concurrency, that is, how much is being done at once, not how many req/sec overall are done (which could almost be explained away purely by the gains in lower latency). These benchmarks seem to only use 10 clients concurrently, max. That's ridiculously low. A few questions I'd like to see answered. How many clients can you connect to each server, and have each one ping once per 5 mins, before the CPU gets overloaded? How much memory is used per connection? How does PyPy+Twisted fair in this?
- 1st1 10y ago> These benchmarks seem to only use 10 clients concurrently, max. That's ridiculously low. I've just update the post with more details on HTTP benchmarks and attached the correct full-results file [1]. The concurrency level for HTTP benchmarks is 300, not 10. To answer other questions, I'll have to run some benchmarks tomorrow :) [1] http://magic.io/blog/uvloop-blazing-fast-python-networking/http-bench.html http://magic.io/blog/uvloop-blazing-fast-python-networking/h...
- windlep 10y agoCool! For the client test I'm referring to, these would be long-lived clients that stay around and just TCP ping for an echo server, rather than HTTP calls that connect/disconnect. In my experience, PyPy+twisted is around 5-25x faster CPython/twisted, and smoked asyncio as well. Would be great to see how uvloop compares there, and of course, someday when PyPy supports Python 3.5, there's no reason it couldn't use uvloop via cffi I'd hope.
- 1st1 10y ago> [..] Would be great to see how uvloop compares there [..] Yep, I'm curious to see what will happen there. Do you have any suggestions on what tool to use to generate the load? > [..] someday when PyPy supports Python 3.5, there's no reason it couldn't use uvloop via cffi I'd hope. We'll figure that out! ;)
- nitely 10y agoAsyncio is pretty fast, but as soon as you write any Python logic, request per second will drop significantly. I'm surprise uvloop is faster than node.js since libuv was developed for the latter. Kudos to the author.
- 1st1 10y ago> Asyncio is pretty fast, but as soon as you write any Python logic, request per second will drop significantly. Sure, but it really depends on how complex your Python code is. > I'm surprise uvloop is faster than node.js since libuv was developed for the latter. Kudos to the author. Thanks!
- tanlermin 10y agoCan't one write the logic in cython also?
- SEJeff 10y agoAlso note that David Beazley (google him if you're not aware) has a competitor to asyncio, which arguably is also a direct competitor to this called curio: http://curio.readthedocs.io/en/latest/ http://curio.readthedocs.io/en/latest/ The caveat is that it uses the new async/await coroutine bits that just landed in Python 3.5, so it only works with Python 3.5+. He also gave a talk on concurrency in python recently at last year's PyCon: https://www.youtube.com/watch?v=MCs5OvhV9S4 https://www.youtube.com/watch?v=MCs5OvhV9S4
- 1st1 10y agoThere are benchmarks of curio in the blog post, FWIW.
- SEJeff 10y agoGood stuff. I saw him RT your announcement of this on twitter.
- cderwin 10y agoUnless I'm mistaken, the decorator @asyncio.coroutine() is equivalent to async def and yield from is functionally a drop-in for await, so you should be able to use it with at least 3.4, maybe 3.3. Not that that's much better though.
- masklinn 10y agoThat makes it functionally equivalent to uvloop (which from my understanding is a drop-in replacement for the built-in asyncio eventloop).
- smegel 10y agoIt is interesting that uvloop-streams is almost identical to gevent in performance. Gevent is based on libev, the project libuv was forked from. What exactly is the -streams addition that makes uvloop-streams perform so much worse than plain uvloop?
- 1st1 10y agoStreams implementation is a pretty big chunk of Python code, that manages flow control, buffering, and integration with coroutines. And you don't always need all that stuff when you're writing a protocol, since you can implement them more efficiently as part of protocol parser.
- smegel 10y agoIs it possible to add a benchmark for gevent using raw sockets not StreamServer (assuming that adds similar overhead).
- 1st1 10y agoAFAIK, StreamServer should actually affect benchmark in a positive way. In gevent case, StreamServer isn't about high-level abstractions, it's about making sure that client sockets always have a READ flag in the IO multiplexor (epoll, kqueue etc)
- nehemiahjacob 10y agoWhat's the point? No Python 2.7
- forgotmypassw 10y agoUnless you're working with legacy project, in which case you wouldn't be replacing the lib for such stuff anyway, I don't see any reason why you would want to use 2.7.x over 3.x unless you desperately need something like gevent.
- gst 10y agogevent got 3.x support a while ago.
- mastazi 10y agoI was waiting for this LOL... I think the parent was being sarcastic here :-)
- ascotan 10y agoI think claims about being faster than X require showing the code used in the benchmarking.
- 1st1 10y agoEverything is in the blog post: https://github.com/MagicStack/vmbench/tree/master/servers https://github.com/MagicStack/vmbench/tree/master/servers
- haberman 10y ago> However, the performance bottleneck in aiohttp turned out to be its HTTP parser, which is so slow, that it matters very little how fast the underlying I/O library is. This is exactly the same observation that motivated the Mongrel web server for Ruby, 10 years ago this year. "Mongrel is a small library that provides a very fast HTTP 1.1 server for Ruby web applications. [...] What makes Mongrel so fast is the careful use of an Ragel [C] extension to provide fast, accurate HTTP 1.1 protocol parsing. This makes the server scream without too many portability issues." -- https://github.com/mongrel/mongrel https://github.com/mongrel/mongrel And its successor Thin: "Thin is a Ruby web server that glues together 3 of the best Ruby libraries in web history: (1) the Mongrel parser, the root of Mongrel speed and security" -- http://code.macournoyer.com/thin/ http://code.macournoyer.com/thin/ In case anyone is still wondering, parsing in Ruby/Python/Lua is pretty slow compared to C/C++. That's why I personally have been really interested for a long time in writing parsers in C that can be used from higher level languages. That way you can get the best of both worlds.
- coderdude 10y agoIt's the best of both worlds but doesn't shield you from the worst of one of the worlds. Untrusted input is still reaching code that has direct access to system memory. Hopefully not, anyway. But probably. Still, it's the way to go if performance is key.
- haberman 10y agoVery true. Thankfully fuzzing tools are getting better all the time. LLVM's libfuzzer is great.
- coderdude 10y agoAbsolutely it's a positive that this is an active area of research. One day these types of problems are going to be something future programmers joke about. Or don't even know existed.
- ericfrederich 10y ago
- MrBra 10y agoTime to take a rest for JS? :)
- pritambarhate 10y agoIt will be great to know how this compares to async frameworks in Java, Netty and Vert.x for example, in the benchmarks.
- 1st1 10y agoFeel free to make a PR to https://github.com/MagicStack/vmbench https://github.com/MagicStack/vmbench!
- distracteddev90 10y agoThe benchmarks for node.js are terribly misleading. The node.js implementations only ever spawn a single process, and thus node is only running on a single core and uses only a single thread. Specifically, the http server example(1), doesn't even bother using the standard library provided Cluster module(2). Cluster is specifically designed for distributing server workloads across multiple cores. All node.js services/applications I've worked on in the past 3 years (that are concerned with scale) utilize a multi-process node architecture. The current benchmark can only claim that a single python process that spawns multiple threads is 2x faster than a single node.js process that spawns only one thread. This fact may be interesting to some, but is irrelevant to real world performance. [1]: https://github.com/MagicStack/vmbench/blob/master/servers/nodejs_http_server.js https://github.com/MagicStack/vmbench/blob/master/servers/no... [2]: https://nodejs.org/api/cluster.html https://nodejs.org/api/cluster.html
- alexchamberlain 10y agoDo the other benchmarkside spawn multiple processes?
- masklinn 10y agoNo, even Go is explicitly configured to only use one scheduler: > We use Python 3.5, and all servers are single-threaded. Additionally, we use GOMAXPROCS=1 for Go code, nodejs does not use cluster, and all Python servers are single-process.
- 1st1 10y agoThere is nothing misleading about the benchmarks. It is explicitly said that ALL frameworks were benchmarked in single-process and single-thread modes. Yes, in production you should run your nodejs app in cluster, your Python apps in a multiprocess configuration, and you should never use GOMAXPROCS=1 for your go apps in production! Running all benchmarks in multiprocess configuration wouldn't add anything new to the results.
- distracteddev90 10y ago
- halayli 10y agoThis isn't a fair comparison. the "HTTP server" presented isn't doing any checks / validation that a typical web server does. Compare https://github.com/MagicStack/vmbench/blob/master/servers/asyncio_http_server.py https://github.com/MagicStack/vmbench/blob/master/servers/as... to https://github.com/nodejs/node/blob/master/lib/_http_outgoing.js https://github.com/nodejs/node/blob/master/lib/_http_outgoin... https://github.com/nodejs/node/blob/master/lib/_http_server.js https://github.com/nodejs/node/blob/master/lib/_http_server.... https://github.com/golang/go/blob/master/src/net/http/server.go https://github.com/golang/go/blob/master/src/net/http/server... The benchmark is almost equivalent to testing raw eventloop performance against a complete http server. To make this test fair, you need to write the same code in asyncio_http_server.py in other languages.
- felixweis 10y agoThere is a TCP and a HTTP benchmark.
- RedCrowbar 10y ago> This isn't a fair comparison. the "HTTP server" presented isn't doing any checks / validation that a typical web server does. How so? It uses a binding to http-parser, just like nodejs.
- halayli 10y agoParsing is just a small part of an http server. You can check the links I posted to see how much work is involved in validating http headers (after parsing), and creating http responses.
- 1st1 10y agoKeep in mind that the HTTP server in the benchmarks actually uses the httptools parser, which is a full-blown HTTP parser. A lot of heavy-lifting is done by the parser. I plan to add a complete HTTP protocol implementation to httptools, but honestly, I don't expect it to be more than 20% slower.
- messel 10y agoLooks good. Wonder why these benchmarks never include an actual database. I'm using firebase with nodejs and no matter how many requests per second my server can respond with, it's ultimately constrained by data requests and memory (ie how many connections can I wait on). When I see numbers like 50k requests per second it's meaningless, unless I have no database or some kind of in memory cache only db.
- sametmax 10y agoIt's not meaningless for new gen apps which trade a lot of db request for message passing. In micro-services app, when you update something, you propagate the change, and you have x client update for one db requests instead of having x + 1 db requests. In that context, broacasting quickly to a lot of clients is important.
- noplay 10y agoDoes it mean that uvloop could allow the use of the same event loop implementation on the three major OS?
- 1st1 10y agoasyncio already allows that. uvloop, as of now, only runs on *nix; but I hope we'll have Windows supported soon too.