5 ms·

I'm the dev behind uvloop. AMA.

by 1st1 10y ago

plainOldText 10y ago
What are some of the non-obvious limitations/peculiarities one should be aware before using this?
- 1st1 10y ago
  Wherever you use asyncio, it should be safe to just start using uvloop (once we graduate it from beta). uvloop shouldn't behave any differently, I've paid special attention to make sure it works exactly the same way as asyncio (down to when its objects are garbage collected).
ruffrey 10y ago
> at least 2x faster than nodejs, gevent, as well as any other Python asynchronous framework I did not see any benchmarks in the repo to support this. How was this statistic determined?
- kevinastone 10y ago
  The benchmark (simple echo server) results were right there in the blog post: http://magic.io/blog/uvloop-blazing-fast-python-networking/tcp-bench.html http://magic.io/blog/uvloop-blazing-fast-python-networking/t...
- 1st1 10y ago
  For benchmarks we have a separate project on the GH: [1] -- you can run the benchmarks on a Linux box pretty easily. Here's an example output: [2] [1] https://github.com/MagicStack/vmbench https://github.com/MagicStack/vmbench [2] http://magic.io/blog/uvloop-blazing-fast-python-networking/tcp-bench.html http://magic.io/blog/uvloop-blazing-fast-python-networking/t...
- e12e 10y ago
  Not sure what you mean, the benchmark section lists ~40k packets/s for nodejs and asyncio, ~100k for uvloop (for 1KiB packages, similar difference for 10 and 100 KiB) - and ~20k req/s for nodejs, and ~37k for uvloop w/httptools. Interestingly, uvloop pulls ahead for 100KiB request size for the http case.
- danappelxx 10y ago
  Building on to this, how does it compare to raw libuv in c?Personally, I'm not surprised that python (especially cython) is faster than node in this case, but I still need to see how much less overhead there is to node.
  - 1st1 10y ago
    > Building on to this, how does it compare to raw libuv in c? Building something in C is very hard. uvloop wraps all libuv primitives in Python objects which know how to manage the memory safely (i.e. not to "free" something before libuv is done with it). So development time wise, uvloop is much better. As for the performance, I guess you'd be able to squeeze another 5-15% if you write the echo server in C. > I'm not surprised that python (especially cython) is faster than node in this case Cython is a statically typed compiled language, it can be anywhere from 2x to 100x faster than CPython.
    - danappelxx 10y ago
      >So development time wise, uvloop is much better. Yeah, I definitely get that. I'm just trying to see the smaller picture here. >Cython is a statically typed compiled language, it can be anywhere from 2x to 100x faster than Python. Ah, my bad for not knowing the difference between Cython and CPython. It seems to me, then, that this isn't really a fair comparison to node, is it? Naturally a statically typed language is going to be faster than a dynamic one. Good on you for including a comparison with Go, though.
      - Sean1708 10y ago
        > It seems to me, then, that this isn't really a fair comparison to node, is it? Isn't node written in C?
        danappelxx 10y ago
        Yes, but JavaScript is still a dynamic, garbage collected language.
breathesalt 10y ago
I'm working on a Python CLI that uses asyncio/aiohttp to make and process requests to a 3rd party API. Anyways, I ran into the 10,000 socket problem today and ended up using a semaphore, that actually boosted the overall performance. Why is that? Is it just because the CPU is overwhelmed otherwise?
- 1st1 10y ago
  It depends. Maybe you aren't closing the sockets properly (shutdown + close). Or maybe, because of how TCP works, the sockets are stuck in timeouts and don't really close for a long period of time. If its something like that, then your old connections aren't really closing, and new ones can't be created. Or maybe it's a simple problem of aiohttp performance -- as shown in the blog post, its HTTP parser is a bit slow. In general, I'd recommend to use a fewer number of sockets and implement some pipelining of API requests.
- takeda 10y ago
  Are you making all connections within a single session? Not long ago I saw example here that someone was creating a new session for every single connection. This is not very optimal way of using it. If you use it within same session, aiohttp will make use of keep-alive, which in turn will reuse existing connections and reduce overhead. You also won't need to use a semaphore, since you can define limit in TCPConnector. Why you had performance issues? As other said, you were making thousands of connections, each socket need to be in TIME_WAIT state for 2 minutes after closing (limitation of TCP, SCTP does not have this problem). So if you use all connections within short time, you'll essentially run out of them. Some people use tcp_tw_reuse/recycle, and that solves this issue, but that makes your connections no longer follow RFC and you might encounter strange issues later on. The advice above should resolve your problem without any hacks.
zoombini29 10y ago
do you have perf benchmarks under heavier concurrency? (e.g., how does it do with 100 concurrent clients? 1000?)
- 1st1 10y ago
  The HTTP benchmarks were actually run under concurrency level of 300. I've just updated the post with extra details. See also the full report of HTTP benchmark [1] [1] http://magic.io/blog/uvloop-blazing-fast-python-networking/http-bench.html http://magic.io/blog/uvloop-blazing-fast-python-networking/h...
continuations 10y ago
1) What makes uvloop which is based on libuv 2x faster than node.js which is also based on libuv? 2) Can uvloop be used with frameworks like flask or django? 3) gevent uses monkey patching to turn blocking libraries such as DB drivers non-blocking, does uvloop do anything similar? If not how does it work with blocking libraries?
- 1st1 10y ago
  1) I don't know :( I've answered a similar question in this thread with a couple of guesses. 2) No, they have a different architecture. Although I heard that there is a project to integrate asyncio into django to get websockets and http/2. 3) asyncio/uvloop require you to use explicit async/await. So, unfortunately, the existing networking code that isn't built for asyncio can't be reused. On the bright side, there are so many asyncio DB drivers and other modules now!
  - sandGorgon 10y ago
    > 2) No, they have a different architecture Having a web framework (a next generation Flask if you will) built ground up with concurrency is the missing key. I have used Flask for many years, but this is the right time to introduce a new framework - because of the internal restructuring and slowdown of Flask's maintainers (and I say this with the utmost respect). If you are keen, this has the potential to be the killer application for Python 3.
    - 1st1 10y ago
      I'm actually thinking about writing such a framework :) I'll call it "spin".
      - happytrails 10y ago
        sounds good! Go forth!
      - sandGorgon 10y ago
        You totally should. I think people are hungry for a next gen async web framework... And if it leverages Python 3, then so much the better. Flask cannot go here even if it wants because its core philosophy is to be WSGI compliant. You don't necessarily have to adhere to that. Just one point, please make sure you have designed DB access as a core part of your framework (e.g. [1]). Too many frameworks discount database interaction until it's too late. Oh and please please choose your name so that it doesn't conflict on Google search. http://www.spinframework.org http://www.spinframework.org [1] http://initd.org/psycopg/docs/advanced.html#async-support http://initd.org/psycopg/docs/advanced.html#async-support vs https://github.com/chtd/psycopg2cffi https://github.com/chtd/psycopg2cffi
jamesmishra 10y ago
It looks like uvloop requires Python 3.5. How practical is it to create a fork for companies stuck on Python 2.7?
- 1st1 10y ago
  Not really practical. uvloop is designed to work in tandem with asyncio, which is a Python 3-only module. asyncio, in turn, requires 'yield from' support, something that Python 2 doesn't have.
  - jquast 10y ago
    The 'Trollius' module backported asyncio to Python 2, requiring syntax changes, though. For example "yield from" to "yield From()". Work on Trollius was stopped a few weeks ago, there wasn't enough interest/use. Call for interested maintainers, http://trollius.readthedocs.io/deprecated.html#deprecated http://trollius.readthedocs.io/deprecated.html#deprecated
    - takeda 10y ago
      The problem with Trollius was that packages that asyncio packages needed explicitly to add support for it to work (because Python 2 does not have yield from) Several packages (I remember aiohttp was one of them) did not want to do that.
- sharpshadow 10y ago
  Today it should be good practice to use containers with Python 2.7 and 3.5 or any other versions you like. With that you can solve most szenarios.
  - takeda 10y ago
    Python is actually designed in such way that you can install multiple major versions and they can coexist perfectly fine together. For example you can install python 2.6, 2.7, 3.3, 3.4, & 3.5 all on one host without any conflicts. The limitation is that many distributions prefer to not maintain different versions of supposedly the same language. If you use RedHat or CentOS you can just use https://ius.io/ https://ius.io/ and get access to the other python versions. This is one of few repos that makes sure the packages don't conflict with system ones.
viraptor 10y ago
No questions, just a thank you for including relevant information about the tests content, concurrency, adding percentile boxes to graphs, etc. (and the tested environment itself!) It's refreshing to see a benchmark taken seriously rather than "we tested some stuff, here are 3 numbers, victory!".
- 1st1 10y ago
  Thanks!
felixweis 10y ago
Pretty awesome stuff. Do you think aiohttp's http parsing would benefit much on a hypothetical pypy3 interpreter?
- 1st1 10y ago
  Maybe. But I think its parser is better to be replaced with the httptools one.
rattray 10y ago
This is awesome! How does it compare to PyPy? Once PyPy3 is available, will this work with it?
- 1st1 10y ago
  > Once PyPy3 is available, will this work with it? We'll find a way!

I'm the dev behind uvloop. AMA.

by 1st1 10y ago

I'm the dev behind uvloop. AMA.

plainOldText 10y ago

What are some of the non-obvious limitations/peculiarities one should be aware before using this?

1st1 10y ago
Wherever you use asyncio, it should be safe to just start using uvloop (once we graduate it from beta). uvloop shouldn't behave any differently, I've paid special attention to make sure it works exactly the same way as asyncio (down to when its objects are garbage collected).

ruffrey 10y ago

> at least 2x faster than nodejs, gevent, as well as any other Python asynchronous framework I did not see any benchmarks in the repo to support this. How was this statistic determined?

kevinastone 10y ago
The benchmark (simple echo server) results were right there in the blog post: http://magic.io/blog/uvloop-blazing-fast-python-networking/tcp-bench.html http://magic.io/blog/uvloop-blazing-fast-python-networking/t...
1st1 10y ago
For benchmarks we have a separate project on the GH: [1] -- you can run the benchmarks on a Linux box pretty easily. Here's an example output: [2] [1] https://github.com/MagicStack/vmbench https://github.com/MagicStack/vmbench [2] http://magic.io/blog/uvloop-blazing-fast-python-networking/tcp-bench.html http://magic.io/blog/uvloop-blazing-fast-python-networking/t...
e12e 10y ago
Not sure what you mean, the benchmark section lists ~40k packets/s for nodejs and asyncio, ~100k for uvloop (for 1KiB packages, similar difference for 10 and 100 KiB) - and ~20k req/s for nodejs, and ~37k for uvloop w/httptools. Interestingly, uvloop pulls ahead for 100KiB request size for the http case.
danappelxx 10y ago
Building on to this, how does it compare to raw libuv in c?Personally, I'm not surprised that python (especially cython) is faster than node in this case, but I still need to see how much less overhead there is to node.
- 1st1 10y ago
  > Building on to this, how does it compare to raw libuv in c? Building something in C is very hard. uvloop wraps all libuv primitives in Python objects which know how to manage the memory safely (i.e. not to "free" something before libuv is done with it). So development time wise, uvloop is much better. As for the performance, I guess you'd be able to squeeze another 5-15% if you write the echo server in C. > I'm not surprised that python (especially cython) is faster than node in this case Cython is a statically typed compiled language, it can be anywhere from 2x to 100x faster than CPython.
  - danappelxx 10y ago
    >So development time wise, uvloop is much better. Yeah, I definitely get that. I'm just trying to see the smaller picture here. >Cython is a statically typed compiled language, it can be anywhere from 2x to 100x faster than Python. Ah, my bad for not knowing the difference between Cython and CPython. It seems to me, then, that this isn't really a fair comparison to node, is it? Naturally a statically typed language is going to be faster than a dynamic one. Good on you for including a comparison with Go, though.
    - Sean1708 10y ago
      > It seems to me, then, that this isn't really a fair comparison to node, is it? Isn't node written in C?
      - danappelxx 10y ago
        Yes, but JavaScript is still a dynamic, garbage collected language.

breathesalt 10y ago

I'm working on a Python CLI that uses asyncio/aiohttp to make and process requests to a 3rd party API. Anyways, I ran into the 10,000 socket problem today and ended up using a semaphore, that actually boosted the overall performance. Why is that? Is it just because the CPU is overwhelmed otherwise?

1st1 10y ago
It depends. Maybe you aren't closing the sockets properly (shutdown + close). Or maybe, because of how TCP works, the sockets are stuck in timeouts and don't really close for a long period of time. If its something like that, then your old connections aren't really closing, and new ones can't be created. Or maybe it's a simple problem of aiohttp performance -- as shown in the blog post, its HTTP parser is a bit slow. In general, I'd recommend to use a fewer number of sockets and implement some pipelining of API requests.
takeda 10y ago
Are you making all connections within a single session? Not long ago I saw example here that someone was creating a new session for every single connection. This is not very optimal way of using it. If you use it within same session, aiohttp will make use of keep-alive, which in turn will reuse existing connections and reduce overhead. You also won't need to use a semaphore, since you can define limit in TCPConnector. Why you had performance issues? As other said, you were making thousands of connections, each socket need to be in TIME_WAIT state for 2 minutes after closing (limitation of TCP, SCTP does not have this problem). So if you use all connections within short time, you'll essentially run out of them. Some people use tcp_tw_reuse/recycle, and that solves this issue, but that makes your connections no longer follow RFC and you might encounter strange issues later on. The advice above should resolve your problem without any hacks.

zoombini29 10y ago

do you have perf benchmarks under heavier concurrency? (e.g., how does it do with 100 concurrent clients? 1000?)

1st1 10y ago
The HTTP benchmarks were actually run under concurrency level of 300. I've just updated the post with extra details. See also the full report of HTTP benchmark [1] [1] http://magic.io/blog/uvloop-blazing-fast-python-networking/http-bench.html http://magic.io/blog/uvloop-blazing-fast-python-networking/h...

continuations 10y ago

1) What makes uvloop which is based on libuv 2x faster than node.js which is also based on libuv? 2) Can uvloop be used with frameworks like flask or django? 3) gevent uses monkey patching to turn blocking libraries such as DB drivers non-blocking, does uvloop do anything similar? If not how does it work with blocking libraries?

1st1 10y ago
1) I don't know :( I've answered a similar question in this thread with a couple of guesses. 2) No, they have a different architecture. Although I heard that there is a project to integrate asyncio into django to get websockets and http/2. 3) asyncio/uvloop require you to use explicit async/await. So, unfortunately, the existing networking code that isn't built for asyncio can't be reused. On the bright side, there are so many asyncio DB drivers and other modules now!
- sandGorgon 10y ago
  > 2) No, they have a different architecture Having a web framework (a next generation Flask if you will) built ground up with concurrency is the missing key. I have used Flask for many years, but this is the right time to introduce a new framework - because of the internal restructuring and slowdown of Flask's maintainers (and I say this with the utmost respect). If you are keen, this has the potential to be the killer application for Python 3.
  - 1st1 10y ago
    I'm actually thinking about writing such a framework :) I'll call it "spin".
    - happytrails 10y ago
      sounds good! Go forth!
    - sandGorgon 10y ago
      You totally should. I think people are hungry for a next gen async web framework... And if it leverages Python 3, then so much the better. Flask cannot go here even if it wants because its core philosophy is to be WSGI compliant. You don't necessarily have to adhere to that. Just one point, please make sure you have designed DB access as a core part of your framework (e.g. [1]). Too many frameworks discount database interaction until it's too late. Oh and please please choose your name so that it doesn't conflict on Google search. http://www.spinframework.org http://www.spinframework.org [1] http://initd.org/psycopg/docs/advanced.html#async-support http://initd.org/psycopg/docs/advanced.html#async-support vs https://github.com/chtd/psycopg2cffi https://github.com/chtd/psycopg2cffi

jamesmishra 10y ago

It looks like uvloop requires Python 3.5. How practical is it to create a fork for companies stuck on Python 2.7?

1st1 10y ago
Not really practical. uvloop is designed to work in tandem with asyncio, which is a Python 3-only module. asyncio, in turn, requires 'yield from' support, something that Python 2 doesn't have.
- jquast 10y ago
  The 'Trollius' module backported asyncio to Python 2, requiring syntax changes, though. For example "yield from" to "yield From()". Work on Trollius was stopped a few weeks ago, there wasn't enough interest/use. Call for interested maintainers, http://trollius.readthedocs.io/deprecated.html#deprecated http://trollius.readthedocs.io/deprecated.html#deprecated
  - takeda 10y ago
    The problem with Trollius was that packages that asyncio packages needed explicitly to add support for it to work (because Python 2 does not have yield from) Several packages (I remember aiohttp was one of them) did not want to do that.
sharpshadow 10y ago
Today it should be good practice to use containers with Python 2.7 and 3.5 or any other versions you like. With that you can solve most szenarios.
- takeda 10y ago
  Python is actually designed in such way that you can install multiple major versions and they can coexist perfectly fine together. For example you can install python 2.6, 2.7, 3.3, 3.4, & 3.5 all on one host without any conflicts. The limitation is that many distributions prefer to not maintain different versions of supposedly the same language. If you use RedHat or CentOS you can just use https://ius.io/ https://ius.io/ and get access to the other python versions. This is one of few repos that makes sure the packages don't conflict with system ones.

viraptor 10y ago

No questions, just a thank you for including relevant information about the tests content, concurrency, adding percentile boxes to graphs, etc. (and the tested environment itself!) It's refreshing to see a benchmark taken seriously rather than "we tested some stuff, here are 3 numbers, victory!".

1st1 10y ago
Thanks!

felixweis 10y ago

Pretty awesome stuff. Do you think aiohttp's http parsing would benefit much on a hypothetical pypy3 interpreter?

1st1 10y ago
Maybe. But I think its parser is better to be replaced with the httptools one.

rattray 10y ago

This is awesome! How does it compare to PyPy? Once PyPy3 is available, will this work with it?

1st1 10y ago
> Once PyPy3 is available, will this work with it? We'll find a way!