5 ms·

> Everytime I talk about my blog post, I am super careful to qualify: in this case, "usually has little to do w/ speed" and other qualifications. The blog post

> Everytime I talk about my blog post, I am super careful to qualify: in this case, "usually has little to do w/ speed" and other qualifications. The blog post as well. I created an asyncpg suite for your bigdata benchmark (https://bitbucket.org/zzzeek/bigdata https://bitbucket.org/zzzeek/bigdata). Here are the results obtained from running it on my laptop against local PostgreSQL 9.5. 300 connections: Python3.5.2 threads -- avg 9798.16 recs/sec Python3.5.2 gevent -- avg 9677.84 recs/sec Python3.5.2 uvloop/asyncpg -- avg 11558.24 r/sec The modified suite is here: https://github.com/elprans/bigdata https://github.com/elprans/bigdata

zzzeek 10y ago
that's great, you've written a very fast driver. My benchmarks are comparing psycopg2 to itself, under threads / gevent vs. asyncio, to show that asyncio is inherently slower. It follows that if you wrote your driver and all of its protocol improvements into a traditional blocking model, it would also be much faster. You've written something so fast that it overcomes the latency of asyncio (this whole thread is a huge plug for your driver, btw). It's not that the latency of asyncio is really that big of a deal. The point I try to make is, asyncio != speed. For databases, 95% of the time, it means, a little less speed. Not a big deal, but if you aren't into writing explicit context switching directives throughout 100% of your code when the OS does it for you just fine, it isn't worth it, unless you are in the zone of thousands of arbitrarily performing connections, which is really more of an incoming client connection thing, not usually a database thing.
- RedCrowbar 10y ago
  > that's great, you've written a very fast driver. My benchmarks are comparing psycopg2 to itself, OK, I ran the asyncio suite (with uvloop event loop): Python3.5.2 asyncio+psycopg+uvloop -- avg 8356.84 recs/sec Really not that slower. There is also an important detail that you forget to mention: the threaded benchmark uses notably more CPU time (about 30% more).

zzzeek 10y ago

that's great, you've written a very fast driver. My benchmarks are comparing psycopg2 to itself, under threads / gevent vs. asyncio, to show that asyncio is inherently slower. It follows that if you wrote your driver and all of its protocol improvements into a traditional blocking model, it would also be much faster. You've written something so fast that it overcomes the latency of asyncio (this whole thread is a huge plug for your driver, btw). It's not that the latency of asyncio is really that big of a deal. The point I try to make is, asyncio != speed. For databases, 95% of the time, it means, a little less speed. Not a big deal, but if you aren't into writing explicit context switching directives throughout 100% of your code when the OS does it for you just fine, it isn't worth it, unless you are in the zone of thousands of arbitrarily performing connections, which is really more of an incoming client connection thing, not usually a database thing.

RedCrowbar 10y ago
> that's great, you've written a very fast driver. My benchmarks are comparing psycopg2 to itself, OK, I ran the asyncio suite (with uvloop event loop): Python3.5.2 asyncio+psycopg+uvloop -- avg 8356.84 recs/sec Really not that slower. There is also an important detail that you forget to mention: the threaded benchmark uses notably more CPU time (about 30% more).