6 ms·

> the 1M row/sec part is easy to explain: Prepared statements were used in all benchmarks where supported. Please don't dismiss the results so easily. The so

> the 1M row/sec part is easy to explain: Prepared statements were used in all benchmarks where supported. Please don't dismiss the results so easily. The source of performance increases here is not the use of prepared statements, but efficient implementation of the binary type I/O and the protocol itself. > I'd be very wary of a new PG driver especially one that implements the wire protocol itself. > pg8000 does this, and also does prepared statements, and while it doesn't have the speed approach here, it also has serious problems supporting all the features and datatypes that psycopg2 does. asyncpg supports 100% of pg_catalog types (we have a test for that.) Additionally, psycopg2 doesn't _really_ support all the features and datatypes and simply returns unparsed text in many cases. You have to write lots of typecasters to convince it otherwise. > It's been around for years and reliably breaks on many Postgresql features (and Postgresql introduces new features like a firehose). Writing and maintaining a driver for Postgresql is not an easy task. Please. Simply because something has been around is not the reason to stiffle new development and innovation.

zzzeek 10y ago
I have no intention to stifle anything, I'm only raising the point that this seems like a really new driver and it's likely that it would take a while before it approaches the stability of psycopg2. That is, the stability of psycogp2 is not to be so lightly thrown in the trash. Does your driver seamlessly handle reading and writing of three-dimensional arrays of JSONB structures, and arrays of custom enumerated types ? edit: > but efficient implementation of the binary type I/O and the protocol itself. that's great too! you don't need async for that either. I guess I am frustrated that you folks put all this work into a great driver and then locked it in the largely useless dungeon of asyncio, where all the libraries are totally useless to those of us who don't think node.js is the programming model for all cases.
- RedCrowbar 10y ago
  > I have no intention to stifle anything, I'm only raising the point that this seems like a really new driver and it's likely that it would take a while before it approaches the stability of psycopg2. That is, the stability of psycogp2 is not to be so lightly thrown in the trash. We in no way question the merits and the stability if psycopg2. We created asyncpg to solve the challenges we face in the development of EdgeDB, it's not just a toy project for fun. > Does your driver seamlessly handle reading and writing of three-dimensional arrays of JSONB structures, and arrays of custom enumerated types ? Yes it does. > edit: > > but efficient implementation of the binary type I/O and the protocol itself. > that's great too! you don't need async for that either. > I guess I am frustrated that you folks put all this work into a great driver and then locked it in the largely useless dungeon of asyncio, where all the libraries are totally useless to those of us who don't think node.js is the programming model for all cases. I'm sorry to hear that. Nobody forces you to use asyncio or asyncpg. But please don't try to claim that something is a "useless dungeon" simply because you disagree.
  - sametmax 10y ago
    The tone used is indeed not the proper one, but while I'm a big asyncio advocate, I think it would be beneficial if you would split your project in 2, separating the IO from the rest (like hyper does https://github.com/Lukasa/hyper https://github.com/Lukasa/hyper). The rational being: - Python is not just async. It currently majoritarly sync. A lot of the community would benefit from your awesome project; - You'd have contrib from the async and sync community; - async may sometime not be worth it for db (http://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/); http://techspot.zzzeek.org/2015/02/15/asynchronous-python-an... - some people will prefer to use threads or gevent; - you will see compat plugin created for current ORM; That aside, I will use your project as is, since I'm deep in asyncio these days. Thank you.
    - 1st1 10y ago
      > I think it would be beneficial if you would split your project in 2, separating the IO from the rest (like hyper does https://github.com/Lukasa/hyper https://github.com/Lukasa/hyper). I'd like to do that, but we don't have resources. If we had a couple capable devs volunteering on the GH to do this, I'd be glad to review patches and guide the development.
      - sametmax 10y ago
        I get that. I have the same problem. And it's not just the ressources, what you need are skilled enough ressources, and that's even harder.
- rickycook 10y ago
  and it's just as frustrating to those of us that do use asyncio (because it really is excellent) when someone needlessly ties it to not use async. it's harder to maintain something that does both
- RedCrowbar 10y ago
  > I guess I am frustrated that you folks put all this work into a great driver and then locked it in the largely useless dungeon of asyncio, where all the libraries are totally useless to those of us who don't think node.js is the programming model for all cases I see your point. Nothing is really "locked away". The protocol parser and data codecs in asyncpg are independent from the I/O model and loop semantics. With effort, it's possible to build a sync interface on top. It's just not a priority, as our cases are async.

zzzeek 10y ago

I have no intention to stifle anything, I'm only raising the point that this seems like a really new driver and it's likely that it would take a while before it approaches the stability of psycopg2. That is, the stability of psycogp2 is not to be so lightly thrown in the trash. Does your driver seamlessly handle reading and writing of three-dimensional arrays of JSONB structures, and arrays of custom enumerated types ? edit: > but efficient implementation of the binary type I/O and the protocol itself. that's great too! you don't need async for that either. I guess I am frustrated that you folks put all this work into a great driver and then locked it in the largely useless dungeon of asyncio, where all the libraries are totally useless to those of us who don't think node.js is the programming model for all cases.

RedCrowbar 10y ago
> I have no intention to stifle anything, I'm only raising the point that this seems like a really new driver and it's likely that it would take a while before it approaches the stability of psycopg2. That is, the stability of psycogp2 is not to be so lightly thrown in the trash. We in no way question the merits and the stability if psycopg2. We created asyncpg to solve the challenges we face in the development of EdgeDB, it's not just a toy project for fun. > Does your driver seamlessly handle reading and writing of three-dimensional arrays of JSONB structures, and arrays of custom enumerated types ? Yes it does. > edit: > > but efficient implementation of the binary type I/O and the protocol itself. > that's great too! you don't need async for that either. > I guess I am frustrated that you folks put all this work into a great driver and then locked it in the largely useless dungeon of asyncio, where all the libraries are totally useless to those of us who don't think node.js is the programming model for all cases. I'm sorry to hear that. Nobody forces you to use asyncio or asyncpg. But please don't try to claim that something is a "useless dungeon" simply because you disagree.
- sametmax 10y ago
  The tone used is indeed not the proper one, but while I'm a big asyncio advocate, I think it would be beneficial if you would split your project in 2, separating the IO from the rest (like hyper does https://github.com/Lukasa/hyper https://github.com/Lukasa/hyper). The rational being: - Python is not just async. It currently majoritarly sync. A lot of the community would benefit from your awesome project; - You'd have contrib from the async and sync community; - async may sometime not be worth it for db (http://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/); http://techspot.zzzeek.org/2015/02/15/asynchronous-python-an... - some people will prefer to use threads or gevent; - you will see compat plugin created for current ORM; That aside, I will use your project as is, since I'm deep in asyncio these days. Thank you.
  - 1st1 10y ago
    > I think it would be beneficial if you would split your project in 2, separating the IO from the rest (like hyper does https://github.com/Lukasa/hyper https://github.com/Lukasa/hyper). I'd like to do that, but we don't have resources. If we had a couple capable devs volunteering on the GH to do this, I'd be glad to review patches and guide the development.
    - sametmax 10y ago
      I get that. I have the same problem. And it's not just the ressources, what you need are skilled enough ressources, and that's even harder.
rickycook 10y ago
and it's just as frustrating to those of us that do use asyncio (because it really is excellent) when someone needlessly ties it to not use async. it's harder to maintain something that does both
RedCrowbar 10y ago
> I guess I am frustrated that you folks put all this work into a great driver and then locked it in the largely useless dungeon of asyncio, where all the libraries are totally useless to those of us who don't think node.js is the programming model for all cases I see your point. Nothing is really "locked away". The protocol parser and data codecs in asyncpg are independent from the I/O model and loop semantics. With effort, it's possible to build a sync interface on top. It's just not a priority, as our cases are async.