4 ms·

Pypy JIT is way faster than CPython

throwaway894345 7y ago
It is, but that’s not saying a lot and it’s still far far slower than Go/Java/C#, and it also isn’t compatible with lots of existing Python code. Such as anything that would like to talk to a Postgres database. Pypy is great, but it can’t fix a broken ecosystem and general lack of leadership in the Python community. :( (Broken ecosystem = lots of things don’t work with lots of other things; package managers are another example; Mypy another)
- aeyes 7y ago
  What's the problem with Postgres on PyPy? Running fine in production here.
  - throwaway894345 7y ago
    There are no supported drivers for it as far as I could find. The common answer is to use psycopg2cffi, but that hasn’t had a commit in like 4 years. The alternatives I could find were in the same shape or worse.
    - anarazel 7y ago
      There's a few pure python drivers. Check https://wiki.postgresql.org/wiki/Python https://wiki.postgresql.org/wiki/Python and https://wiki.python.org/moin/PostgreSQL https://wiki.python.org/moin/PostgreSQL E.g. pg8000 looks like it's being maintained?
      - throwaway894345 7y ago
        I recall looking at pg8000 in particular, but it didn't inspire confidence for our production use case. It had only 20 commits in 2019 and fewer than 100 stars. I also didn't have a good understanding of the performance tradeoffs (pypy/pg8000 vs C library).
- j88439h84 7y ago
  Poetry and mypy are fine. Postgres works on pypy. But generally.. yeah you're right.
  - throwaway894345 7y ago
    My experience with Mypy has not been pleasant. It doesn't work with a whole host of packages including zip packages (PEP 441) or those that aren't built for Mypy directly (you have to sprinkle empty .pytyped files in various places in your project IIRC or mypy will ignore the type hints you provide; I ran into this with the `black` package. Further, getting it to find installed packages is tremendously difficult--it generally just fails with "couldn't find package" or something similarly useless and the docs it links you to are not helpful in my experience. Mypy also barfs on recursive types, so you can't express things like JSON, nor can you constrain generic types based on protocols. Similarly, Postgres doesn't work for Pypy unless you're willing to use libraries that are unmaintained or only very lightly maintained, and even then who knows what the performance tradeoffs will be. Those are pretty substantial caveats for a production use case. I haven't used Poetry, but I've heard good things. I'm very cautious though, because I've also heard high praise about pip/virtualenv, pip-tools, pipenv, etc and every single one had spectacular problems on the happy path (30 minutes to resolve dependencies with pipenv is my favorite).
    - j88439h84 7y ago
      > who knows what the performance tradeoffs will be Sure that's true, but performance is about measurement. Poetry has some missing pieces still (I can list them if you're interested but I'm on my phone) but it's the first time I've felt ok recommending a packaging tool to beginners. The experience is way better than the alternatives like pipenv.
      - throwaway894345 7y ago
        > Sure that's true, but performance is about measurement. I have a very low degree of confidence that a pure Python solution (even JIT optimized) is going to compete with a native solution, not to mention that by all appearances it isn't sufficiently well-supported to meet our criteria. Measuring the performance for me feels like a waste of time; it's easier just to forego Pypy until the ecosystem matures. > Poetry has some missing pieces still (I can list them if you're interested but I'm on my phone) but it's the first time I've felt ok recommending a packaging tool to beginners. The experience is way better than the alternatives like pipenv. This is good to know. I'll have to give it a try.

It is, but that’s not saying a lot and it’s still far far slower than Go/Java/C#, and it also isn’t compatible with lots of existing Python code. Such as anything that would like to talk to a Postgres database. Pypy is great, but it can’t fix a broken ecosystem and general lack of leadership in the Python community. :( (Broken ecosystem = lots of things don’t work with lots of other things; package managers are another example; Mypy another)

aeyes 7y ago
What's the problem with Postgres on PyPy? Running fine in production here.
- throwaway894345 7y ago
  There are no supported drivers for it as far as I could find. The common answer is to use psycopg2cffi, but that hasn’t had a commit in like 4 years. The alternatives I could find were in the same shape or worse.
  - anarazel 7y ago
    There's a few pure python drivers. Check https://wiki.postgresql.org/wiki/Python https://wiki.postgresql.org/wiki/Python and https://wiki.python.org/moin/PostgreSQL https://wiki.python.org/moin/PostgreSQL E.g. pg8000 looks like it's being maintained?
    - throwaway894345 7y ago
      I recall looking at pg8000 in particular, but it didn't inspire confidence for our production use case. It had only 20 commits in 2019 and fewer than 100 stars. I also didn't have a good understanding of the performance tradeoffs (pypy/pg8000 vs C library).
j88439h84 7y ago
Poetry and mypy are fine. Postgres works on pypy. But generally.. yeah you're right.
- throwaway894345 7y ago
  My experience with Mypy has not been pleasant. It doesn't work with a whole host of packages including zip packages (PEP 441) or those that aren't built for Mypy directly (you have to sprinkle empty .pytyped files in various places in your project IIRC or mypy will ignore the type hints you provide; I ran into this with the `black` package. Further, getting it to find installed packages is tremendously difficult--it generally just fails with "couldn't find package" or something similarly useless and the docs it links you to are not helpful in my experience. Mypy also barfs on recursive types, so you can't express things like JSON, nor can you constrain generic types based on protocols. Similarly, Postgres doesn't work for Pypy unless you're willing to use libraries that are unmaintained or only very lightly maintained, and even then who knows what the performance tradeoffs will be. Those are pretty substantial caveats for a production use case. I haven't used Poetry, but I've heard good things. I'm very cautious though, because I've also heard high praise about pip/virtualenv, pip-tools, pipenv, etc and every single one had spectacular problems on the happy path (30 minutes to resolve dependencies with pipenv is my favorite).
  - j88439h84 7y ago
    > who knows what the performance tradeoffs will be Sure that's true, but performance is about measurement. Poetry has some missing pieces still (I can list them if you're interested but I'm on my phone) but it's the first time I've felt ok recommending a packaging tool to beginners. The experience is way better than the alternatives like pipenv.
    - throwaway894345 7y ago
      > Sure that's true, but performance is about measurement. I have a very low degree of confidence that a pure Python solution (even JIT optimized) is going to compete with a native solution, not to mention that by all appearances it isn't sufficiently well-supported to meet our criteria. Measuring the performance for me feels like a waste of time; it's easier just to forego Pypy until the ecosystem matures. > Poetry has some missing pieces still (I can list them if you're interested but I'm on my phone) but it's the first time I've felt ok recommending a packaging tool to beginners. The experience is way better than the alternatives like pipenv. This is good to know. I'll have to give it a try.