5 ms·

I never really did much with PyPy, do people mostly use it in a deployed application setting? I ask because looking over at the PyPy Speed page... https://spee

I never really did much with PyPy, do people mostly use it in a deployed application setting? I ask because looking over at the PyPy Speed page... https://speed.pypy.org/ https://speed.pypy.org/ Looks like Django is insanely faster under PyPy. Feels like a potential waste not to use PyPy on a deployed web app in most cases. I wonder how FastAPI scales with PyPy and other Python interpreters.

jmg_ 3y ago
Note they're comparing to CPython v3.7 and while https://speed.python.org https://speed.python.org doesn't go back to 3.7, the improvements between 3.8 to 3.12 are pretty massive. I don't doubt PyPy is faster than CPython, but it would be very interesting to see latest PyPy compared to latest CPython.
JulianWasTaken 3y ago
At least for me / my former $WORK, the answer was "definitely yes we used it in deployed applications", we saved more than $1M (this in ~2015) annually in infrastructure costs by quite literally switching from CPython to PyPy and never really looked back, but obviously there are many considerations involved there that are going to be specific to your (company's) application(s) and infrastructure.
doix 3y ago
I used it at my old job specifically to speed up a Django application. After a bunch of profiling, I narrowed down a bunch of our performance problems to the ORM doing a bunch of __setattr__ (or something, it was a long time ago). We could have started rewriting everything and using .values() (or something, basically get back tuples instead of python objects) but switching to PyPy got the performance to good enough without having to sacrifice DX. It's been a long time, so I'm a bit fuzzy on the details, but I definitely recommend PyPy for Django unless there's some reason you can't use it.
mdaniel 3y ago
In my specific circumstance it behaved like a statically linked, one-.tar-and-ready mechanism to get python onto Flatcar (nee CoreOS, but not the modern one). So my case for it wasn't speed it was the dead simple deploy
lend000 3y ago
I used it in production for a while, but it caused instability with pandas and often froze so I had to take it out. It does have some serious speed benefits for simple / pure-Python without compiled libraries.
eugenekolo 3y ago
I run it in a production environment (side project). I also use it locally when developing when necessary. It really does speed up loops by 5x or so. So when you're trying to say... test 100 million+ iterations of something, pypy will run that in something like 2 minutes versus cpython can take me 15 minutes. Honestly it's an amazing performance gain for 0 effort, and I have yet to run into a limitation with it.
ktm5j 3y ago
I've put a bit of effort into trying to make it work with the python apps we write at my company. We have a lot of electrical engineers who only write python and we're butting up against the limit of what the language can do as far as the data we need to process.. It's worked well in some cases, but the main situation where we needed it to work was with a GUI application where there was no separation between computation and the GUI elements.. I tried really hard, and got frustratingly close, but could not build wxPython that would work using pypy. But it's super cool imo, as someone who has to write/maintain python code it's awesome to see some innovative work to make the language faster.
- stuaxo 3y ago
  Gtk could be an option it used to not work well at all, but since they improved their C binding support it got better.
  - ktm5j 3y ago
    Good to know, thanks!
- giancarlostoro 3y ago
  I havent used it in a long time but did you try Qt as well? PySide worked insanely nicely for me. Although this wasnt with PyPy to be fair.
  - ktm5j 3y ago
    Well the thing is that it's code that's already been written, and honestly is kind of a mess. It would take more effort than it's worth to try and rewrite using a different GUI framework. I much prefer Qt but this was written before I started here so it is what it is :/ The longer the test runs we end up losing data because they're looping over a dataframe that keeps growing and growing and the program just can't finish that loop in the 2 seconds that it has to run. But it's not a big enough deal for us to fix
    - 3y ago
stuaxo 3y ago
I think you have to start your Django project with PyPy, it's always a gamble which dependencies will work with it, and for a project already going the chances are that something won't work. If you find something like that it's worth reporting, there are less of them over time.
- giancarlostoro 3y ago
  Awesome I will keep this in mind since I am working on a new Django project, thanks!

jmg_ 3y ago

Note they're comparing to CPython v3.7 and while https://speed.python.org https://speed.python.org doesn't go back to 3.7, the improvements between 3.8 to 3.12 are pretty massive. I don't doubt PyPy is faster than CPython, but it would be very interesting to see latest PyPy compared to latest CPython.

JulianWasTaken 3y ago

At least for me / my former $WORK, the answer was "definitely yes we used it in deployed applications", we saved more than $1M (this in ~2015) annually in infrastructure costs by quite literally switching from CPython to PyPy and never really looked back, but obviously there are many considerations involved there that are going to be specific to your (company's) application(s) and infrastructure.

doix 3y ago

I used it at my old job specifically to speed up a Django application. After a bunch of profiling, I narrowed down a bunch of our performance problems to the ORM doing a bunch of __setattr__ (or something, it was a long time ago). We could have started rewriting everything and using .values() (or something, basically get back tuples instead of python objects) but switching to PyPy got the performance to good enough without having to sacrifice DX. It's been a long time, so I'm a bit fuzzy on the details, but I definitely recommend PyPy for Django unless there's some reason you can't use it.

mdaniel 3y ago

In my specific circumstance it behaved like a statically linked, one-.tar-and-ready mechanism to get python onto Flatcar (nee CoreOS, but not the modern one). So my case for it wasn't speed it was the dead simple deploy

lend000 3y ago

I used it in production for a while, but it caused instability with pandas and often froze so I had to take it out. It does have some serious speed benefits for simple / pure-Python without compiled libraries.

eugenekolo 3y ago

I run it in a production environment (side project). I also use it locally when developing when necessary. It really does speed up loops by 5x or so. So when you're trying to say... test 100 million+ iterations of something, pypy will run that in something like 2 minutes versus cpython can take me 15 minutes. Honestly it's an amazing performance gain for 0 effort, and I have yet to run into a limitation with it.

ktm5j 3y ago

I've put a bit of effort into trying to make it work with the python apps we write at my company. We have a lot of electrical engineers who only write python and we're butting up against the limit of what the language can do as far as the data we need to process.. It's worked well in some cases, but the main situation where we needed it to work was with a GUI application where there was no separation between computation and the GUI elements.. I tried really hard, and got frustratingly close, but could not build wxPython that would work using pypy. But it's super cool imo, as someone who has to write/maintain python code it's awesome to see some innovative work to make the language faster.

stuaxo 3y ago
Gtk could be an option it used to not work well at all, but since they improved their C binding support it got better.
- ktm5j 3y ago
  Good to know, thanks!
giancarlostoro 3y ago
I havent used it in a long time but did you try Qt as well? PySide worked insanely nicely for me. Although this wasnt with PyPy to be fair.
- ktm5j 3y ago
  Well the thing is that it's code that's already been written, and honestly is kind of a mess. It would take more effort than it's worth to try and rewrite using a different GUI framework. I much prefer Qt but this was written before I started here so it is what it is :/ The longer the test runs we end up losing data because they're looping over a dataframe that keeps growing and growing and the program just can't finish that loop in the 2 seconds that it has to run. But it's not a big enough deal for us to fix
  - 3y ago

stuaxo 3y ago

I think you have to start your Django project with PyPy, it's always a gamble which dependencies will work with it, and for a project already going the chances are that something won't work. If you find something like that it's worth reporting, there are less of them over time.

giancarlostoro 3y ago
Awesome I will keep this in mind since I am working on a new Django project, thanks!