5 ms·

> Right now, PyPy is probable the biggest reason why many users don't migrate to Python 3 yet. Does PyPy have that many users?

pinaraf 10y ago
Well, it weights in favor of Python 2, and considering that until recently the incencitives for Python 3 were still light... It's not really about having users, but having the possibility of switching from CPython to PyPy if you face performance issues.
- joejev 10y ago
  PyPy is still not really a performance option for numerical computing.
  - xapata 10y ago
    Are you not satisfied with NumPy and associated tools?
    - dagw 10y ago
      Very satisfied, and that is why I can't really switch to PyPy
brianwawok 10y ago
Doesn't seem that way to me. For actual real world apps with a lot of IO, your PyPy benefit is kinda small. For actual number crunching, I guess. Which is the whole scientific side of Python. Python is weird that it is 50% webapp hipsters ala Ruby, and 50% scientists in lab coats sitting in the lab crunching.
- rjeli 10y ago
  Well, PyPy does not actually help much with raw number crunching -- a scientist will use numpy for native matrix math, or numba to JIT a tight loop that can't be writted in numpy easily. PyPy really helps in the in-between cases: a few weeks ago I had to write a script to pull a few million rows from a database, process them nontrivially, and dump them to a file. What do I use here? CPython and Ruby will take days. Perl will probably take days, unless I am just doing text processing which perl is optimized for. I can write C or C++, and have the headache of debugging memory errors in a script that will only run once. Or, I can write PyPy and have immensely convenient syntax that will run reasonably quickly (Luajit also occupies this space for me).
  - brianwawok 10y ago
    Well if this task is trivial to parallelize.. I would spin up 100 GCE machines for 1 penny each per hour after today's price drop (combined total of $1 per hour for 100 CPUs of action).. and pull those few million files and dump to a cloud drive in 1 hour flat. It's a tricky sweet spot PyPy has. Certain tasks that need CPU power but not too much CPU power and you want to spend time to make it run in parallel but not too much time to make it run across many machines. (By no means am I a PyPy or CPython expert. Just toyed with them and want to understand the bounds...)
    - zardeh 10y ago
      But now you have to manage that deployment which is also nontrivial.
      - brianwawok 10y ago
        Depends on your setup I guess? I have a a GCE cluster setup where I can: 1) Point a metadata variable at my git repo 2) Spin up 100 GCEs 3) Each will pip install requirements.txt from the git checkout 4) Each will read a metadata to get the command and run it. (Could hardcode run.py or something). Realize not everyone has this sitting around, but you can do it in an afternoon. Then gives you instant Python supercomputer for anything you want across many projects.
        zardeh 10y ago
        The setup for that sounds significantly nontrivial, although interesting.
- dr_zoidberg 10y ago
  Scientific Python benefits more from (properly) vectorized numpy code, Cython, Numba and Numexpr than what it gains from PyPy. From my experience, one of the biggest issues with PyPy is that it's not a magic pill you take and suddenly all your Python code runs faster -- that's how they "sell" it. But in many cases, the idiomatic code for CPython results in slow PyPy execution, and idiomatic PyPy code results in slow(er) CPython execution. That means that you have to refactor your code, and whereas you won't be breaking compatibility with CPython, your CPython performance will (most often) suffer. So, unless you're the sole user of your code, or you have control or decision power over most of your users, (or you just don't care to keep good CPython performance) your project might not benefit: "yeah, in PyPy it runs 2x faster, but 80% of our clients use CPython and are complaining over a 5x slow down -- ditch that code and get back to the previous version" may end up being the feedback from someone above you in hierarchy.
- eeZi 10y ago
  > Python is weird that it is 50% webapp hipsters ala Ruby, and 50% scientists in lab coats sitting in the lab crunching. Don't forget about the ops guys using it for automation!
  - brianwawok 10y ago
    Oh yes my ansible brothers and their lack of Python3 support ;)
    - kasey_junk 10y ago
      Where brothers == inbred cousins
      - brianwawok 10y ago
        Hahaha kinda.. but if Ansible brothers are inbred cousins, Puppet and Chef brothers are certified insane asylum.

pinaraf 10y ago

Well, it weights in favor of Python 2, and considering that until recently the incencitives for Python 3 were still light... It's not really about having users, but having the possibility of switching from CPython to PyPy if you face performance issues.

joejev 10y ago
PyPy is still not really a performance option for numerical computing.
- xapata 10y ago
  Are you not satisfied with NumPy and associated tools?
  - dagw 10y ago
    Very satisfied, and that is why I can't really switch to PyPy

brianwawok 10y ago

Doesn't seem that way to me. For actual real world apps with a lot of IO, your PyPy benefit is kinda small. For actual number crunching, I guess. Which is the whole scientific side of Python. Python is weird that it is 50% webapp hipsters ala Ruby, and 50% scientists in lab coats sitting in the lab crunching.

rjeli 10y ago
Well, PyPy does not actually help much with raw number crunching -- a scientist will use numpy for native matrix math, or numba to JIT a tight loop that can't be writted in numpy easily. PyPy really helps in the in-between cases: a few weeks ago I had to write a script to pull a few million rows from a database, process them nontrivially, and dump them to a file. What do I use here? CPython and Ruby will take days. Perl will probably take days, unless I am just doing text processing which perl is optimized for. I can write C or C++, and have the headache of debugging memory errors in a script that will only run once. Or, I can write PyPy and have immensely convenient syntax that will run reasonably quickly (Luajit also occupies this space for me).
- brianwawok 10y ago
  Well if this task is trivial to parallelize.. I would spin up 100 GCE machines for 1 penny each per hour after today's price drop (combined total of $1 per hour for 100 CPUs of action).. and pull those few million files and dump to a cloud drive in 1 hour flat. It's a tricky sweet spot PyPy has. Certain tasks that need CPU power but not too much CPU power and you want to spend time to make it run in parallel but not too much time to make it run across many machines. (By no means am I a PyPy or CPython expert. Just toyed with them and want to understand the bounds...)
  - zardeh 10y ago
    But now you have to manage that deployment which is also nontrivial.
    - brianwawok 10y ago
      Depends on your setup I guess? I have a a GCE cluster setup where I can: 1) Point a metadata variable at my git repo 2) Spin up 100 GCEs 3) Each will pip install requirements.txt from the git checkout 4) Each will read a metadata to get the command and run it. (Could hardcode run.py or something). Realize not everyone has this sitting around, but you can do it in an afternoon. Then gives you instant Python supercomputer for anything you want across many projects.
      - zardeh 10y ago
        The setup for that sounds significantly nontrivial, although interesting.
dr_zoidberg 10y ago
Scientific Python benefits more from (properly) vectorized numpy code, Cython, Numba and Numexpr than what it gains from PyPy. From my experience, one of the biggest issues with PyPy is that it's not a magic pill you take and suddenly all your Python code runs faster -- that's how they "sell" it. But in many cases, the idiomatic code for CPython results in slow PyPy execution, and idiomatic PyPy code results in slow(er) CPython execution. That means that you have to refactor your code, and whereas you won't be breaking compatibility with CPython, your CPython performance will (most often) suffer. So, unless you're the sole user of your code, or you have control or decision power over most of your users, (or you just don't care to keep good CPython performance) your project might not benefit: "yeah, in PyPy it runs 2x faster, but 80% of our clients use CPython and are complaining over a 5x slow down -- ditch that code and get back to the previous version" may end up being the feedback from someone above you in hierarchy.
eeZi 10y ago
> Python is weird that it is 50% webapp hipsters ala Ruby, and 50% scientists in lab coats sitting in the lab crunching. Don't forget about the ops guys using it for automation!
- brianwawok 10y ago
  Oh yes my ansible brothers and their lack of Python3 support ;)
  - kasey_junk 10y ago
    Where brothers == inbred cousins
    - brianwawok 10y ago
      Hahaha kinda.. but if Ansible brothers are inbred cousins, Puppet and Chef brothers are certified insane asylum.