4 ms·

PyPy 1.7 released: widening the sweet spot

160 points by kingkilr 15y ago

mattdeboard 15y ago
It is hard to tell from some of the release statements when they mean a particular thing is an improvement over CPython or over previous performance by PyPy, e.g.: >Specialized list implementation. There is a branch that implements lists of integers/floats/strings as compactly as array.array. This should drastically improve performance/memory impact of some applications Is this improvement over CPython or over previous implementation in PyPy? There are several such comments.
- vladev 15y ago
  Currently, PyPy is faster than CPython at pretty much anything (except when C is involved, for example, but they are working on that, too). You can see their benchmarks and comparisons at http://speed.pypy.org/ http://speed.pypy.org/. What I think they mean by those statements is that this will improve performance over the previous version of PyPy - therefore - improve over CPython as well.
- nickik 15y ago
  If something is new its an improvment over CPython oderwise its a improvment over pypy. See here for lists: http://morepypy.blogspot.com/2011/10/more-compact-lists-with-list-strategies.html http://morepypy.blogspot.com/2011/10/more-compact-lists-with...
- Raphael_Amiard 15y ago
  I'd say this particular improvement is necessarily in pypy, because plain lists are always boxed in CPython, which doesn't perform any of that kind of type specific optimizations, being a plain old interpreter.
- masklinn 15y ago
  > Is this improvement over CPython or over previous implementation in PyPy? Both.
levesque 15y ago
I, for one, will certainly consider switching to pypy when they have a stable, usable implementation of numpy :)
- wladimir 15y ago
  You can donate towards Numpy in Pypy on their main page.
  - MostAwesomeDude 15y ago
    You can also donate code. I chipped in a bit of work since I used Numpy at the time and needed a couple features. It's not that hard.
    - kingkilr 15y ago
      The NumPyPy codebase is fun! You can to learn a bit about all different parts of PyPy, you can play with jit stuff, Python interp stuff, or just plain numeric stuff. It's programmed in a pretty nice meta-programming way in Python, and it's very well tested.
- wisty 15y ago
  It's likely to take a while. See Travis's post: http://technicaldiscovery.blogspot.com/2011/10/thoughts-on-porting-numpy-to-pypy.html http://technicaldiscovery.blogspot.com/2011/10/thoughts-on-p... There are two main problems he sees - numpy is a moving target (as it continues to evolve), and the C-API of numpy is one of its most useful features (as lots of numpy extensions like Scipy and matplotlib may require it).
  - fijal 15y ago
    As you might know, we disagree with Travis when it comes to timelines. C API is not really useful without CPython C API and we're trying hard to make pypy interoperate with C. Look for f2pypy for an example.
YoukaiCountry 15y ago
I took the plunge and switched over to pypy with version 1.6. Besides the occasional minor headache, most things work as expected and the huge speed-up makes it more than worth it.
binarycrusader 15y ago
PyPy performance is good, but the project is still not very mature. It's not quite yet ready as a drop-in replacement. In particular, the build process is really painful (3+ hours on a high-end Xeon workstation) and quite immature. I was particularly annoyed to discover that the build process isn't incremental -- if it fails for any reason, you get to start all over again. The CPython extension support is also still very experimental (that is, if you have any Python extension modules written in C, be prepared for a lot of fun getting them to work). PyPy also doesn't support anywhere near the number of OS platforms that Python does natively. It's very much a Linux, Mac, or Windows thing only. And it's heavily focused on x86 if you want to use the JIT version.
- _mrc 15y ago
  Why the downvotes? I find this analysis useful. Is it incorrect?
  - fijal 15y ago
    Yes, 3+ hours sounds incorrect. Also, it's not like KDE builds in 15 minutes either. I don't see how a long build time makes the project immature, feel free to suggest the connection.
- dripton 15y ago
  3 hours? You're doing something wrong. I just built PyPy in 49 minutes on a $99 3 GHz Phenom II.
  - obtu 15y ago
    You probably have more RAM?
- MostAwesomeDude 15y ago
  The build is very all-or-nothing in its code analysis. Feel free to suggest ways to make this incremental -- the code isn't compiled per-module like C-style languages. The C API is not something you should target. Write pure Python instead of C extensions. If you're wrapping a C API, use Cython, which mostly stays within the boundaries of sanity (and might someday grow a ctypes target). Most of the C API (for certain values of "most") is only a recompile away. Which obscure UNIX flavor did you need supported? The biggest obstacle to supporting other platforms is a lack of buildslaves and people well-versed in those platforms.

160 points by kingkilr 15y ago

It is hard to tell from some of the release statements when they mean a particular thing is an improvement over CPython or over previous performance by PyPy, e.g.: >Specialized list implementation. There is a branch that implements lists of integers/floats/strings as compactly as array.array. This should drastically improve performance/memory impact of some applications Is this improvement over CPython or over previous implementation in PyPy? There are several such comments.

vladev 15y ago
Currently, PyPy is faster than CPython at pretty much anything (except when C is involved, for example, but they are working on that, too). You can see their benchmarks and comparisons at http://speed.pypy.org/ http://speed.pypy.org/. What I think they mean by those statements is that this will improve performance over the previous version of PyPy - therefore - improve over CPython as well.
nickik 15y ago
If something is new its an improvment over CPython oderwise its a improvment over pypy. See here for lists: http://morepypy.blogspot.com/2011/10/more-compact-lists-with-list-strategies.html http://morepypy.blogspot.com/2011/10/more-compact-lists-with...
Raphael_Amiard 15y ago
I'd say this particular improvement is necessarily in pypy, because plain lists are always boxed in CPython, which doesn't perform any of that kind of type specific optimizations, being a plain old interpreter.
masklinn 15y ago
> Is this improvement over CPython or over previous implementation in PyPy? Both.

levesque 15y ago

I, for one, will certainly consider switching to pypy when they have a stable, usable implementation of numpy :)

wladimir 15y ago
You can donate towards Numpy in Pypy on their main page.
- MostAwesomeDude 15y ago
  You can also donate code. I chipped in a bit of work since I used Numpy at the time and needed a couple features. It's not that hard.
  - kingkilr 15y ago
    The NumPyPy codebase is fun! You can to learn a bit about all different parts of PyPy, you can play with jit stuff, Python interp stuff, or just plain numeric stuff. It's programmed in a pretty nice meta-programming way in Python, and it's very well tested.
wisty 15y ago
It's likely to take a while. See Travis's post: http://technicaldiscovery.blogspot.com/2011/10/thoughts-on-porting-numpy-to-pypy.html http://technicaldiscovery.blogspot.com/2011/10/thoughts-on-p... There are two main problems he sees - numpy is a moving target (as it continues to evolve), and the C-API of numpy is one of its most useful features (as lots of numpy extensions like Scipy and matplotlib may require it).
- fijal 15y ago
  As you might know, we disagree with Travis when it comes to timelines. C API is not really useful without CPython C API and we're trying hard to make pypy interoperate with C. Look for f2pypy for an example.

YoukaiCountry 15y ago

I took the plunge and switched over to pypy with version 1.6. Besides the occasional minor headache, most things work as expected and the huge speed-up makes it more than worth it.

binarycrusader 15y ago

PyPy performance is good, but the project is still not very mature. It's not quite yet ready as a drop-in replacement. In particular, the build process is really painful (3+ hours on a high-end Xeon workstation) and quite immature. I was particularly annoyed to discover that the build process isn't incremental -- if it fails for any reason, you get to start all over again. The CPython extension support is also still very experimental (that is, if you have any Python extension modules written in C, be prepared for a lot of fun getting them to work). PyPy also doesn't support anywhere near the number of OS platforms that Python does natively. It's very much a Linux, Mac, or Windows thing only. And it's heavily focused on x86 if you want to use the JIT version.

_mrc 15y ago
Why the downvotes? I find this analysis useful. Is it incorrect?
- fijal 15y ago
  Yes, 3+ hours sounds incorrect. Also, it's not like KDE builds in 15 minutes either. I don't see how a long build time makes the project immature, feel free to suggest the connection.
dripton 15y ago
3 hours? You're doing something wrong. I just built PyPy in 49 minutes on a $99 3 GHz Phenom II.
- obtu 15y ago
  You probably have more RAM?
MostAwesomeDude 15y ago
The build is very all-or-nothing in its code analysis. Feel free to suggest ways to make this incremental -- the code isn't compiled per-module like C-style languages. The C API is not something you should target. Write pure Python instead of C extensions. If you're wrapping a C API, use Cython, which mostly stays within the boundaries of sanity (and might someday grow a ctypes target). Most of the C API (for certain values of "most") is only a recompile away. Which obscure UNIX flavor did you need supported? The biggest obstacle to supporting other platforms is a lack of buildslaves and people well-versed in those platforms.