7 ms·

Has anyone done any recent memory benchmarks with PyPy? I'd like to see magnitude of the memory trade-off for using a JIT compiler. As a web developer my progr

Has anyone done any recent memory benchmarks with PyPy? I'd like to see magnitude of the memory trade-off for using a JIT compiler. As a web developer my programs are mostly IO-bound, not CPU-bound. I'm also bootstrapping and trying to squeeze as much as I can out of my 512MB linode.

kingkilr 16y ago
PyPy's objects are smaller than CPython's, however the steady state interpreter is larger, and the JIT adds some additional overhead due to bookeeping and generated machine code.
- kwellman 16y ago
  PyPy is a very interesting project, but I suppose there's just no "killer feature" for me to start using it at the moment.
silentbicycle 16y ago
Comparisons with LuaJIT (http://luajit.org/ http://luajit.org/) would be particularly nice.
- acqq 16y ago
  But the speed difference is still too big. Even if PyPy is two times faster than CPython, LuaJit still remains in most of the cases the order of magnitude faster, they are comparable only when calculation bottleneck are bignum routines and not rest of the language: http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=python&lang2=luajit http://shootout.alioth.debian.org/u32/benchmark.php?test=all... In short, if you care about the speed and LuaJit is an option, the choice is obvious.
  - igouy 16y ago
    Ummm, better to point to PyPy :: LuaJIT ? http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=pypy&lang2=luajit http://shootout.alioth.debian.org/u32/benchmark.php?test=all... Caveat - notice how many of those Lua programs were written by Mike Pall.
    - silentbicycle 16y ago
      The ones I've read are relatively straightforward Lua, though. Some shootout programs look really hairy compared to normal code in their language. (The "optimized Haskell" shootout programs were, at one point, though I haven't followed it for a while.) With Lua / LuaJIT, that doesn't seem to be the case. Besides, Mike Pall is using some of the shootout benchmarks to tune LuaJIT, so it's not surprising he has many of the top submissions.
      - igouy 16y ago
        It isn't a matter of being surprised that Mike Pall has contributed programs, more a statement of the obvious lest we forget. Mike Pall is kind-of a good programmer, and that may well effect how the programs perform.
        silentbicycle 16y ago
        I think it has more to do with how his runtime performs. :) Tuning Lua code really isn't that hard, the language is tiny and has both semantics and performance characteristics that are easy to reason about accurately. There's a good sample chapter from _Lua Programming Gems_ on Lua performance tuning (http://www.lua.org/gems/sample.pdf http://www.lua.org/gems/sample.pdf), FWIW. That and a good profiler will get you far.
- metamemetics 16y ago
  LuaJIT is a nigh untouchable work of art, we might never see a faster dynamic language JIT. PyPy is significant because of the nature of Python itself: the language specification is much more complex than Lua and there are currently many more users and applications. [the more complicated the language specification, the harder it's going to be to prove things about, the harder it's going to be to write a compiler]
  - Raphael_Amiard 16y ago
    > LuaJIT is a nigh untouchable work of art, we might never see a faster dynamic language JIT. Well Mike Pall disagrees in that he says there are still a lot of possible optimizations. Also all the process of by hand optimization for specific architecture can theoretically be automatized and decoupled. You're totally right about Python complexity though.
  - Zak 16y ago
    >we might never see a faster dynamic language JIT Lisp implementations have been performing at a similar level for years now - many with the aid of AOT compilation, some not. Currently, Racket and SBCL are comparable to LuaJIT on the Alioth microbenchmarks - faster at some and slower at others.
- dripton 16y ago
  You want me to port all my Project Euler solutions from Python to Lua so that I can tell you that LuaJIT is much faster than any Python implementation, which you already know?
  - silentbicycle 16y ago
    Not at all, I'm just asking for comments from anybody who has experience with both. I used to use Python quite a bit, but switched to Lua a few years ago and PyPy really hasn't been on my radar.
  - mikemike 16y ago
    Well, here's Euler14 in Lua for a start: http://lua-users.org/lists/lua-l/2010-09/msg00568.html http://lua-users.org/lists/lua-l/2010-09/msg00568.html But obviously for a proper comparison of the runtimes the benchmarks need to be implemented the 'same way' and not simply yield the 'same result'.
dripton 16y ago
Yes, I will also include peak memory used next time.

kingkilr 16y ago

PyPy's objects are smaller than CPython's, however the steady state interpreter is larger, and the JIT adds some additional overhead due to bookeeping and generated machine code.

kwellman 16y ago
PyPy is a very interesting project, but I suppose there's just no "killer feature" for me to start using it at the moment.

silentbicycle 16y ago

Comparisons with LuaJIT (http://luajit.org/ http://luajit.org/) would be particularly nice.

acqq 16y ago
But the speed difference is still too big. Even if PyPy is two times faster than CPython, LuaJit still remains in most of the cases the order of magnitude faster, they are comparable only when calculation bottleneck are bignum routines and not rest of the language: http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=python&lang2=luajit http://shootout.alioth.debian.org/u32/benchmark.php?test=all... In short, if you care about the speed and LuaJit is an option, the choice is obvious.
- igouy 16y ago
  Ummm, better to point to PyPy :: LuaJIT ? http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=pypy&lang2=luajit http://shootout.alioth.debian.org/u32/benchmark.php?test=all... Caveat - notice how many of those Lua programs were written by Mike Pall.
  - silentbicycle 16y ago
    The ones I've read are relatively straightforward Lua, though. Some shootout programs look really hairy compared to normal code in their language. (The "optimized Haskell" shootout programs were, at one point, though I haven't followed it for a while.) With Lua / LuaJIT, that doesn't seem to be the case. Besides, Mike Pall is using some of the shootout benchmarks to tune LuaJIT, so it's not surprising he has many of the top submissions.
    - igouy 16y ago
      It isn't a matter of being surprised that Mike Pall has contributed programs, more a statement of the obvious lest we forget. Mike Pall is kind-of a good programmer, and that may well effect how the programs perform.
      - silentbicycle 16y ago
        I think it has more to do with how his runtime performs. :) Tuning Lua code really isn't that hard, the language is tiny and has both semantics and performance characteristics that are easy to reason about accurately. There's a good sample chapter from _Lua Programming Gems_ on Lua performance tuning (http://www.lua.org/gems/sample.pdf http://www.lua.org/gems/sample.pdf), FWIW. That and a good profiler will get you far.
metamemetics 16y ago
LuaJIT is a nigh untouchable work of art, we might never see a faster dynamic language JIT. PyPy is significant because of the nature of Python itself: the language specification is much more complex than Lua and there are currently many more users and applications. [the more complicated the language specification, the harder it's going to be to prove things about, the harder it's going to be to write a compiler]
- Raphael_Amiard 16y ago
  > LuaJIT is a nigh untouchable work of art, we might never see a faster dynamic language JIT. Well Mike Pall disagrees in that he says there are still a lot of possible optimizations. Also all the process of by hand optimization for specific architecture can theoretically be automatized and decoupled. You're totally right about Python complexity though.
- Zak 16y ago
  >we might never see a faster dynamic language JIT Lisp implementations have been performing at a similar level for years now - many with the aid of AOT compilation, some not. Currently, Racket and SBCL are comparable to LuaJIT on the Alioth microbenchmarks - faster at some and slower at others.
dripton 16y ago
You want me to port all my Project Euler solutions from Python to Lua so that I can tell you that LuaJIT is much faster than any Python implementation, which you already know?
- silentbicycle 16y ago
  Not at all, I'm just asking for comments from anybody who has experience with both. I used to use Python quite a bit, but switched to Lua a few years ago and PyPy really hasn't been on my radar.
- mikemike 16y ago
  Well, here's Euler14 in Lua for a start: http://lua-users.org/lists/lua-l/2010-09/msg00568.html http://lua-users.org/lists/lua-l/2010-09/msg00568.html But obviously for a proper comparison of the runtimes the benchmarks need to be implemented the 'same way' and not simply yield the 'same result'.

dripton 16y ago

Yes, I will also include peak memory used next time.