6 ms·

PyPy: the other new compiler project

52 points by corbet 16y ago

metamemetics 16y ago
PyPy while great news for Python, is still a snail compared to another high-level language JIT, LuaJIT: http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=luajit&lang2=pypy http://shootout.alioth.debian.org/u32/benchmark.php?test=all... edit: bit of background: Lua is a minimalist dynamic language. It can be used anywhere in your web development request stack since mod_lua was included in Apache 2.3.
- Raphael_Amiard 16y ago
  And it also has nothing to do with it from an architectural point of view. LuaJIT's code is not generally reusable. It's highly optimized, often platform specific C and assembler code. PyPy is an experimental (in the good way) and successful attempt at using a subset of python to make a faster python interpreter than the native C interpreter, while in the process creating a platform to create other optimized dynamic languages interpreters. It's code is highly general and readable, especially compared to something as specific and low level than LuaJIT. In the process, the devs have created a new language, rpython, that is as fast as C while being much more expressive. Lua is also a much simpler language than python, and therefore a much better target to fine tuned optimizations. You could almost say that LuaJIT is a pragmatic project, and pypy an idealistic one. And they're both succeeding. But they don't have the same goals at all.Please dont compare them without giving a bit of background. It's very misleading.
  - monopede 16y ago
    Do you have any references for your "as fast as C" part? That seems like a very dubious claim. If your ultimate goal is performance, I believe PyPy is fundamentally flawed because it already introduces two levels of indirection that cannot easily be removed. Of course their goal seems to be to simply be faster than the python interpreter and remain maintainable at the same time. They appear to be succeeding at that and I congratulate them on that. LuaJIT takes the right approach w.r.t. performance. The interpreter for LuaJIT 2 is well-optimised and not actually that hard to understand. This goes together with a well-chosen set of interpreter instruction set and intermediate representation. All these layers require careful design, and Mike Pall didn't come up with this over night.
    - deleted 16y ago
      [deleted]
    - stephenjudkins 16y ago
      Here's how RPython is "faster than C": 1. PyPy's Python interpreter is written in RPython, a subset of Python. 2. CPython is an interpreter written in C. 3. When compiled into a JIT, PyPy's Python interpreter is now generally faster than CPython, sometimes by a wide margin. 4. Code written in RPython is faster than code written for a nearly identical purpose in C. The C code in question has been the target of aggressive optimizations for years while the RPython code is relatively immature. Now, for many applications I'm sure C remains faster than RPython. But for the application RPython is designed for--writing an interpreter for a dynamic language--I think it is safe to say that RPython appears faster than C. The fact that the PyPy's interpreter is cleaner, safer, and more maintainable than equivalent C code is also a huge win. Again, the point of the original post is that RPython could become an simple implementation language to create fast interpreters for dynamic languages. It could also become a target language for other compilers. Anyways, as others have pointed out, Lua is a considerably simpler language than Python. LuaJIT is a really remarkable piece of engineering, especially since it was written by one guy, but it's really a completely different project than PyPy. Its aim is to make Lua--and only Lua--extremely fast. If PyPy's Python implementation gets no faster (which I think is unlikely) the potential ability to easily create new dynamic languages that are similarly speedy is itself an amazing achievement.
      - woodson 16y ago
        I find it interesting that people say Lua is a "simple" language, as if that's a flaw. Also, simple in which respect? To optimise? To parse? Note that this is no fanboyism. It's just that these performance claims are mostly phony anyways. How much % of general python code is in RPython? It's good to see this kind of progress, for both python and lua, as it opens up room for dynamic languages in new areas and helps to change the mindset of some people. However, this picking on each other is more than expendable IMHO..
        Raphael_Amiard 16y ago
- buster 16y ago
  While true, lua is more directed towards another audience, namely a small, embeddable scripting language with as little overhead as possible (IMO). This is not meant degrading, but i guess it has some implications. Lua is great for embedded scripting though.
ableal 16y ago
Off-topic: I was intrigued by the 'ATS' in the shootout (don't remember it being there a couple of years ago). I had followed the link posted by rodion_89 (2008 blog), into http://shootout.alioth.debian.org/#acceptable http://shootout.alioth.debian.org/#acceptable (nice opening and closing) and http://shootout.alioth.debian.org/u64/performance.php?test=binarytrees http://shootout.alioth.debian.org/u64/performance.php?test=b... - where 'ATS' sits at #3 (behind C and C++, edging out Java6). The code (e.g. http://shootout.alioth.debian.org/u64/program.php?test=binarytrees&lang=ats&id=3 http://shootout.alioth.debian.org/u64/program.php?test=binar... ) does not look too ugly, but apparently it may be twice as verbose as C++ ( http://shootout.alioth.debian.org/u32/ats.php http://shootout.alioth.debian.org/u32/ats.php ). So, on to http://www.ats-lang.org/ http://www.ats-lang.org/ : ATS is a programming language with a highly expressive type system rooted in the framework Applied Type System. In particular, both dependent types and linear types are available in ATS. The current implementation of ATS (ATS/Anairiats) is written in ATS itself. It can be as efficient as C/C++ (see The Computer Language Benchmarks Game for concrete evidence) and supports a variety of programming paradigms that include: - Functional programming. [...] - Imperative programming. [...] - Concurrent programming. [...] - Modular programming. [...]

52 points by corbet 16y ago

PyPy while great news for Python, is still a snail compared to another high-level language JIT, LuaJIT: http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=luajit&lang2=pypy http://shootout.alioth.debian.org/u32/benchmark.php?test=all... edit: bit of background: Lua is a minimalist dynamic language. It can be used anywhere in your web development request stack since mod_lua was included in Apache 2.3.

Raphael_Amiard 16y ago
And it also has nothing to do with it from an architectural point of view. LuaJIT's code is not generally reusable. It's highly optimized, often platform specific C and assembler code. PyPy is an experimental (in the good way) and successful attempt at using a subset of python to make a faster python interpreter than the native C interpreter, while in the process creating a platform to create other optimized dynamic languages interpreters. It's code is highly general and readable, especially compared to something as specific and low level than LuaJIT. In the process, the devs have created a new language, rpython, that is as fast as C while being much more expressive. Lua is also a much simpler language than python, and therefore a much better target to fine tuned optimizations. You could almost say that LuaJIT is a pragmatic project, and pypy an idealistic one. And they're both succeeding. But they don't have the same goals at all.Please dont compare them without giving a bit of background. It's very misleading.
- monopede 16y ago
  Do you have any references for your "as fast as C" part? That seems like a very dubious claim. If your ultimate goal is performance, I believe PyPy is fundamentally flawed because it already introduces two levels of indirection that cannot easily be removed. Of course their goal seems to be to simply be faster than the python interpreter and remain maintainable at the same time. They appear to be succeeding at that and I congratulate them on that. LuaJIT takes the right approach w.r.t. performance. The interpreter for LuaJIT 2 is well-optimised and not actually that hard to understand. This goes together with a well-chosen set of interpreter instruction set and intermediate representation. All these layers require careful design, and Mike Pall didn't come up with this over night.
  - deleted 16y ago
    [deleted]
  - stephenjudkins 16y ago
    Here's how RPython is "faster than C": 1. PyPy's Python interpreter is written in RPython, a subset of Python. 2. CPython is an interpreter written in C. 3. When compiled into a JIT, PyPy's Python interpreter is now generally faster than CPython, sometimes by a wide margin. 4. Code written in RPython is faster than code written for a nearly identical purpose in C. The C code in question has been the target of aggressive optimizations for years while the RPython code is relatively immature. Now, for many applications I'm sure C remains faster than RPython. But for the application RPython is designed for--writing an interpreter for a dynamic language--I think it is safe to say that RPython appears faster than C. The fact that the PyPy's interpreter is cleaner, safer, and more maintainable than equivalent C code is also a huge win. Again, the point of the original post is that RPython could become an simple implementation language to create fast interpreters for dynamic languages. It could also become a target language for other compilers. Anyways, as others have pointed out, Lua is a considerably simpler language than Python. LuaJIT is a really remarkable piece of engineering, especially since it was written by one guy, but it's really a completely different project than PyPy. Its aim is to make Lua--and only Lua--extremely fast. If PyPy's Python implementation gets no faster (which I think is unlikely) the potential ability to easily create new dynamic languages that are similarly speedy is itself an amazing achievement.
    - woodson 16y ago
      I find it interesting that people say Lua is a "simple" language, as if that's a flaw. Also, simple in which respect? To optimise? To parse? Note that this is no fanboyism. It's just that these performance claims are mostly phony anyways. How much % of general python code is in RPython? It's good to see this kind of progress, for both python and lua, as it opens up room for dynamic languages in new areas and helps to change the mindset of some people. However, this picking on each other is more than expendable IMHO..
      - Raphael_Amiard 16y ago
buster 16y ago
While true, lua is more directed towards another audience, namely a small, embeddable scripting language with as little overhead as possible (IMO). This is not meant degrading, but i guess it has some implications. Lua is great for embedded scripting though.

ableal 16y ago

Off-topic: I was intrigued by the 'ATS' in the shootout (don't remember it being there a couple of years ago). I had followed the link posted by rodion_89 (2008 blog), into http://shootout.alioth.debian.org/#acceptable http://shootout.alioth.debian.org/#acceptable (nice opening and closing) and http://shootout.alioth.debian.org/u64/performance.php?test=binarytrees http://shootout.alioth.debian.org/u64/performance.php?test=b... - where 'ATS' sits at #3 (behind C and C++, edging out Java6). The code (e.g. http://shootout.alioth.debian.org/u64/program.php?test=binarytrees&lang=ats&id=3 http://shootout.alioth.debian.org/u64/program.php?test=binar... ) does not look too ugly, but apparently it may be twice as verbose as C++ ( http://shootout.alioth.debian.org/u32/ats.php http://shootout.alioth.debian.org/u32/ats.php ). So, on to http://www.ats-lang.org/ http://www.ats-lang.org/ : ATS is a programming language with a highly expressive type system rooted in the framework Applied Type System. In particular, both dependent types and linear types are available in ATS. The current implementation of ATS (ATS/Anairiats) is written in ATS itself. It can be as efficient as C/C++ (see The Computer Language Benchmarks Game for concrete evidence) and supports a variety of programming paradigms that include: - Functional programming. [...] - Imperative programming. [...] - Concurrent programming. [...] - Modular programming. [...]