10 ms·

Unladen Swallow: Making CPython Faster Using LLVM

105 points by durin42 18y ago

curtis 18y ago
The Project Plan at http://code.google.com/p/unladen-swallow/wiki/ProjectPlan http://code.google.com/p/unladen-swallow/wiki/ProjectPlan is way more interesting than the linked Google code page.
mcav 18y ago
A worthy goal, though I would have rather they just work based off of Python 3.0. By the time they achieve real, substantial progress, the momentum should be securely in 3.0's favor.
- mgreenbe 18y ago
  I don't think the tide is going to completely turn in the next three or four months. If this project meets its goals, I think CP2.6 -> US2.6 -> US3.0 is a more alluring migration path. If it truly succeeds, and CPython uses LLVM as its core, then the point is moot. I think 2.6 is the right place to start -- it's more stable and has a larger codebase. To hedge a little, I say all this from a distance: I don't use Python on a daily -- or even monthly -- basis.
- spamizbad 18y ago
  I'm not so sure Python 3.0 was really designed to carry any real momentum for "Python 3000" instead, it seems to serve as a base platform for future migration. I understand Guido and the Python team haven't explicitly stated that, but that certainly seems to be what they signaled. I don't expect see widespread support until 3.2, with Python 3.1 likely being the signal for people to begin serious migration efforts. As a Python developer myself, I couldn't move to 3.0 even if I wanted to: none of the open-source libraries I depend on have even begun to look into 3.0 support (Numpy won't even support Python 2.6 until its upcoming 1.3 release). Besides that, from a practical standpoint I'd guess the Unladen-Swallow team wants a codebase on more solid footing, like the mature 2.6.x series, as opposed to Python 3.x which is in a greater state of flux.
  - palish 18y ago
    Actually, Numpy 1.3 is available and does work with Python 2.6, in case you want to upgrade: C:\Users\Shawn>python Python 2.6.1 (r261:67517, Dec 4 2008, 16:51:00) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.array( [1, 2, 3] ) array([1, 2, 3]) >>> The Numpy SourceForge download page has the 2.6 installer: http://sourceforge.net/project/showfiles.php?group_id=1369&package_id=175103 http://sourceforge.net/project/showfiles.php?group_id=1369&#... That said, I'd like to take a moment to express my frustration with installing or compiling PyGtkGLExt ( http://gtkglext.sourceforge.net/ http://gtkglext.sourceforge.net/ ) on Windows with Python 2.6. The only installers I've found for PyGtkGLExt are for Python 2.4 ( http://www.stephanebrunet.net/gtkglext/pygtkglext-1.1.0.win32-py2.4.exe http://www.stephanebrunet.net/gtkglext/pygtkglext-1.1.0.win3... ) and Python 2.5 ( http://www.stephanebrunet.net/gtkglext/pygtkglext-1.1.0.win32-py2.5.exe http://www.stephanebrunet.net/gtkglext/pygtkglext-1.1.0.win3... ). Building it from source seems like a daunting task. Various build system utilities have to be installed first, such as MingW + MSYS, and pkg-config (which seems incredibly difficult to set up correctly on Windows). Also, it appears that the default maketools installed with MingW are too out-of-date to be used to build it. So far, upgrading MingW's autoconf / m4 / other tools is proving to be difficult at best. So at this point I'm planning to downgrade from Python 2.6 to 2.5 to save time, since it's basically impossible for non-ninjas to install or build PyGtkGLExt on Windows. I guess sticking with older versions is sometimes beneficial...
jwilliams 18y ago
There are benchmarks on the Release page: http://code.google.com/p/unladen-swallow/wiki/Releases http://code.google.com/p/unladen-swallow/wiki/Releases They seem to actually have a solid (early) release, and it does demonstrate some impressive speed improvements.
- bd 18y ago
  It would be interesting to see how it compares to Psyco. They do have one test for Psyco, but unfortunately results are not included in the wiki (Spitfire, which is Psycoed version of SlowSpitfire). From my experience, Psyco performs very well exactly in situations where also they got more significant speedups (like pickling/unpickling).
deleted 18y ago
[deleted]
pieter 18y ago
Good, the more projects that start using LLVM the better. There's also a GSOC project idea at the mono page (http://www.mono-project.com/StudentProjects#LLVM_Back-End http://www.mono-project.com/StudentProjects#LLVM_Back-End) for targetting LLVM from the mono JIT. This way more projects can share the same optimizations, which would hopefully mean less double efforts..
- bravura 18y ago
  How does unladen-swallow compare to PyPy? PyPy has an LLVM backend: http://codespeak.net/pypy/dist/pypy/doc/home.html http://codespeak.net/pypy/dist/pypy/doc/home.html
  - dave_au 18y ago
    The last time the LLVM backend was mentioned on the PyPy blog it was mentioned near the phrase "unmaintained and at least partially broken" Link: http://morepypy.blogspot.com/2008/10/sprint-discussions-release-planning.html http://morepypy.blogspot.com/2008/10/sprint-discussions-rele...
lacker 18y ago
An interesting note from the wiki page: For charting the long-term performance trend of the project, Unladen Swallow makes use of Google's standard internal performance measurement framework. Project members will post regular performance updates to the mailing lists.
mustpax 18y ago
The relevant papers section is very cool. Great reading for VM/JIT compiler geeks: http://code.google.com/p/unladen-swallow/wiki/RelevantPapers http://code.google.com/p/unladen-swallow/wiki/RelevantPapers God, I'm a total sucker for a project with a good bibliography. Must resist urge to go to grad school.
- batasrki 18y ago
  Yeah, I'll be reading through those as well. I like nothing better than reading a paper out of my depth and then learning as much as I can about the subject in order to understand what I've read.
snprbob86 18y ago
I love the name! In case anyone missed the reference: http://www.armory.com/swallowscenes.html http://www.armory.com/swallowscenes.html
curtis 18y ago
I've been assuming that somebody inside Google was busy porting Python to run on top of the V8 JavaScript engine. Turns out some Google guys are working on refactoring CPython into a VM with dynamic compilation instead. Go figure. (Strictly speaking these two alternatives are not mutually exclusive, of course.)

105 points by durin42 18y ago

The Project Plan at http://code.google.com/p/unladen-swallow/wiki/ProjectPlan http://code.google.com/p/unladen-swallow/wiki/ProjectPlan is way more interesting than the linked Google code page.

mcav 18y ago

A worthy goal, though I would have rather they just work based off of Python 3.0. By the time they achieve real, substantial progress, the momentum should be securely in 3.0's favor.

mgreenbe 18y ago
I don't think the tide is going to completely turn in the next three or four months. If this project meets its goals, I think CP2.6 -> US2.6 -> US3.0 is a more alluring migration path. If it truly succeeds, and CPython uses LLVM as its core, then the point is moot. I think 2.6 is the right place to start -- it's more stable and has a larger codebase. To hedge a little, I say all this from a distance: I don't use Python on a daily -- or even monthly -- basis.
spamizbad 18y ago
I'm not so sure Python 3.0 was really designed to carry any real momentum for "Python 3000" instead, it seems to serve as a base platform for future migration. I understand Guido and the Python team haven't explicitly stated that, but that certainly seems to be what they signaled. I don't expect see widespread support until 3.2, with Python 3.1 likely being the signal for people to begin serious migration efforts. As a Python developer myself, I couldn't move to 3.0 even if I wanted to: none of the open-source libraries I depend on have even begun to look into 3.0 support (Numpy won't even support Python 2.6 until its upcoming 1.3 release). Besides that, from a practical standpoint I'd guess the Unladen-Swallow team wants a codebase on more solid footing, like the mature 2.6.x series, as opposed to Python 3.x which is in a greater state of flux.
- palish 18y ago
  Actually, Numpy 1.3 is available and does work with Python 2.6, in case you want to upgrade: C:\Users\Shawn>python Python 2.6.1 (r261:67517, Dec 4 2008, 16:51:00) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.array( [1, 2, 3] ) array([1, 2, 3]) >>> The Numpy SourceForge download page has the 2.6 installer: http://sourceforge.net/project/showfiles.php?group_id=1369&package_id=175103 http://sourceforge.net/project/showfiles.php?group_id=1369&#... That said, I'd like to take a moment to express my frustration with installing or compiling PyGtkGLExt ( http://gtkglext.sourceforge.net/ http://gtkglext.sourceforge.net/ ) on Windows with Python 2.6. The only installers I've found for PyGtkGLExt are for Python 2.4 ( http://www.stephanebrunet.net/gtkglext/pygtkglext-1.1.0.win32-py2.4.exe http://www.stephanebrunet.net/gtkglext/pygtkglext-1.1.0.win3... ) and Python 2.5 ( http://www.stephanebrunet.net/gtkglext/pygtkglext-1.1.0.win32-py2.5.exe http://www.stephanebrunet.net/gtkglext/pygtkglext-1.1.0.win3... ). Building it from source seems like a daunting task. Various build system utilities have to be installed first, such as MingW + MSYS, and pkg-config (which seems incredibly difficult to set up correctly on Windows). Also, it appears that the default maketools installed with MingW are too out-of-date to be used to build it. So far, upgrading MingW's autoconf / m4 / other tools is proving to be difficult at best. So at this point I'm planning to downgrade from Python 2.6 to 2.5 to save time, since it's basically impossible for non-ninjas to install or build PyGtkGLExt on Windows. I guess sticking with older versions is sometimes beneficial...

jwilliams 18y ago

There are benchmarks on the Release page: http://code.google.com/p/unladen-swallow/wiki/Releases http://code.google.com/p/unladen-swallow/wiki/Releases They seem to actually have a solid (early) release, and it does demonstrate some impressive speed improvements.

bd 18y ago
It would be interesting to see how it compares to Psyco. They do have one test for Psyco, but unfortunately results are not included in the wiki (Spitfire, which is Psycoed version of SlowSpitfire). From my experience, Psyco performs very well exactly in situations where also they got more significant speedups (like pickling/unpickling).

deleted 18y ago

[deleted]

pieter 18y ago

Good, the more projects that start using LLVM the better. There's also a GSOC project idea at the mono page (http://www.mono-project.com/StudentProjects#LLVM_Back-End http://www.mono-project.com/StudentProjects#LLVM_Back-End) for targetting LLVM from the mono JIT. This way more projects can share the same optimizations, which would hopefully mean less double efforts..

bravura 18y ago
How does unladen-swallow compare to PyPy? PyPy has an LLVM backend: http://codespeak.net/pypy/dist/pypy/doc/home.html http://codespeak.net/pypy/dist/pypy/doc/home.html
- dave_au 18y ago
  The last time the LLVM backend was mentioned on the PyPy blog it was mentioned near the phrase "unmaintained and at least partially broken" Link: http://morepypy.blogspot.com/2008/10/sprint-discussions-release-planning.html http://morepypy.blogspot.com/2008/10/sprint-discussions-rele...

lacker 18y ago

An interesting note from the wiki page: For charting the long-term performance trend of the project, Unladen Swallow makes use of Google's standard internal performance measurement framework. Project members will post regular performance updates to the mailing lists.

mustpax 18y ago

The relevant papers section is very cool. Great reading for VM/JIT compiler geeks: http://code.google.com/p/unladen-swallow/wiki/RelevantPapers http://code.google.com/p/unladen-swallow/wiki/RelevantPapers God, I'm a total sucker for a project with a good bibliography. Must resist urge to go to grad school.

batasrki 18y ago
Yeah, I'll be reading through those as well. I like nothing better than reading a paper out of my depth and then learning as much as I can about the subject in order to understand what I've read.

snprbob86 18y ago

I love the name! In case anyone missed the reference: http://www.armory.com/swallowscenes.html http://www.armory.com/swallowscenes.html

curtis 18y ago

I've been assuming that somebody inside Google was busy porting Python to run on top of the V8 JavaScript engine. Turns out some Google guys are working on refactoring CPython into a VM with dynamic compilation instead. Go figure. (Strictly speaking these two alternatives are not mutually exclusive, of course.)