6 ms·

NumPy aware dynamic Python compiler using LLVM

52 points by momeara 15y ago

dbecker 15y ago
I just flipped through the docs, and this looks amazing. It would be nice to see some benchmarks. If Numba lives up to expectations, Travis Oliphant should be knighted.
- momeara 15y ago
  What benchmarks would be the most relevant? speed.pypy.org?
  - dbecker 15y ago
    I think many people would think that's a good set of benchmarks. Personally, numerical computing is more important to me. So the 7 benchmarks listed at julialang.org seem great to me.
  - mynegation 15y ago
    There is also a computer language benchmark shootout: http://shootout.alioth.debian.org/ http://shootout.alioth.debian.org/
iskander 15y ago
This is pretty close to what I'm doing for my PhD research-- it's an annoyingly hard problem! Travis will probably make smarter design choices than me, but my project has ended up being of limited use due to the non-obvious subset of Python we are able to accelerate. Advice for anyone else trying to add a JIT on top of an existing language: either craft a DSL with extremely well defined boundaries or support the whole language from the beginning.
iskander 15y ago
From the numpy mailing list (http://mail.scipy.org/pipermail/numpy-discussion/2012-March/061303.html http://mail.scipy.org/pipermail/numpy-discussion/2012-March/...): (summarizing how Numba differs from Unladen Swallow) ----- >> what is the difference to http://www.python.org/dev/peps/pep-3146/ http://www.python.org/dev/peps/pep-3146/ ? To fully expound the difference would take a lot of discussion. But, summarizing: * Numba is not nearly as ambitious as US (Numba will be fine with some user-directed information and with a subset of the language that gets compiled) * Numba will focus on compilation rather than JITing --- in other words it won't be trying to detect hot-spots and compile segments (actually LLVM is not a good candidate for that sort of thing as it is not a very fast or memory-efficient compiler for a lot of reasons, I believe this is why PyPy does not use it anymore, for example). * Numba will be much closer to Cython in spirit than Unladen Swallow (or PyPy) --- people who just use Cython for a loop or two will be able to use Numba instead * Numba will borrow the idea of call-sites from IronPython wherein a function is replaced by a generic function that dispatches based on input types to either cached compiled code for the types specified or the generic function. * Numba will be mainly about trying to leverage the code-generation of LLVM which multiple hardware manufacturers are using (Intel's OpenCL support, Nvidia's PTX backend, Apple's CLang, etc.) for NumPy arrays -Travis
vgnet 15y ago
Some background (announcement, talk): https://news.ycombinator.com/item?id=3692055 https://news.ycombinator.com/item?id=3692055
deleted 15y ago
[deleted]
alexholehouse 15y ago
Judging my what I heard, Numba was quite the talk of the town at Pycon (though this could be totally biased - I wasn't there so can't comment first hand!)
keveman 15y ago
A somewhat similar project : https://github.com/copperhead/copperhead https://github.com/copperhead/copperhead

52 points by momeara 15y ago

I just flipped through the docs, and this looks amazing. It would be nice to see some benchmarks. If Numba lives up to expectations, Travis Oliphant should be knighted.

momeara 15y ago
What benchmarks would be the most relevant? speed.pypy.org?
- dbecker 15y ago
  I think many people would think that's a good set of benchmarks. Personally, numerical computing is more important to me. So the 7 benchmarks listed at julialang.org seem great to me.
- mynegation 15y ago
  There is also a computer language benchmark shootout: http://shootout.alioth.debian.org/ http://shootout.alioth.debian.org/

iskander 15y ago

This is pretty close to what I'm doing for my PhD research-- it's an annoyingly hard problem! Travis will probably make smarter design choices than me, but my project has ended up being of limited use due to the non-obvious subset of Python we are able to accelerate. Advice for anyone else trying to add a JIT on top of an existing language: either craft a DSL with extremely well defined boundaries or support the whole language from the beginning.

iskander 15y ago

From the numpy mailing list (http://mail.scipy.org/pipermail/numpy-discussion/2012-March/061303.html http://mail.scipy.org/pipermail/numpy-discussion/2012-March/...): (summarizing how Numba differs from Unladen Swallow) ----- >> what is the difference to http://www.python.org/dev/peps/pep-3146/ http://www.python.org/dev/peps/pep-3146/ ? To fully expound the difference would take a lot of discussion. But, summarizing: * Numba is not nearly as ambitious as US (Numba will be fine with some user-directed information and with a subset of the language that gets compiled) * Numba will focus on compilation rather than JITing --- in other words it won't be trying to detect hot-spots and compile segments (actually LLVM is not a good candidate for that sort of thing as it is not a very fast or memory-efficient compiler for a lot of reasons, I believe this is why PyPy does not use it anymore, for example). * Numba will be much closer to Cython in spirit than Unladen Swallow (or PyPy) --- people who just use Cython for a loop or two will be able to use Numba instead * Numba will borrow the idea of call-sites from IronPython wherein a function is replaced by a generic function that dispatches based on input types to either cached compiled code for the types specified or the generic function. * Numba will be mainly about trying to leverage the code-generation of LLVM which multiple hardware manufacturers are using (Intel's OpenCL support, Nvidia's PTX backend, Apple's CLang, etc.) for NumPy arrays -Travis

vgnet 15y ago

Some background (announcement, talk): https://news.ycombinator.com/item?id=3692055 https://news.ycombinator.com/item?id=3692055

deleted 15y ago

[deleted]

alexholehouse 15y ago

Judging my what I heard, Numba was quite the talk of the town at Pycon (though this could be totally biased - I wasn't there so can't comment first hand!)

keveman 15y ago

A somewhat similar project : https://github.com/copperhead/copperhead https://github.com/copperhead/copperhead