5 ms·

Cython, Numpy, Numba (and others) are what make me skeptical that any of the numerical computing competitors to Python (Julia, or to a lesser extent Lua/JS/Cloj

Cython, Numpy, Numba (and others) are what make me skeptical that any of the numerical computing competitors to Python (Julia, or to a lesser extent Lua/JS/Clojure even compiled Scala etc) can displace it at. Why would you abandon this wonderfully friendly, malleable language with easily the largest set of options for anything you might like to do, for something nominally faster on an artificial benchmark (which inevitably ignores the above 3 libraries), when by dipping into any of these you get C-speed in such an accessible and friendly way, most of the time beating the competitors easily on performance. The only real threats to Python to me come would from a properly parallelized language that would use the GPU/multicore natively in vector form. I don't see that anywhere yet.

zzleeper 11y ago
I've seen some very impressive Julia libraries that perform almost at the level of C code, so I'm not sure if the gap that Julia needs to fill is that large. Also, a quick question: which one is preferred nowadays, Cython or Numba?
- dagw 11y ago
  Cython is much more mature, works on basically all python code and basically never leads to slowdowns compared to pure python. Unfortunately non of that can really be said about Numba. That being said, when Numba works it's great and is much easier to work with than cython.
  - synparb 11y ago
    I'll echo @dagw's comments. Cython has been rock solid for me for a long time and it is my go to for any sort of external c/c++ library interfacing. That said, I find myself using Numba more and more in the places that I can as Numba has gotten significantly better in the last 6 months or so. It's still not a complete replacement for cython (and I don't think it ever will or intends to be), but for hot spot numerical calculations it's really nice since it's much faster to test things out since it doesn't involve the boiler plate required by Cython, and doesn't require the (often slow) compilation times. In Numba 0.21.0, on-disk caching of jit'd code was also introduced, which was one of the major sticking points for us to put Numba into production in areas where we needed faster start-up times. Before we could really only use Cython because we required the start-up times available only from AOT compilation. That all said, Numba has been a bit buggy for me at times, although these get squashed pretty quickly. I've only found a single bug in Cython in all of the years I've been using it, and it's amazing how quickly Robert Bradshaw or Stefan Behnel respond and fix things considering Cython is not their full time job.
- shoyer 11y ago
  It really depends on your use case. Cython is more flexible and suitable for libraries, but Numba is easier to write. I elaborated on this in a recent blog post: http://eng.climate.com/2015/04/09/numba-vs-cython-how-to-choose/ http://eng.climate.com/2015/04/09/numba-vs-cython-how-to-cho...
- andreasvc 11y ago
  The gap that Julia has to fill in terms of library ecosystem is definitely large. Numba is specific to numeric/array-oriented code. Cython is general-purpose and can also be used to implement e.g. datastructures.
  - tadlan 11y ago
    Numba has list, Named tuples, and is working on define your own jit classes.
junke 11y ago
Regarding the GPU, you might find an interest for MGL-MAT[1]: "MAT is a linear algebra library for working with multi-dimensional arrays which supports efficient interfacing to foreign and CUDA code. BLAS and CUBLAS bindings are available.". The author's blog is quite interesting to read (see for example [2]). [1] http://cliki.net/mgl-mat http://cliki.net/mgl-mat [2] I http://quotenil.com/On-the-Design-of-Matrix-Libraries.html http://quotenil.com/On-the-Design-of-Matrix-Libraries.html
pathsjs 11y ago
The problem with the Python ecosystem is that there exist a lot of libraries for scientific computation mostly incompatible with each other (blaze, numpy, numba, numexpr, dask...). Some of them try to reintroduce types in order to compile to native code, thereby losing the advantages of dynamic typing. In short, it is a mess, and everyone is trying to add on this because the existing ecosystem is big and the cost of rewriting everything is deemed too high, so people try to fix python limitations at the library level, sometimes making python more cumbersome to write tha a conventional statically typed language (I am looking at theano). I don't think that Lua is that much of an improvement, and I do not have big hopes for the JVM. The growing number of projects trying to improve the situation with value types, off-heap allocations and C compatibility are a sign that the VM is not suitable for scientific computing and will not be much better for a while. I instead hope that Nim will take a leading role for scientific computation. It has an easy syntax and is flexible enough to write libraries such as numpy. At the same time it is fast and C compatibility is trivial. Of course, there is a library ecosystem to build, but I think that this is more doable than trying to work against the language itself. I am trying to start this effort with https://github.com/unicredit/linear-algebra https://github.com/unicredit/linear-algebra but the road is very long and I hope it takes off. I cannot really comment on Julia as I do not know it welll enough
- osullivj 11y ago
  So is this official Unicredit open source? I'm impressed if it is...
  - pathsjs 11y ago
    Yes, there are a few open-source projects that you can check on our github :-)
- shoyer 11y ago
  > The problem with the Python ecosystem is that there exist a lot of libraries for scientific computation mostly incompatible with each other (blaze, numpy, numba, numexpr, dask...). I actually disagree completely with your specific examples. These projects have mostly orthogonal goals and are actually quite compatible with each other -- they all speak NumPy arrays. In fact, three of them (Blaze, Numba and Dask) are sponsored by the same company (Continuum Analytics). Yes, the SciPy ecosystem is a complex beast. There are lots of complementary projects and its not always clear what the best tool for the job is. There are certainly improvements we could make for easier cross-compatibility between libraries (and there are likely projects that could be consolidated), but the number of options you have for scientific computing in Python is an indication of a very robust ecosystem.
versteegen 11y ago
I disagree, and see the proliferation of all these performance-focused libraries/compilers as proving that CPython's slow speed is a huge problem. I haven't used Numba, but after my experience with others I conclude that you just can't totally escape the fact that CPython is slow. Having to use tools like Cython is a big tax that's cost me weeks or months of time. Firstly numpy/scipy don't help if you can't write your algorithms in terms of operations on multidimensional arrays, and numpy overheads (creating and accessing ndarrays) are actually prohibitively large if the arrays aren't large, easy to be slower than straight Python. First I tried writing code (scientific stuff) in type-annotated Cython. But it turns out that data structures are the bottleneck if your algorithms need to read/write something other than a bunch of numpy ndarrays. If you try to use lists, dicts, etc, you still go through the Python runtime so get little speed benefit over Python. (Cython optimises ndarray accesses.) So I ended up writing C++ code and interfacing to it using Cython. But now I have to write a huge amount of code to translate between the Python/Numpy and C++ datastructures. And it's bug prone due to memory allocation and ownership. Using multiple poorly compatible languages is a miserable experience. Julia sounds fantastic. Don't get me wrong though, I love Python, but it wasn't designed for scientific computing. And most of the time, numpy, scipy & friends are all that you need or want.
- Tarlitz 11y ago
  Your second point is spot on. I realized this when I tried to optimize my numpy code that does millions of dot products between 3x1 vectors and 3x3 matrices to no avail. I realized there is an awesome little library called tinyarray, that doesn't have the overhead of numpy, and is compatible with the basic numpy syntax. I exchanged all my numpy arrays for tinyarrays and got the easiest 10x speed up ever.
  - versteegen 11y ago
    Thanks for mentioning that, very useful! If you look in the numpy source you can begin to understand why the overheads are so large: for every operation it's first necessary to apply rules for broadcasting between different sizes and types, plus often being callable with a number of different function signatures (that includes __getitem__). And in many cases all of that is implemented in Python.

zzleeper 11y ago

I've seen some very impressive Julia libraries that perform almost at the level of C code, so I'm not sure if the gap that Julia needs to fill is that large. Also, a quick question: which one is preferred nowadays, Cython or Numba?

dagw 11y ago
Cython is much more mature, works on basically all python code and basically never leads to slowdowns compared to pure python. Unfortunately non of that can really be said about Numba. That being said, when Numba works it's great and is much easier to work with than cython.
- synparb 11y ago
  I'll echo @dagw's comments. Cython has been rock solid for me for a long time and it is my go to for any sort of external c/c++ library interfacing. That said, I find myself using Numba more and more in the places that I can as Numba has gotten significantly better in the last 6 months or so. It's still not a complete replacement for cython (and I don't think it ever will or intends to be), but for hot spot numerical calculations it's really nice since it's much faster to test things out since it doesn't involve the boiler plate required by Cython, and doesn't require the (often slow) compilation times. In Numba 0.21.0, on-disk caching of jit'd code was also introduced, which was one of the major sticking points for us to put Numba into production in areas where we needed faster start-up times. Before we could really only use Cython because we required the start-up times available only from AOT compilation. That all said, Numba has been a bit buggy for me at times, although these get squashed pretty quickly. I've only found a single bug in Cython in all of the years I've been using it, and it's amazing how quickly Robert Bradshaw or Stefan Behnel respond and fix things considering Cython is not their full time job.
shoyer 11y ago
It really depends on your use case. Cython is more flexible and suitable for libraries, but Numba is easier to write. I elaborated on this in a recent blog post: http://eng.climate.com/2015/04/09/numba-vs-cython-how-to-choose/ http://eng.climate.com/2015/04/09/numba-vs-cython-how-to-cho...
andreasvc 11y ago
The gap that Julia has to fill in terms of library ecosystem is definitely large. Numba is specific to numeric/array-oriented code. Cython is general-purpose and can also be used to implement e.g. datastructures.
- tadlan 11y ago
  Numba has list, Named tuples, and is working on define your own jit classes.

junke 11y ago

Regarding the GPU, you might find an interest for MGL-MAT[1]: "MAT is a linear algebra library for working with multi-dimensional arrays which supports efficient interfacing to foreign and CUDA code. BLAS and CUBLAS bindings are available.". The author's blog is quite interesting to read (see for example [2]). [1] http://cliki.net/mgl-mat http://cliki.net/mgl-mat [2] I http://quotenil.com/On-the-Design-of-Matrix-Libraries.html http://quotenil.com/On-the-Design-of-Matrix-Libraries.html

pathsjs 11y ago

The problem with the Python ecosystem is that there exist a lot of libraries for scientific computation mostly incompatible with each other (blaze, numpy, numba, numexpr, dask...). Some of them try to reintroduce types in order to compile to native code, thereby losing the advantages of dynamic typing. In short, it is a mess, and everyone is trying to add on this because the existing ecosystem is big and the cost of rewriting everything is deemed too high, so people try to fix python limitations at the library level, sometimes making python more cumbersome to write tha a conventional statically typed language (I am looking at theano). I don't think that Lua is that much of an improvement, and I do not have big hopes for the JVM. The growing number of projects trying to improve the situation with value types, off-heap allocations and C compatibility are a sign that the VM is not suitable for scientific computing and will not be much better for a while. I instead hope that Nim will take a leading role for scientific computation. It has an easy syntax and is flexible enough to write libraries such as numpy. At the same time it is fast and C compatibility is trivial. Of course, there is a library ecosystem to build, but I think that this is more doable than trying to work against the language itself. I am trying to start this effort with https://github.com/unicredit/linear-algebra https://github.com/unicredit/linear-algebra but the road is very long and I hope it takes off. I cannot really comment on Julia as I do not know it welll enough

osullivj 11y ago
So is this official Unicredit open source? I'm impressed if it is...
- pathsjs 11y ago
  Yes, there are a few open-source projects that you can check on our github :-)
shoyer 11y ago
> The problem with the Python ecosystem is that there exist a lot of libraries for scientific computation mostly incompatible with each other (blaze, numpy, numba, numexpr, dask...). I actually disagree completely with your specific examples. These projects have mostly orthogonal goals and are actually quite compatible with each other -- they all speak NumPy arrays. In fact, three of them (Blaze, Numba and Dask) are sponsored by the same company (Continuum Analytics). Yes, the SciPy ecosystem is a complex beast. There are lots of complementary projects and its not always clear what the best tool for the job is. There are certainly improvements we could make for easier cross-compatibility between libraries (and there are likely projects that could be consolidated), but the number of options you have for scientific computing in Python is an indication of a very robust ecosystem.

versteegen 11y ago

I disagree, and see the proliferation of all these performance-focused libraries/compilers as proving that CPython's slow speed is a huge problem. I haven't used Numba, but after my experience with others I conclude that you just can't totally escape the fact that CPython is slow. Having to use tools like Cython is a big tax that's cost me weeks or months of time. Firstly numpy/scipy don't help if you can't write your algorithms in terms of operations on multidimensional arrays, and numpy overheads (creating and accessing ndarrays) are actually prohibitively large if the arrays aren't large, easy to be slower than straight Python. First I tried writing code (scientific stuff) in type-annotated Cython. But it turns out that data structures are the bottleneck if your algorithms need to read/write something other than a bunch of numpy ndarrays. If you try to use lists, dicts, etc, you still go through the Python runtime so get little speed benefit over Python. (Cython optimises ndarray accesses.) So I ended up writing C++ code and interfacing to it using Cython. But now I have to write a huge amount of code to translate between the Python/Numpy and C++ datastructures. And it's bug prone due to memory allocation and ownership. Using multiple poorly compatible languages is a miserable experience. Julia sounds fantastic. Don't get me wrong though, I love Python, but it wasn't designed for scientific computing. And most of the time, numpy, scipy & friends are all that you need or want.

Tarlitz 11y ago
Your second point is spot on. I realized this when I tried to optimize my numpy code that does millions of dot products between 3x1 vectors and 3x3 matrices to no avail. I realized there is an awesome little library called tinyarray, that doesn't have the overhead of numpy, and is compatible with the basic numpy syntax. I exchanged all my numpy arrays for tinyarrays and got the easiest 10x speed up ever.
- versteegen 11y ago
  Thanks for mentioning that, very useful! If you look in the numpy source you can begin to understand why the overheads are so large: for every operation it's first necessary to apply rules for broadcasting between different sizes and types, plus often being callable with a number of different function signatures (that includes __getitem__). And in many cases all of that is implemented in Python.