6 ms·

A beginners guide to using Python for performance computing

46 points by milvakili 15y ago

How fast we get with python. By the way i m a perl geek:)

pjin 15y ago
> Finally, for comparison we implemented this in simple C++ (nothing fancy) without any Python. One would expect that the C++ code would be faster but surprisingly, not by much! Given the fact that it's so easy to develop with Python, this speed reduction is not very significant. This is the problem I have with Python+Scipy v. MATLAB/C++/YourFavoritePlatform comparisons. I agree that Python code can be clean and clear, but once you start to worry about vectorizing, inlining, manual loop optimizations, or a C or Fortran FFI, then you end up with the same solution that say MATLAB users have to deal with by writing mex functions in C. Granted there are differences between Language A v. Language B, one may be more concise or their functions are _even higher-order_ than the other's. Point is, once you go down the road of "let's optimize this dynamic language script," your choices are not that different. You lose the "ease" of developing with Python, and if you really performance then it's ultimately inferior to just writing it in C++ or even Fortran.
- andreasvc 15y ago
  I think you're forgetting the scenario where you have a lot of Python code and only a small part is performance critical. You can speed that part up with something like Cython, and directly integrate it with the rest of the code. If you would have to rewrite it all in C++ it would be a tedious and buggy process, the same goes for ad-hoc interfacing with text files.
- juiceandjuice 15y ago
  Many times performance computing is about 1% optimizable code and 99% setup. Python is much lovelier for that 99% than matlab is.
- Derbasti 15y ago
  I have done that in the past. But writing a mex function is a lot more tedious than writing a weave.inline string. Think of all the type conversions you have to put up with in mex. Think about how you have to set up a compile infrastructure and how you have to recompile manually for every target platform. In most cases, I will use weave.inline only for the innermost loop, and all the compilation issues and type conversions will be taken care of for me.
hugh3 15y ago
I once tried taking one of my C++ simulation codes (dealing with diagonalization of big memory-hogging matrices) and rewriting it using scipy. It was a lot slower. I don't have a good idea of exactly what situations you'll find scipy being significantly slower than C++. Maybe it was the fact I was playing with thousands-by-thousands arrays. But I do worry that there are folks out there burning millions of CPU hours running scipy without realizing that they really ought to be burning mere 10^5s of CPU hours.
- stephth 15y ago
  I often wonder why there is so little interest in a high level language that would compile to pure C++, with a syntax that makes code more succinct and friendlier, just like CoffeeScript compiles to JavaScript. We have a clear problem: higher level languages make programmers more productive, but they have performance issues. Because of those, developers that need performance fall back to C++ (look at videogame development, C++ is there to stay). So why not make a language that is high level, but compiles to pure, cross platform, C++? Think of the productivity win: a higher level language that rarely needs optimizations. Obviously it would have to come with some lower level semantics, like memory management (although it could maybe be solved somehow like Apple did with Automatic Reference Counting, which is basically a preprocessor) or static typing (although the compiler can sometimes be instructed to guess the type, see the := operator in ooc [1]), but even if some lower level semantics are unavoidable, it still seems like a huge win. Yet every time I saw the idea mentioned online it's been either mostly ignored or treated as stupid idea, and the very few projects that attempted to go in that direction never took off. Maybe I'm missing something? If so I'd love to know what it is. No matter how fancy and awesome higher level languages are, we keep going back to C++ for performance, and that's probably never going to change (at least until processors are stupidly fast). So why not make it easier? The CoffeeScript approach has largely been proven to work. [1] http://docs.ooc-lang.org/language/syntax.html#declarations http://docs.ooc-lang.org/language/syntax.html#declarations (note that ooc is a good step in this direction, it compiles to C99, but then takes step back: it depends on a garbage collector)
  - SwellJoe 15y ago
kyt 15y ago
One of the advantages to implementing these types of problems in C++ is you can use compile-time optimizations, often resulting in 2-3x speed-up. The results may be a lot different when compiling with just -O2 -ffast-math.
spatters 15y ago
Cython is pretty excellent for this sort of thing. The article contains a link to a Cython solution written by Travis Oliphant showing how concise (and fast) it is. I just used Cython to speed up a non-vectorisable bit of code in a Gibbs sampling algorithm and I was impressed by the performance increase and ease of use.
kingkilr 15y ago
I wrote an implementation of this using Python's array module, and doing manual index calculations for PyPy: http://www.reddit.com/r/programming/comments/hh8uj/a_beginners_guide_to_using_python_for_performance/c1vgqdo http://www.reddit.com/r/programming/comments/hh8uj/a_beginne... . We'll soon have NumPy multi-dimensional arrays and thus can just run that code, but for now... we're doing OK I think.

46 points by milvakili 15y ago

How fast we get with python. By the way i m a perl geek:)

pjin 15y ago

> Finally, for comparison we implemented this in simple C++ (nothing fancy) without any Python. One would expect that the C++ code would be faster but surprisingly, not by much! Given the fact that it's so easy to develop with Python, this speed reduction is not very significant. This is the problem I have with Python+Scipy v. MATLAB/C++/YourFavoritePlatform comparisons. I agree that Python code can be clean and clear, but once you start to worry about vectorizing, inlining, manual loop optimizations, or a C or Fortran FFI, then you end up with the same solution that say MATLAB users have to deal with by writing mex functions in C. Granted there are differences between Language A v. Language B, one may be more concise or their functions are _even higher-order_ than the other's. Point is, once you go down the road of "let's optimize this dynamic language script," your choices are not that different. You lose the "ease" of developing with Python, and if you really performance then it's ultimately inferior to just writing it in C++ or even Fortran.

andreasvc 15y ago
I think you're forgetting the scenario where you have a lot of Python code and only a small part is performance critical. You can speed that part up with something like Cython, and directly integrate it with the rest of the code. If you would have to rewrite it all in C++ it would be a tedious and buggy process, the same goes for ad-hoc interfacing with text files.
juiceandjuice 15y ago
Many times performance computing is about 1% optimizable code and 99% setup. Python is much lovelier for that 99% than matlab is.
Derbasti 15y ago
I have done that in the past. But writing a mex function is a lot more tedious than writing a weave.inline string. Think of all the type conversions you have to put up with in mex. Think about how you have to set up a compile infrastructure and how you have to recompile manually for every target platform. In most cases, I will use weave.inline only for the innermost loop, and all the compilation issues and type conversions will be taken care of for me.

hugh3 15y ago

I once tried taking one of my C++ simulation codes (dealing with diagonalization of big memory-hogging matrices) and rewriting it using scipy. It was a lot slower. I don't have a good idea of exactly what situations you'll find scipy being significantly slower than C++. Maybe it was the fact I was playing with thousands-by-thousands arrays. But I do worry that there are folks out there burning millions of CPU hours running scipy without realizing that they really ought to be burning mere 10^5s of CPU hours.

stephth 15y ago
I often wonder why there is so little interest in a high level language that would compile to pure C++, with a syntax that makes code more succinct and friendlier, just like CoffeeScript compiles to JavaScript. We have a clear problem: higher level languages make programmers more productive, but they have performance issues. Because of those, developers that need performance fall back to C++ (look at videogame development, C++ is there to stay). So why not make a language that is high level, but compiles to pure, cross platform, C++? Think of the productivity win: a higher level language that rarely needs optimizations. Obviously it would have to come with some lower level semantics, like memory management (although it could maybe be solved somehow like Apple did with Automatic Reference Counting, which is basically a preprocessor) or static typing (although the compiler can sometimes be instructed to guess the type, see the := operator in ooc [1]), but even if some lower level semantics are unavoidable, it still seems like a huge win. Yet every time I saw the idea mentioned online it's been either mostly ignored or treated as stupid idea, and the very few projects that attempted to go in that direction never took off. Maybe I'm missing something? If so I'd love to know what it is. No matter how fancy and awesome higher level languages are, we keep going back to C++ for performance, and that's probably never going to change (at least until processors are stupidly fast). So why not make it easier? The CoffeeScript approach has largely been proven to work. [1] http://docs.ooc-lang.org/language/syntax.html#declarations http://docs.ooc-lang.org/language/syntax.html#declarations (note that ooc is a good step in this direction, it compiles to C99, but then takes step back: it depends on a garbage collector)
- SwellJoe 15y ago

kyt 15y ago

One of the advantages to implementing these types of problems in C++ is you can use compile-time optimizations, often resulting in 2-3x speed-up. The results may be a lot different when compiling with just -O2 -ffast-math.

spatters 15y ago

Cython is pretty excellent for this sort of thing. The article contains a link to a Cython solution written by Travis Oliphant showing how concise (and fast) it is. I just used Cython to speed up a non-vectorisable bit of code in a Gibbs sampling algorithm and I was impressed by the performance increase and ease of use.

kingkilr 15y ago

I wrote an implementation of this using Python's array module, and doing manual index calculations for PyPy: http://www.reddit.com/r/programming/comments/hh8uj/a_beginners_guide_to_using_python_for_performance/c1vgqdo http://www.reddit.com/r/programming/comments/hh8uj/a_beginne... . We'll soon have NumPy multi-dimensional arrays and thus can just run that code, but for now... we're doing OK I think.