11 ms·

How to Make Python Run as Fast as Julia (2015)

203 points by aaronchall 9y ago

pjmlp 9y ago
I guess, it will take loosing market share to Julia for Python devs actually start taking into consideration making PyPy the canonical implementation. Rewriting libraries in C isn't making Python code run faster. The latest Julia conf had lots of cool presentations, and even though 1.0 might come only at the end of the year (with luck), the uptake among the research community is quite good for such a young language.
- marmaduke 9y ago
  There's little reason to move to PyPy when Numba is available, at least for scientific code. For backend work, I find the bottleneck is never Python, tbh
  - srean 9y ago
    Either Numba has improved by an order of magnitude since I looked last (congratulations that's fantastic, its a difficult problem to solve), or you haven't met the edge cases yet
- JulianWasTaken 9y ago
  What would it mean to make it the "canonical implementation"? Is there something preventing you from using PyPy today?
  - pjmlp 9y ago
    I mean the version installed by default that anyone can use without requiring root. I stop using Python long time ago.
    - orf 9y ago
      What would peoples reaction be when their existing small Python scripts take at least 3x as long to run?
      - pjmlp 9y ago
        If the community focused on making PyPy the canonical implementation, that would be surely a major effort to improve that scenario. Lisp, Scheme, Dylan, JavaScript, Julia don't suffer from such huge speed bump, on the contrary.
        orf 9y ago
        Neither does C. But that, and all the other languages you listed, are not Python. They have differing implementations, constraints and resources that make the comparison unfair and IMO invalid.
        pjmlp 9y ago
        C is not in the same league as the languages I mentioned, in regards to productivity and memory safety, while enjoying quite good performance. What is unfair about being as dynamic as Python and yet enjoying top quality JIT compilers? Just the community not caring about it and forcing everyone else to go down into C or somewhere else, it seems.
        orf 9y ago
        > Just the community not caring about it and forcing everyone else to go down into C or somewhere else, it seems. That, or the inherent dynamic nature of Python that makes it ridiculously hard to JIT (especially without breaking compatibility), and that a tiny subset of developers have enough experience (and will) to work on it as there is no Google willing to funnel buckets of money into making a V8 for Python, to the point where PyPy has reached an OK state after 10 years of development and a codebase that dwarfs CPython in size and complexity. I hate people who deride the community for not caring enough about adding a JIT. Yeah, that sure is the reason. Not mind-boggling technical limitations, nope, just pure laziness. Why don't you try? Exactly.
  - dagw 9y ago
    Is there something preventing you from using PyPy today? A lot of libraries aren't tested with/don't work with pypy.
    - orf 9y ago
      File issues with them. Use the github editor to add `pypy` to their `gitlab-ci.yml`, submit a merge request, and see if any tests fail.
- coldtea 9y ago
  >Rewriting libraries in C isn't making Python code run faster. No, but nobody cares for the purity of only-Python code. They care for running their calculation fast, and if that takes C-enabled modules they are totally fine with it. With things like numpy available, it's not like they have to write those calculations themselves. Besides Numpy smokes PyPy for the same kind of calculations (not that one would care to rewrite the tons of stuff available in there in pure Python).
  - srean 9y ago
    >No, but nobody cares for the purity of only-Python code. Let me call BS on that. I would prefer not to be forced to drop down to C if I could have it. I am quite sure many feel that way. I would definitely like the ease of calling an existing C or Fortran library that already does what I need. But that I need (almost to a fault) to context switch between languages is certainly not a high point especially when I am prototyping something new and for which there aren't any good libraries yet.
    - bpicolo 9y ago
      Numpy doesn't force you to drop down to C, though. That it's C under the hood is almost entirely irrelevant when making use of it
      - srean 9y ago
        Yes it absolutely doesnt. I was talking about things when you have to reach beyond Numpy/Scipy/Pandas/Scikits. For C or Fortran libraries and tools like those that are already there, Python is very pleasant to use.
  - pjmlp 9y ago
    If people didn't care, they wouldn't be moving into Julia, Go, Chapel,...
    - Majestic121 9y ago
      Do you have any source for this affirmation that people are moving from Python to Julia/Go/etc ? Those language are definitely growing, but so is Python and while there is movement between the languages I am not sure is a massive unilateral migration as you seem to imply.
- ChrisSD 9y ago
  Pypy is not that much faster than Python and is certainly still much slower than using a C library. Indeed, there's a limit to how fast python can be considering how many operations it needs to do for even simple statements (such as summing two integers[0]). This is the cost of being so extremely dynamic, more so than Javascript and many other scripting languages. [0] https://jakevdp.github.io/blog/2014/05/09/why-python-is-slow/#Python-Addition https://jakevdp.github.io/blog/2014/05/09/why-python-is-slow...
  - pjmlp 9y ago
    Even after reading that post, I doubt Python can be more dynamic than Lisp or Smalltalk, and yet they are behind the genesis of JIT compilers.
    - jerf 9y ago
      I don't know that it "is", but it certainly can be. In Python there's just too many ways to screw with the meaning of "a.b" at runtime; it could be in the class dict, it could be a property, it could be modified via a couple of methods on the class, it could be screwed with a couple of other ways too IIRC. You have to write some sort of code that either proves these things can't happen, which gets really hard as the program scales up, or code that checks for them happening. Other languages that can be "dynamic" but still have only one or two things to be checked can be much more easily JIT'ed.
      - pjmlp 9y ago
        Smalltalk has a message called becomes, where you literally change the complete representation of an object to something else. You can change code on the fly on the debugger and just do a redo of the failed call. When you change a specific class, or its meta-class, the whole system is automatically updated, reflecting the changes. I can give many more examples.
        pwang 9y ago
        I believe much similar magic is possible in Python, but you get to implement it on top of the relatively simple interactions between instance dict, class dict, and metaclass. They would not be bullet-proof, and they are not built into the default implementation. But they are very possible.
marmaduke 9y ago
It's easy to optimize small kernels. Less straightforward is optimizing an entire library in Python, especially if pieces need to be composed. For this, the most interesting library I've come across is loopy which handles things like fusing kernels, unrolling loops _before_ code generation. https://documen.tician.de/loopy/ https://documen.tician.de/loopy/
kristofferc 9y ago
Isn't adding things like memoization or writing the code in C (Cython) missing the whole point of the benchmark, which is to test the overhead of recursion / function calls in the language itself?
- knlje 9y ago
  Users of programming languages interested in the end result (output of the program etc) do not care. They want the fastest performing language for the job. Julia website has been misleading people. Due to those claims, I spent a week porting some of my simulation code to Julia before I realized that it is actually slower in (optimized) Julia than in optimized Python.
  - kristofferc 9y ago
    For those who only care about the output of fib(20) there are more efficient methods than any of the Julia or Python implementations posted in the link, e.g. lookup tables. The assumption in benchmarks is of course that the results carry over to other use cases. Here, what is being tested is the overhead in recursion, nothing else. The fact that it happens to be Fibonacci-numbers that is being computed is irrelevant.
    - coldtea 9y ago
      Why, is anybody restricted to using recursion in their solutions?
      - kristofferc 9y ago
        You are not restricted to recursion, loops, functions etc, but these language constructs might be useful in situations, and therefore the performance (overhead) of these constructs is interesting.
    - Beltiras 9y ago
      Isn't that basically what he does when he adds caching? I know it's not a static lookup table but you could prep it by invoking it with a sufficiently large n.
stevesimmons 9y ago
Can the HN title add "(2015)"?
drej 9y ago
There are a few things I consider when I try to improve performance of my code. 1) I really like the advice of "make it run, make it right, make it fast" [0]. First I build it, then I write a thorough test harness (or make multiple versions run at the same time and check their results at runtime), then I rewrite it. 2) I don't really like optimising to the core, like it's mentioned in the article, things like Cython and numba often add dependencies that don't port well and they may reduce legibility and maintainability (that's more about Cython than numba). What I find most useful, and this should be obvious, that algorithm/data structure changes often yield the biggest benefits. 3) The old rule that if you speed up code that only runs 1% of the time, you're not gaining that much, I try to keep that in mind. (There are exceptions, of course.) 4) Performance is not just runtime, it starts with coding it up in the first place. So start to finish, you're not always better off with a fast language. If it's a one-off, it can be more performant to use a slow language, if it's a tool that will run untouched, you might want to spend some time tinkering with it. Etc. All of this seems obvious in retrospect, but it took me a while to appreciate these principles. [0] https://www.facebook.com/notes/kent-beck/mastering-programming/1184427814923414 https://www.facebook.com/notes/kent-beck/mastering-programmi...
- eesmith 9y ago
  A side historical note which doesn't affect anything; http://wiki.c2.com/?MakeItWorkMakeItRightMakeItFast http://wiki.c2.com/?MakeItWorkMakeItRightMakeItFast traces that expression back to Stephen C. Johnson and Brian W. Kernighan's "The C Language and Models for Systems Programming" in Byte magazine (August 1983) - "the strategy is definitely: first make it work, then make it right, and, finally, make it fast."
  - moomin 9y ago
    The make it work/make it right principle is also referred to as "tracer bullets" in certain circles: if you get the bare bones full process working it's much easier to flesh it out than doing it one "fully specced" component at a time.
hojijoji 9y ago
Make "Python" run faster by shifting all the runtime heavy computation to C-implementations... this article is missing the whole point
- coldtea 9y ago
  Python is a scripting language, so its strong point is being used as a glue, and half of its standard library is implemented in C anyway. Plus, the "C-implementations" he mentions are available as readily usable modules (numpy), or semi-transparent jit/aot compilers only needing a few annotations (Cython, numba), not actual C you have to write. Besides, isn't the whole point: finish your project fast with the language you know using whatever it makes available to easily speed your code up? As opposed to: "be a purist and not use wrapped libs written in another language". Who cares for that? Even if it comes up, it's to avoid the hassle of having to deal with an additional language, setup etc -- which for numpy, Cython etc is almost none-existent (as you don't need to actually deal with C). And of course, despite the purity of Julia's "single language", the hassle of moving to a totally different language, which few use, is not yet stable in syntax, compiler etc, and has fewer libs, should also be considered...
  - deleted 9y ago
    [deleted]
  - FabHK 9y ago
    The specific purpose of the benchmark, though, is to compare implementations of the same algorithm natively in the language itself, as explained explicitly on the Julia website just under the table of benchmark results (see quote below). As such, I do think the article misses the point somewhat. Of course, if there's a numpy function that does what you want, you'd use it in real life. But what if there isn't? The nice thing about Julia is that the function can be written in Julia itself, and fast. > It is important to note that these benchmark implementations are not written for absolute maximal performance (the fastest code to compute fib(20) is the constant literal 6765). Rather, all of the benchmarks are written to test the performance of specific algorithms implemented in each language. In particular, all languages use the same algorithm: the Fibonacci benchmarks are all recursive while the pi summation benchmarks are all iterative; the “algorithm” for random matrix multiplication is to call the most obvious built-in/standard random-number and matmul routines (or to directly call BLAS if the language does not provide a high-level matmul), except where a matmul/BLAS call is not possible (such as in JavaScript). The point of these benchmarks is to compare the performance of specific algorithms across language implementations, not to compare the fastest means of computing a result, which in most high-level languages relies on calling C code. https://julialang.org https://julialang.org
unfamiliar 9y ago
I had a lot of experience a few years ago with Julia, having ported a lot of my numerical code in the language. I ended up severely disappointed with it for moderately complex numerical computing projects. The language was really nice; the problem was terrible performance, and I believe that this was caused by bad design of the memory semantics. Even in optimised code, there were temporary objects and copies all over the place, and they were very hard to eliminate without resorting to using global arrays everywhere.
- vanderZwan 9y ago
  > Even in optimised code, there were temporary objects and copies all over the place, and they were very hard to eliminate without resorting to using global arrays everywhere. Could you elaborate a bit with some examples, because this goes against everything I understand about the strengths of the language: (without having used it beyond a few first examples myself) - you can define types and operations on them down to the bits themselves, giving you a lot of control over the memory (and definitely a lot more than Python) - it has a lot of support for in-place mutation out of the box - globals are almost always bad news for performance Sure, adding two arrays with C = A + B .. produces an intermediate array, yes, but if you know A and B are not reused, wouldn't using: A += B C = A .. instead be enough?
  - ffriend 9y ago
    Even better would be to write (see dot vectorization [1]): C .= A .+ B Benchmarks for 3 matrices of size 1000x1000: julia> using BenchmarkTools julia> @benchmark C = A + B BenchmarkTools.Trial: memory estimate: 7.63 MiB allocs estimate: 2 -------------- minimum time: 2.359 ms (0.00% GC) median time: 2.713 ms (0.00% GC) mean time: 3.794 ms (28.81% GC) maximum time: 62.708 ms (95.27% GC) -------------- samples: 1314 evals/sample: 1 julia> @benchmark C .= A .+ B BenchmarkTools.Trial: memory estimate: 128 bytes allocs estimate: 4 -------------- minimum time: 1.232 ms (0.00% GC) median time: 1.320 ms (0.00% GC) mean time: 1.356 ms (0.00% GC) maximum time: 2.572 ms (0.00% GC) -------------- samples: 3651 evals/sample: 1 Note that memory usage dropped from 7.63MiB to 128 bytes. [1]: https://docs.julialang.org/en/stable/manual/functions/#man-vectorized-1 https://docs.julialang.org/en/stable/manual/functions/#man-v...
cs702 9y ago
If you really care about extracting every possible ounce of performance out of Python's scientific stack, which relies extensively on Numpy, the best practical guide I have found for doing that is "From Python to Numpy," by Nicolas P. Rougier of Inria: https://www.labri.fr/perso/nrougier/from-python-to-numpy/ https://www.labri.fr/perso/nrougier/from-python-to-numpy/ Here's a typical example of the kinds of optimizations this guide teaches you, in this case by avoiding the creation of temporary copies of Numpy arrays in memory: # Create two int arrays, each filled with with one billion 1's. X = np.ones(1000000000, dtype=np.int) Y = np.ones(1000000000, dtype=np.int) # Add 2 * Y to X, element by element: # Slowest %time X = X + 2.0 * Y 100 loops, best of 3: 3.61 ms per loop # A bit faster %time X = X + 2 * Y 100 loops, best of 3: 3.47 ms per loop # Much faster %time X += 2 * Y 100 loops, best of 3: 2.79 ms per loop # Fastest %time np.add(X, Y, out=X); np.add(X, Y, out=X) 100 loops, best of 3: 1.57 ms per loop That's a 2.3x speed improvement (from 3.61 ms to 1.57 ms) on a simple vector operation (your mileage will vary!).[1] This only scratches the surface. The guide goes into quite a bit of explicit detail about how Numpy arrays are constructed and stored in memory and always explains the underlying reasons why some operations are faster than others. In addition, the guide has a section titled "Beyond Numpy" that points to even more ways of improving performance, e.g., by using Cython, Numba, PyCUDA, and a range of other tools. I highly recommend reading the whole thing! -- [1] Example copied from here: https://www.labri.fr/perso/nrougier/from-python-to-numpy/#anatomy-of-an-array https://www.labri.fr/perso/nrougier/from-python-to-numpy/#an...
- stabbles 9y ago
  In Julia this would read using broadcasting: X = ones(Int, 1_000_000_000) Y = ones(Int, 1_000_000_000) # explicit dot notation X .+= 2 .* Y # or with a macro @. X += 2 * Y The + and * do not allocate, the operations are fused into element-wise operations so the vectors are traversed only once, and the loop is compiled to vectorized code. No need for Numpy / BLAS.
  - FractalNerve 9y ago
    That's impressive! What does fusion mean? I've heard fusion multiple times and I've the feel that fusion (in haskell at least) is something very advanced that's not really thought. Can you explain?
fnord123 9y ago
Python's only slow if you use it wrong: http://apenwarr.ca/diary/2011-10-pycodeconf-apenwarr.pdf http://apenwarr.ca/diary/2011-10-pycodeconf-apenwarr.pdf
- jernfrost 9y ago
  If python did not have a performance problem you wouldn't need to implement any python libraries in C. The point of having Julia is avoiding the usage of two languages. If you can have C performance in a script language why bother with python? The only reason to pick python is momentum, libraries and mindshare. The language by itself is nothing special. Julia is faster, is a better glue language, easier to use and more powerful.
  - joshuamorton 9y ago
    If Julia didn't have a performance problem, you wouldn't need to call out to a BLAS. >is a better glue language I'm confused, I thought you said you were avoiding the usage of two languages. If so, what is Julia "gluing"?
    - jernfrost 9y ago
      You don't have to as Julia can still get high performance. And BLAS is actually an example of Julia strenghts. The ability to pick exact code specializations help effectively pick the most suitable BLAS function. Python can't pick correct function based on type and number of arguments. As for gluing. That is often needed not for performance reasons but because you don't want to reimplement lots of proven code which is well devugged and tested. One of the reasons to pick BLAS. I use Julia a lot for shell like scripting. That means running a lot of unix commands I don't know how was implemented like the ios code signing tools. Doing this in Julia is far superior to doing it in python. I was actually shocked how clunky it was in python when I attempted a rewrite.
      - joshuamorton 9y ago
        >Python can't pick correct function based on type and number of arguments. What? Numpy does this. >Doing this in Julia is far superior to doing it in python. Then, quite simply, you're doing it wrong in python. Show me an example of this system code that's nice in julia and clunky in python and I'll show you a superior implementation in python.
jernfrost 9y ago
This is totally missing the point of the Julia benchmarks. They were meant to show that you can implement fast low level code in Julia. That is why the python code also had to be written with loops otherwise it would really just test the performance of a library and not the language. The point of Julia is to not have to deal with both C/C++/fortran and python code when creating a high performance solution. You don't always have a high performance library to utilize for your particular problem.
- Recurecur 9y ago
  Exactly. Being able to do fast things using typical procedural idioms, and being able to implement new, fast routines without dropping to a different, lower-level language are huge wins. Python is a nice language syntactically, it's just too bad that fundamental design decisions made it intrinsically slow.
temporaryred 9y ago
From looking at most of these examples it's clear that Python will be able to perform really well with projects like numba, Cython and PyPy in the picture. My impressions after using Cython are that Cython can for the most part even outperform Julia. Couple that with the fact that with Julia there will probably be a few steps before you squeeze the maximum performance possible of out your algorithm, and it makes Cython a no brainier for a developer that is already using Python. That is until you consider that most of these examples do not showcase (what is in my humble opinion) Julia's true strength - homoiconicity and an opt in extensible type system that is part of the language. It's trivial to make a new Class in Python and set up behavior that one would be interested in. One could easily implement decorators, metaclasses, descriptors, properties and other dunder methods to customize class behavior to the heart's desire. But Cython (as far as I understand) does not support these Python features. If an existing library implements an interface using these Python features, it's not obvious to me how it would be possible to use Cython to improve performance. Julia on the other hand has no such restrictions. I would say it is desired to use macros and metaprogramming in everyday programming. I've been a Python user for over 6 years now. I've only looked at Julia for a few months now and I already can see concepts and ideas in Julia that have no analogue in Python. But most if not all Python programming features translate to Julia one way or the other. I love Python, but I can't help but feel that the lack of a opt in type system and more powerful metaprogramming constructs is hurting it in these comparisons.
goatlover 9y ago
You can use Numpy inside Julia with the Pycall Library. Julia can also load R, C & Fortran code.

203 points by aaronchall 9y ago

I guess, it will take loosing market share to Julia for Python devs actually start taking into consideration making PyPy the canonical implementation. Rewriting libraries in C isn't making Python code run faster. The latest Julia conf had lots of cool presentations, and even though 1.0 might come only at the end of the year (with luck), the uptake among the research community is quite good for such a young language.

marmaduke 9y ago
There's little reason to move to PyPy when Numba is available, at least for scientific code. For backend work, I find the bottleneck is never Python, tbh
- srean 9y ago
  Either Numba has improved by an order of magnitude since I looked last (congratulations that's fantastic, its a difficult problem to solve), or you haven't met the edge cases yet
JulianWasTaken 9y ago
What would it mean to make it the "canonical implementation"? Is there something preventing you from using PyPy today?
- pjmlp 9y ago
  I mean the version installed by default that anyone can use without requiring root. I stop using Python long time ago.
  - orf 9y ago
    What would peoples reaction be when their existing small Python scripts take at least 3x as long to run?
    - pjmlp 9y ago
      If the community focused on making PyPy the canonical implementation, that would be surely a major effort to improve that scenario. Lisp, Scheme, Dylan, JavaScript, Julia don't suffer from such huge speed bump, on the contrary.
      - orf 9y ago
        Neither does C. But that, and all the other languages you listed, are not Python. They have differing implementations, constraints and resources that make the comparison unfair and IMO invalid.
        pjmlp 9y ago
        C is not in the same league as the languages I mentioned, in regards to productivity and memory safety, while enjoying quite good performance. What is unfair about being as dynamic as Python and yet enjoying top quality JIT compilers? Just the community not caring about it and forcing everyone else to go down into C or somewhere else, it seems.
        orf 9y ago
        > Just the community not caring about it and forcing everyone else to go down into C or somewhere else, it seems. That, or the inherent dynamic nature of Python that makes it ridiculously hard to JIT (especially without breaking compatibility), and that a tiny subset of developers have enough experience (and will) to work on it as there is no Google willing to funnel buckets of money into making a V8 for Python, to the point where PyPy has reached an OK state after 10 years of development and a codebase that dwarfs CPython in size and complexity. I hate people who deride the community for not caring enough about adding a JIT. Yeah, that sure is the reason. Not mind-boggling technical limitations, nope, just pure laziness. Why don't you try? Exactly.
- dagw 9y ago
  Is there something preventing you from using PyPy today? A lot of libraries aren't tested with/don't work with pypy.
  - orf 9y ago
    File issues with them. Use the github editor to add `pypy` to their `gitlab-ci.yml`, submit a merge request, and see if any tests fail.
coldtea 9y ago
>Rewriting libraries in C isn't making Python code run faster. No, but nobody cares for the purity of only-Python code. They care for running their calculation fast, and if that takes C-enabled modules they are totally fine with it. With things like numpy available, it's not like they have to write those calculations themselves. Besides Numpy smokes PyPy for the same kind of calculations (not that one would care to rewrite the tons of stuff available in there in pure Python).
- srean 9y ago
  >No, but nobody cares for the purity of only-Python code. Let me call BS on that. I would prefer not to be forced to drop down to C if I could have it. I am quite sure many feel that way. I would definitely like the ease of calling an existing C or Fortran library that already does what I need. But that I need (almost to a fault) to context switch between languages is certainly not a high point especially when I am prototyping something new and for which there aren't any good libraries yet.
  - bpicolo 9y ago
    Numpy doesn't force you to drop down to C, though. That it's C under the hood is almost entirely irrelevant when making use of it
    - srean 9y ago
      Yes it absolutely doesnt. I was talking about things when you have to reach beyond Numpy/Scipy/Pandas/Scikits. For C or Fortran libraries and tools like those that are already there, Python is very pleasant to use.
- pjmlp 9y ago
  If people didn't care, they wouldn't be moving into Julia, Go, Chapel,...
  - Majestic121 9y ago
    Do you have any source for this affirmation that people are moving from Python to Julia/Go/etc ? Those language are definitely growing, but so is Python and while there is movement between the languages I am not sure is a massive unilateral migration as you seem to imply.
ChrisSD 9y ago
Pypy is not that much faster than Python and is certainly still much slower than using a C library. Indeed, there's a limit to how fast python can be considering how many operations it needs to do for even simple statements (such as summing two integers[0]). This is the cost of being so extremely dynamic, more so than Javascript and many other scripting languages. [0] https://jakevdp.github.io/blog/2014/05/09/why-python-is-slow/#Python-Addition https://jakevdp.github.io/blog/2014/05/09/why-python-is-slow...
- pjmlp 9y ago
  Even after reading that post, I doubt Python can be more dynamic than Lisp or Smalltalk, and yet they are behind the genesis of JIT compilers.
  - jerf 9y ago
    I don't know that it "is", but it certainly can be. In Python there's just too many ways to screw with the meaning of "a.b" at runtime; it could be in the class dict, it could be a property, it could be modified via a couple of methods on the class, it could be screwed with a couple of other ways too IIRC. You have to write some sort of code that either proves these things can't happen, which gets really hard as the program scales up, or code that checks for them happening. Other languages that can be "dynamic" but still have only one or two things to be checked can be much more easily JIT'ed.
    - pjmlp 9y ago
      Smalltalk has a message called becomes, where you literally change the complete representation of an object to something else. You can change code on the fly on the debugger and just do a redo of the failed call. When you change a specific class, or its meta-class, the whole system is automatically updated, reflecting the changes. I can give many more examples.
      - pwang 9y ago
        I believe much similar magic is possible in Python, but you get to implement it on top of the relatively simple interactions between instance dict, class dict, and metaclass. They would not be bullet-proof, and they are not built into the default implementation. But they are very possible.

marmaduke 9y ago

It's easy to optimize small kernels. Less straightforward is optimizing an entire library in Python, especially if pieces need to be composed. For this, the most interesting library I've come across is loopy which handles things like fusing kernels, unrolling loops _before_ code generation. https://documen.tician.de/loopy/ https://documen.tician.de/loopy/

kristofferc 9y ago

Isn't adding things like memoization or writing the code in C (Cython) missing the whole point of the benchmark, which is to test the overhead of recursion / function calls in the language itself?

knlje 9y ago
Users of programming languages interested in the end result (output of the program etc) do not care. They want the fastest performing language for the job. Julia website has been misleading people. Due to those claims, I spent a week porting some of my simulation code to Julia before I realized that it is actually slower in (optimized) Julia than in optimized Python.
- kristofferc 9y ago
  For those who only care about the output of fib(20) there are more efficient methods than any of the Julia or Python implementations posted in the link, e.g. lookup tables. The assumption in benchmarks is of course that the results carry over to other use cases. Here, what is being tested is the overhead in recursion, nothing else. The fact that it happens to be Fibonacci-numbers that is being computed is irrelevant.
  - coldtea 9y ago
    Why, is anybody restricted to using recursion in their solutions?
    - kristofferc 9y ago
      You are not restricted to recursion, loops, functions etc, but these language constructs might be useful in situations, and therefore the performance (overhead) of these constructs is interesting.
  - Beltiras 9y ago
    Isn't that basically what he does when he adds caching? I know it's not a static lookup table but you could prep it by invoking it with a sufficiently large n.

stevesimmons 9y ago

Can the HN title add "(2015)"?

drej 9y ago

There are a few things I consider when I try to improve performance of my code. 1) I really like the advice of "make it run, make it right, make it fast" [0]. First I build it, then I write a thorough test harness (or make multiple versions run at the same time and check their results at runtime), then I rewrite it. 2) I don't really like optimising to the core, like it's mentioned in the article, things like Cython and numba often add dependencies that don't port well and they may reduce legibility and maintainability (that's more about Cython than numba). What I find most useful, and this should be obvious, that algorithm/data structure changes often yield the biggest benefits. 3) The old rule that if you speed up code that only runs 1% of the time, you're not gaining that much, I try to keep that in mind. (There are exceptions, of course.) 4) Performance is not just runtime, it starts with coding it up in the first place. So start to finish, you're not always better off with a fast language. If it's a one-off, it can be more performant to use a slow language, if it's a tool that will run untouched, you might want to spend some time tinkering with it. Etc. All of this seems obvious in retrospect, but it took me a while to appreciate these principles. [0] https://www.facebook.com/notes/kent-beck/mastering-programming/1184427814923414 https://www.facebook.com/notes/kent-beck/mastering-programmi...

eesmith 9y ago
A side historical note which doesn't affect anything; http://wiki.c2.com/?MakeItWorkMakeItRightMakeItFast http://wiki.c2.com/?MakeItWorkMakeItRightMakeItFast traces that expression back to Stephen C. Johnson and Brian W. Kernighan's "The C Language and Models for Systems Programming" in Byte magazine (August 1983) - "the strategy is definitely: first make it work, then make it right, and, finally, make it fast."
- moomin 9y ago
  The make it work/make it right principle is also referred to as "tracer bullets" in certain circles: if you get the bare bones full process working it's much easier to flesh it out than doing it one "fully specced" component at a time.

hojijoji 9y ago

Make "Python" run faster by shifting all the runtime heavy computation to C-implementations... this article is missing the whole point

coldtea 9y ago
Python is a scripting language, so its strong point is being used as a glue, and half of its standard library is implemented in C anyway. Plus, the "C-implementations" he mentions are available as readily usable modules (numpy), or semi-transparent jit/aot compilers only needing a few annotations (Cython, numba), not actual C you have to write. Besides, isn't the whole point: finish your project fast with the language you know using whatever it makes available to easily speed your code up? As opposed to: "be a purist and not use wrapped libs written in another language". Who cares for that? Even if it comes up, it's to avoid the hassle of having to deal with an additional language, setup etc -- which for numpy, Cython etc is almost none-existent (as you don't need to actually deal with C). And of course, despite the purity of Julia's "single language", the hassle of moving to a totally different language, which few use, is not yet stable in syntax, compiler etc, and has fewer libs, should also be considered...
- deleted 9y ago
  [deleted]
- FabHK 9y ago
  The specific purpose of the benchmark, though, is to compare implementations of the same algorithm natively in the language itself, as explained explicitly on the Julia website just under the table of benchmark results (see quote below). As such, I do think the article misses the point somewhat. Of course, if there's a numpy function that does what you want, you'd use it in real life. But what if there isn't? The nice thing about Julia is that the function can be written in Julia itself, and fast. > It is important to note that these benchmark implementations are not written for absolute maximal performance (the fastest code to compute fib(20) is the constant literal 6765). Rather, all of the benchmarks are written to test the performance of specific algorithms implemented in each language. In particular, all languages use the same algorithm: the Fibonacci benchmarks are all recursive while the pi summation benchmarks are all iterative; the “algorithm” for random matrix multiplication is to call the most obvious built-in/standard random-number and matmul routines (or to directly call BLAS if the language does not provide a high-level matmul), except where a matmul/BLAS call is not possible (such as in JavaScript). The point of these benchmarks is to compare the performance of specific algorithms across language implementations, not to compare the fastest means of computing a result, which in most high-level languages relies on calling C code. https://julialang.org https://julialang.org

unfamiliar 9y ago

I had a lot of experience a few years ago with Julia, having ported a lot of my numerical code in the language. I ended up severely disappointed with it for moderately complex numerical computing projects. The language was really nice; the problem was terrible performance, and I believe that this was caused by bad design of the memory semantics. Even in optimised code, there were temporary objects and copies all over the place, and they were very hard to eliminate without resorting to using global arrays everywhere.

vanderZwan 9y ago
> Even in optimised code, there were temporary objects and copies all over the place, and they were very hard to eliminate without resorting to using global arrays everywhere. Could you elaborate a bit with some examples, because this goes against everything I understand about the strengths of the language: (without having used it beyond a few first examples myself) - you can define types and operations on them down to the bits themselves, giving you a lot of control over the memory (and definitely a lot more than Python) - it has a lot of support for in-place mutation out of the box - globals are almost always bad news for performance Sure, adding two arrays with C = A + B .. produces an intermediate array, yes, but if you know A and B are not reused, wouldn't using: A += B C = A .. instead be enough?
- ffriend 9y ago
  Even better would be to write (see dot vectorization [1]): C .= A .+ B Benchmarks for 3 matrices of size 1000x1000: julia> using BenchmarkTools julia> @benchmark C = A + B BenchmarkTools.Trial: memory estimate: 7.63 MiB allocs estimate: 2 -------------- minimum time: 2.359 ms (0.00% GC) median time: 2.713 ms (0.00% GC) mean time: 3.794 ms (28.81% GC) maximum time: 62.708 ms (95.27% GC) -------------- samples: 1314 evals/sample: 1 julia> @benchmark C .= A .+ B BenchmarkTools.Trial: memory estimate: 128 bytes allocs estimate: 4 -------------- minimum time: 1.232 ms (0.00% GC) median time: 1.320 ms (0.00% GC) mean time: 1.356 ms (0.00% GC) maximum time: 2.572 ms (0.00% GC) -------------- samples: 3651 evals/sample: 1 Note that memory usage dropped from 7.63MiB to 128 bytes. [1]: https://docs.julialang.org/en/stable/manual/functions/#man-vectorized-1 https://docs.julialang.org/en/stable/manual/functions/#man-v...

cs702 9y ago

If you really care about extracting every possible ounce of performance out of Python's scientific stack, which relies extensively on Numpy, the best practical guide I have found for doing that is "From Python to Numpy," by Nicolas P. Rougier of Inria: https://www.labri.fr/perso/nrougier/from-python-to-numpy/ https://www.labri.fr/perso/nrougier/from-python-to-numpy/ Here's a typical example of the kinds of optimizations this guide teaches you, in this case by avoiding the creation of temporary copies of Numpy arrays in memory: # Create two int arrays, each filled with with one billion 1's. X = np.ones(1000000000, dtype=np.int) Y = np.ones(1000000000, dtype=np.int) # Add 2 * Y to X, element by element: # Slowest %time X = X + 2.0 * Y 100 loops, best of 3: 3.61 ms per loop # A bit faster %time X = X + 2 * Y 100 loops, best of 3: 3.47 ms per loop # Much faster %time X += 2 * Y 100 loops, best of 3: 2.79 ms per loop # Fastest %time np.add(X, Y, out=X); np.add(X, Y, out=X) 100 loops, best of 3: 1.57 ms per loop That's a 2.3x speed improvement (from 3.61 ms to 1.57 ms) on a simple vector operation (your mileage will vary!).[1] This only scratches the surface. The guide goes into quite a bit of explicit detail about how Numpy arrays are constructed and stored in memory and always explains the underlying reasons why some operations are faster than others. In addition, the guide has a section titled "Beyond Numpy" that points to even more ways of improving performance, e.g., by using Cython, Numba, PyCUDA, and a range of other tools. I highly recommend reading the whole thing! -- [1] Example copied from here: https://www.labri.fr/perso/nrougier/from-python-to-numpy/#anatomy-of-an-array https://www.labri.fr/perso/nrougier/from-python-to-numpy/#an...

stabbles 9y ago
In Julia this would read using broadcasting: X = ones(Int, 1_000_000_000) Y = ones(Int, 1_000_000_000) # explicit dot notation X .+= 2 .* Y # or with a macro @. X += 2 * Y The + and * do not allocate, the operations are fused into element-wise operations so the vectors are traversed only once, and the loop is compiled to vectorized code. No need for Numpy / BLAS.
- FractalNerve 9y ago
  That's impressive! What does fusion mean? I've heard fusion multiple times and I've the feel that fusion (in haskell at least) is something very advanced that's not really thought. Can you explain?

fnord123 9y ago

Python's only slow if you use it wrong: http://apenwarr.ca/diary/2011-10-pycodeconf-apenwarr.pdf http://apenwarr.ca/diary/2011-10-pycodeconf-apenwarr.pdf

jernfrost 9y ago
If python did not have a performance problem you wouldn't need to implement any python libraries in C. The point of having Julia is avoiding the usage of two languages. If you can have C performance in a script language why bother with python? The only reason to pick python is momentum, libraries and mindshare. The language by itself is nothing special. Julia is faster, is a better glue language, easier to use and more powerful.
- joshuamorton 9y ago
  If Julia didn't have a performance problem, you wouldn't need to call out to a BLAS. >is a better glue language I'm confused, I thought you said you were avoiding the usage of two languages. If so, what is Julia "gluing"?
  - jernfrost 9y ago
    You don't have to as Julia can still get high performance. And BLAS is actually an example of Julia strenghts. The ability to pick exact code specializations help effectively pick the most suitable BLAS function. Python can't pick correct function based on type and number of arguments. As for gluing. That is often needed not for performance reasons but because you don't want to reimplement lots of proven code which is well devugged and tested. One of the reasons to pick BLAS. I use Julia a lot for shell like scripting. That means running a lot of unix commands I don't know how was implemented like the ios code signing tools. Doing this in Julia is far superior to doing it in python. I was actually shocked how clunky it was in python when I attempted a rewrite.
    - joshuamorton 9y ago
      >Python can't pick correct function based on type and number of arguments. What? Numpy does this. >Doing this in Julia is far superior to doing it in python. Then, quite simply, you're doing it wrong in python. Show me an example of this system code that's nice in julia and clunky in python and I'll show you a superior implementation in python.

jernfrost 9y ago

This is totally missing the point of the Julia benchmarks. They were meant to show that you can implement fast low level code in Julia. That is why the python code also had to be written with loops otherwise it would really just test the performance of a library and not the language. The point of Julia is to not have to deal with both C/C++/fortran and python code when creating a high performance solution. You don't always have a high performance library to utilize for your particular problem.

Recurecur 9y ago
Exactly. Being able to do fast things using typical procedural idioms, and being able to implement new, fast routines without dropping to a different, lower-level language are huge wins. Python is a nice language syntactically, it's just too bad that fundamental design decisions made it intrinsically slow.

temporaryred 9y ago

From looking at most of these examples it's clear that Python will be able to perform really well with projects like numba, Cython and PyPy in the picture. My impressions after using Cython are that Cython can for the most part even outperform Julia. Couple that with the fact that with Julia there will probably be a few steps before you squeeze the maximum performance possible of out your algorithm, and it makes Cython a no brainier for a developer that is already using Python. That is until you consider that most of these examples do not showcase (what is in my humble opinion) Julia's true strength - homoiconicity and an opt in extensible type system that is part of the language. It's trivial to make a new Class in Python and set up behavior that one would be interested in. One could easily implement decorators, metaclasses, descriptors, properties and other dunder methods to customize class behavior to the heart's desire. But Cython (as far as I understand) does not support these Python features. If an existing library implements an interface using these Python features, it's not obvious to me how it would be possible to use Cython to improve performance. Julia on the other hand has no such restrictions. I would say it is desired to use macros and metaprogramming in everyday programming. I've been a Python user for over 6 years now. I've only looked at Julia for a few months now and I already can see concepts and ideas in Julia that have no analogue in Python. But most if not all Python programming features translate to Julia one way or the other. I love Python, but I can't help but feel that the lack of a opt in type system and more powerful metaprogramming constructs is hurting it in these comparisons.

goatlover 9y ago

You can use Numpy inside Julia with the Pycall Library. Julia can also load R, C & Fortran code.