4 ms·

I really want to get into Julia. As I have been using an increasing amount of numerical code NumPy, SciPy and even my own Cython code has started to fail me per

by ninjin 13y ago

I really want to get into Julia. As I have been using an increasing amount of numerical code NumPy, SciPy and even my own Cython code has started to fail me performance-wise. The Julia community is really friendly and I tried rolled out a few implementations of the inner-most loop of a Recursive Neural Net in a variety of languages. https://github.com/ninjin/ppod/tree/master/hck/rnn_perf https://github.com/ninjin/ppod/tree/master/hck/rnn_perf Disappointingly enough I was unable to push the performance of Julia beyond that of pure (and ugly) Matlab. I tried asking over at #julia, but it seems that my implementation should be reasonably canonical. This is really a pity for me since I would love to remain highly productive in terms of code and use something scripting-esque rather than have to re-consider going back to C. I have also had a look at Nimrod, but the scientific library support appears somewhat lacking at the moment.

idunning 13y ago
I used Julia's in-built profiler to look at your code. Almost all the work is done, as you'd hope, on lines 20, 22, 33, and 35. I added @inbounds to lines 20 and 35, figuring that line 22 is mostly just the effort of calculating tanh. This dropped the running time on my computer from 10.7 seconds to 6.4 seconds, so about 60% of original running time. If I compile with gcc (gcc rnn_perf.c -O2 -o rnn_perf -lm) and run that, I get 6.2 (O3 is worse). That might not be how you compiled the C version, but regardless it looks like your C time was about 46% of the original Julia time, so its getting close. Three-fifths of the work now is in the tanh line. I'll submit a PR.
- StefanKarpinski 13y ago
  Having people like Iain – with this kind of knowledge of numerical programming and optimization – help you out with your code is one of the major benefits of Julia programming :-)
  - Bootvis 13y ago
    This is some quality advertising on your and Iain's part. It makes me want to use Julia even more when I get the chance.
- ninjin 13y ago
  Thanks a ton Iain, it pushed it down by 60% on my end as well. About the C code, I used -O2, the specific compiler flags are in the rnn_perf.sh file. https://github.com/ninjin/ppod/commit/ce7665a2cfd045332e98616036476a0704a68c7d https://github.com/ninjin/ppod/commit/ce7665a2cfd045332e9861... Julia is really tempting for my next project, the benefits of scripting without all the OOP cruft and a community that just shines. Also, did I mention the awesome meta programming?
- JPKab 13y ago
  Wow. If this is what the community in Julia is like, count me in. I've been focusing on mastering the scientific libraries of Python and waiting until I hit performance walls before I took Julia on, but perhaps I'll start now.
andreasvc 13y ago
> my own Cython code has started to fail me performance-wise. I think this is an odd statement, seeing as how Cython allows you to write pure C code without making any use of the Python C API, so by definition Cython can be as efficient as C minus a single function call from Python to C. I think your benchmark is way too small to be interesting. You're only testing raw numerical performance, and of course C wins there. Any of these languages allow you to wrap a C library with more or less convenience. I think the more interesting questions are how well these languages allow you to build larger applications with performance-sensitive components, how extensive the ecosystem of libraries is, and the quality of debugging and profiling facilities.
bachase 13y ago
I also starting working on a neural network implementation for my first foray into Julia: https://github.com/bachase/nnadl-julia https://github.com/bachase/nnadl-julia. The notes file on the optimization branch is a very rough outline of things I discovered while iterating on performance. Outside of Iain's great advice and without profiling your code, for a larger sized network, you might benefit from reordering loops for column major storage, or doing normal matrix multiplication (which uses BLAS) followed by a devectorized tanh. As for the language, I love being able to get something working quickly while writing in a Matlab/"mathy" style. The built in profiler and timer tools are then excellent for zeroing in on hotspots and memory allocations. I'm just amazed at how simply I can drill down from scripting style code to LLVM IR to native asm all in an ijulia notebook browser window. On the downside, I'm not a fan of having to manually devectorize to avoid temporaries, especially given the success of c++ libraries like Blitz that figure it out for you. I'm sure Julia will improve in this area as it matures.
Serow225 13y ago
Just curious, is it possible to vectorize your MATLAB implementation? In your test code you don't use the loop index in the calcs, so that makes it hard for me to tell if it is or is not possible to vectorize it for your actual application.
mandor 13y ago
I was curious to know how modern C++ would compare. Here is a version (not 100% sure about the result because I was not 100% sure of the meaning of some of the numpy constructs when applied to matrixes). http://chronos.isir.upmc.fr/~mouret/tmp/nn_eigen3.cc http://chronos.isir.upmc.fr/~mouret/tmp/nn_eigen3.cc It seems pretty fast!

by ninjin 13y ago

idunning 13y ago

I used Julia's in-built profiler to look at your code. Almost all the work is done, as you'd hope, on lines 20, 22, 33, and 35. I added @inbounds to lines 20 and 35, figuring that line 22 is mostly just the effort of calculating tanh. This dropped the running time on my computer from 10.7 seconds to 6.4 seconds, so about 60% of original running time. If I compile with gcc (gcc rnn_perf.c -O2 -o rnn_perf -lm) and run that, I get 6.2 (O3 is worse). That might not be how you compiled the C version, but regardless it looks like your C time was about 46% of the original Julia time, so its getting close. Three-fifths of the work now is in the tanh line. I'll submit a PR.

StefanKarpinski 13y ago
Having people like Iain – with this kind of knowledge of numerical programming and optimization – help you out with your code is one of the major benefits of Julia programming :-)
- Bootvis 13y ago
  This is some quality advertising on your and Iain's part. It makes me want to use Julia even more when I get the chance.
ninjin 13y ago
Thanks a ton Iain, it pushed it down by 60% on my end as well. About the C code, I used -O2, the specific compiler flags are in the rnn_perf.sh file. https://github.com/ninjin/ppod/commit/ce7665a2cfd045332e98616036476a0704a68c7d https://github.com/ninjin/ppod/commit/ce7665a2cfd045332e9861... Julia is really tempting for my next project, the benefits of scripting without all the OOP cruft and a community that just shines. Also, did I mention the awesome meta programming?
JPKab 13y ago
Wow. If this is what the community in Julia is like, count me in. I've been focusing on mastering the scientific libraries of Python and waiting until I hit performance walls before I took Julia on, but perhaps I'll start now.

andreasvc 13y ago

> my own Cython code has started to fail me performance-wise. I think this is an odd statement, seeing as how Cython allows you to write pure C code without making any use of the Python C API, so by definition Cython can be as efficient as C minus a single function call from Python to C. I think your benchmark is way too small to be interesting. You're only testing raw numerical performance, and of course C wins there. Any of these languages allow you to wrap a C library with more or less convenience. I think the more interesting questions are how well these languages allow you to build larger applications with performance-sensitive components, how extensive the ecosystem of libraries is, and the quality of debugging and profiling facilities.

bachase 13y ago

I also starting working on a neural network implementation for my first foray into Julia: https://github.com/bachase/nnadl-julia https://github.com/bachase/nnadl-julia. The notes file on the optimization branch is a very rough outline of things I discovered while iterating on performance. Outside of Iain's great advice and without profiling your code, for a larger sized network, you might benefit from reordering loops for column major storage, or doing normal matrix multiplication (which uses BLAS) followed by a devectorized tanh. As for the language, I love being able to get something working quickly while writing in a Matlab/"mathy" style. The built in profiler and timer tools are then excellent for zeroing in on hotspots and memory allocations. I'm just amazed at how simply I can drill down from scripting style code to LLVM IR to native asm all in an ijulia notebook browser window. On the downside, I'm not a fan of having to manually devectorize to avoid temporaries, especially given the success of c++ libraries like Blitz that figure it out for you. I'm sure Julia will improve in this area as it matures.

Serow225 13y ago

Just curious, is it possible to vectorize your MATLAB implementation? In your test code you don't use the loop index in the calcs, so that makes it hard for me to tell if it is or is not possible to vectorize it for your actual application.

mandor 13y ago

I was curious to know how modern C++ would compare. Here is a version (not 100% sure about the result because I was not 100% sure of the meaning of some of the numpy constructs when applied to matrixes). http://chronos.isir.upmc.fr/~mouret/tmp/nn_eigen3.cc http://chronos.isir.upmc.fr/~mouret/tmp/nn_eigen3.cc It seems pretty fast!