I also starting working on a neural network implementation for my first foray into Julia: https://github.com/bachase/nnadl-julia https://github.com/bachase/nna

I also starting working on a neural network implementation for my first foray into Julia: https://github.com/bachase/nnadl-julia https://github.com/bachase/nnadl-julia. The notes file on the optimization branch is a very rough outline of things I discovered while iterating on performance. Outside of Iain's great advice and without profiling your code, for a larger sized network, you might benefit from reordering loops for column major storage, or doing normal matrix multiplication (which uses BLAS) followed by a devectorized tanh. As for the language, I love being able to get something working quickly while writing in a Matlab/"mathy" style. The built in profiler and timer tools are then excellent for zeroing in on hotspots and memory allocations. I'm just amazed at how simply I can drill down from scripting style code to LLVM IR to native asm all in an ijulia notebook browser window. On the downside, I'm not a fan of having to manually devectorize to avoid temporaries, especially given the success of c++ libraries like Blitz that figure it out for you. I'm sure Julia will improve in this area as it matures.