At the time this post was written, there was no good autodiff implementation for any language that anyone actually used for ML, meaning Python, Matlab, or Java.

At the time this post was written, there was no good autodiff implementation for any language that anyone actually used for ML, meaning Python, Matlab, or Java. That's changed now; for example, Python autograd [1] is quite seamless and plays well with the entire Numpy stack. The bigger trend, though, has been towards DSLs such as Theano or Tensorflow (and Torch to a lesser degree) that include autodiff as a first class design consideration. The ability to build complicated networks out of modular parts, and have the gradient calculations automatically fall out, has been a major driver of the recent explosive progress in deep learning research, as has easy portability between CPU and GPU execution. It'll be exciting to see what happens as these tools mature and especially if they find applications in other fields. [1] https://github.com/HIPS/autograd https://github.com/HIPS/autograd