This is great! I'm exactly the person that post was intended for, with the limited ML background. I'm following both Andrew Ng's and Richard Socher's lectures.

This is great! I'm exactly the person that post was intended for, with the limited ML background. I'm following both Andrew Ng's and Richard Socher's lectures. I got stuck around chain rule. Speaking of Socher, he mentions in his dissertation about alternatives to using the Sigmoid: "Various other recent nonlinearities exist such as the hard tanh (which is faster to compute) or rectified linear: f(x) = max(0,x) (which does not suffer from the vanishing gradient problem as badly)." I wonder if one of these alternatives would improve on the XOR example? Am hoping to implement some ANNs on a mobile GPU, using iOS + Metal. But 10,000 epochs for the XOR example sounds a bit scary - as in turning an iPhone into a hand warmer.