3 ms·
Cannot upvote this enough. I think this is the only way to really grok backpropagation. The hours of staring at the update formula till your eyes glaze over th
by srean 9y ago
Cannot upvote this enough.
I think this is the only way to really grok backpropagation. The hours of staring at the update formula till your eyes glaze over the subscripts and superscripts and the summations would not give you as good an understanding as implementing a toy neural net with just a single hidden layer. Its actually a whole lot easier than parsing those low-level notation. It can be done better with high level notation but then you would need familiarity with the relevant mathematical abstractions.
- jclos 9y agoYes, once you've dealt with implementing it, the notation just makes sense as the most compact way of formalizing what you just did.
- annnnd 9y agoI respectfully disagree. Of course you need to really understand backpropagation for any advanced stuff, but it is easier to ignore it at the beginning. Take Keras container, copy some MNIST example from somewhere and tweak it. Then, when you have a general feel of how training NNs works, gradually learn about each concept - BP should of course be one of the first. By the time you get into math stuff it will probably make much more sense because you will understand how it applies to your case. But I guess the approach depends on how you best learn, so there is no wrong answer. Just jump in!