4 ms·
This is a very, very convoluted way of understanding backprop. If you want to understand it I highly recommend writing a neural network starting from a basic p
by teruakohatu 6y ago
This is a very, very convoluted way of understanding backprop.
If you want to understand it I highly recommend writing a neural network starting from a basic perceptron (no hidden layer) or even implementing basic linear regression with stochastic gradient descent. Then work your way up to a network with one hidden layer.
Try to focus on the maths and avoid looking at code examples.
- selimthegrim 6y agoIs scholastic gradient descent anything like grad student descent? ;)
- reitzensteinm 6y agoThis is a great book that breaks down the process step by step. I had my own neural nets working before I touched a framework, and it demystified what would have otherwise seemed like black magic. https://www.amazon.com.au/Make-Your-Own-Neural-Network-ebook/dp/B01EER4Z4G https://www.amazon.com.au/Make-Your-Own-Neural-Network-ebook...
- 29athrowaway 6y agoThe book I used to learn the basics many years ago was "An Introduction to Neural Networks", by Kevin Gurney. It's a bit outdated now, but I still consider it valuable.
- cultyyy123 6y agoThis is a very, very convoluted way of gate keeping. > Try to focus on the maths and avoid looking at code examples. Straight out dictatorship. > If you want to understand it I highly recommend writing a neural network starting from a basic perceptron (no hidden layer) or even implementing basic linear regression with stochastic gradient descent. Then work your way up to a network with one hidden layer. Like it's only way to understand backpropagation. This quote is still valid to date. “Today's scientists have substituted mathematics for experiments, and they wander off through equation after equation, and eventually build a structure which has no relation to reality. ” - Nikola Tesla Trust me, you can't fool people for too long. Abstractions always wins the game.
- Vhano 6y agoI agree. Why talk in precise terms when you can do abstract mythology instead.
- datameta 6y agoI absolutely second the suggestion of working through the math of a shallow neural network to start. I would also suggest that once one thinks they have a sufficient grasp of the steps to try to implement them at a low level without frameworks, just math libraries. Personally, I found Octave (highly similar to MATLAB) to be a nice granularity tier for learning the process programmatically.