3 ms·
I strongly advise everybody with one day free (and not much better things to do) to implement a basic fully connected feedforward neural network (the classical
by antirez 10y ago
I strongly advise everybody with one day free (and not much better things to do) to implement a basic fully connected feedforward neural network (the classical stuff, basically), and give it a try against the MNIST handwritten digits database. It's a relatively simple project that learns you the basic. On top of that to understand the more complex stuff becomes more approachable. To me this is the parallel task to implement a basic interpreter in order to understand how higher level languages and compilers work. You don't need to write compilers normally, as you don't need to write your own AI stack, but it's the only path to fully understand the basics.
You'll see it learning to recognize the digits, you can print the digits that it misses and you'll see they are actually harder even for humans sometimes, or sometimes you'll see why it can't understand the digit while it's trivial for you (for instance it's an 8 but just the lower circle is so small).
Also back propagation is an algorithm which is simple to develop an intuition about. Even if you forget the details N years later the idea is one of the stuff you'll never forget.
- hood_syntax 10y agoI fully agree, in the AI class I took, the first two projects we implemented feedforward neural networks with backpropagation. The math may take some time to understand, but the concept is very natural. Definitely my favorite part of that class.
- sandGorgon 10y agoQuick question - what, modern day tutorial/mooc would you recommend for this ?
- antirez 10y agoNot sure, I think it's better to use older documentation which is not so centered on RNNs and Convolutional networks. The old docs very well describe a basic net with Backpropagation.
- ahmedbaracat 10y agoI would highly recommend Andrej Karpathy's Hacker's guide to Neural Networks. A step by step guide with little math to cover the basics of NN. http://karpathy.github.io/neuralnets/ http://karpathy.github.io/neuralnets/
- antirez 10y agoWow, that surely is thought for programmers.
- Uehreka 10y agoneuralnetworksanddeeplearning.com is a free online book by Michael Nielsen. I'm reading it right now, and it's wonderful. In the first chapter, he walks you through implementing a basic neural network in Python that works with the MNIST data and uses no libraries (except numpy). He's really good at explaining a lot of the math for a more programmer-ish audience that might not have studied multivariate calculus.
- snippyhollow 10y agoI wrote this quite some time ago exactly to hold the hands of neophytes who want to do such endeavour. http://nbviewer.jupyter.org/github/syhw/DL4H/blob/master/from_logistic_regression_to_deep_nets.ipynb http://nbviewer.jupyter.org/github/syhw/DL4H/blob/master/fro...
- Omnipresent 10y agoIs there a part 2 to this?
- snippyhollow 10y agoPart 2 was never finished but supposed to be text around this http://nbviewer.jupyter.org/github/syhw/DL4H/blob/master/DNN_notebook.ipynb http://nbviewer.jupyter.org/github/syhw/DL4H/blob/master/DNN...
- namelezz 10y agoDid that once and realized the hard part was how you tested the correctness of your NN implementation.
- toxik 10y agoNumeric differentiation, huzzah!