6 ms·
A Neural Network in 11 Lines of Python
- avyfain 11y agoThanks for this. It is really well explained. However, it did feel like it went downhill when you explained how the updating happens. At least it was less of a walkthrough than the previous sections. Also, lines 22 and 23 in the three layer network are not intuitive for a beginner. Before, syn0 was (3,1) and now its (3,4), and the new syn1 is (4,1). Your previous explanation for this initialization was much better. But again, awesome work, just being nitpicky. Really appreciate it!
- williamtrask 11y agoHey avyfain, I'll look out for that in my next post. Thanks for the note.
- matt1 11y agoI recently spent some time learning how backpropagation works and wound up putting together this step by step tutorial that you might find helpful: http://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/ http://mattmazur.com/2015/03/17/a-step-by-step-backpropagati...
- danso 11y agoAt the risk of being pedantic...if you're not familiar with data/math work in Python, the `np` word refers to "numpy", which is an extension to Python that includes array and matrix math...so the OP needs 12 lines, the first being: import numpy as np
- scott_karana 11y agoThank you! Non-obvious for sure.
- Bostonian 11y agoI knew that np stood for numpy but had to Google to realize that the .T suffix meant transpose.
- afarrell 11y agoThat isn't pedantic at all. Python tutorials should always include imports.
- ksenzee 11y agoNo, definitely not pedantic. In fact, I have only just now realized that "numpy" means "NumPy," instead of being a nonsense word that rhymes with "lumpy." Sigh.
- blazespin 11y ago... Isn't grammatically correct. Pedantic, I know.
- nine_k 11y agoFamous antigravity in one line in Python: https://xkcd.com/353/ https://xkcd.com/353/
- nikmobi 11y agovery nice, thank you!
- junto 11y agoAnyone fancy translating this to C#?
- cryowaffle 11y agoPretty trivial if you have a math library, I don't see anything here that is special. Why not give it a shot yourself?
- candu 11y agoTo help you get started, some relevant documentation: http://docs.scipy.org/doc/numpy/reference/ufuncs.html http://docs.scipy.org/doc/numpy/reference/ufuncs.html http://docs.scipy.org/doc/numpy/reference/routines.random.html http://docs.scipy.org/doc/numpy/reference/routines.random.ht... http://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.T.html#numpy.ndarray.T http://docs.scipy.org/doc/numpy/reference/generated/numpy.nd... ;)
- fizixer 11y agoAlso recommended, 7 part video series (link is video 1): https://www.youtube.com/watch?v=bxe2T-V8XRs https://www.youtube.com/watch?v=bxe2T-V8XRs
- jbssm 11y agoCan you suggest any good video explanation about NN but with some code with Python but using an actual NN library (like theano)?
- fizixer 11y agoHonestly I'm a beginner myself. Other then google search that you can do yourself, here's what might be helpful: https://triangleinequality.wordpress.com/2014/08/12/theano-autoencoders-and-mnist/ https://triangleinequality.wordpress.com/2014/08/12/theano-a...
- BrainInAJar 11y agoThis is a really fantastic article, I've been looking for a code-based "Neural network implementation for idiots" thing like this.
- ratsimihah 11y agoYES! Thank you!
- elyase 11y agoJust for comparison this is almost the same using a library [1]: import numpy as np X = np.array([ [0,0,1],[0,1,1],[1,0,1],[1,1,1]]) y = np.array([[0,0,1,1]]).T from keras.models import Sequential from keras.layers.core import Dense model = Sequential([Dense(3, 1, init='uniform', activation='sigmoid')]) model.compile(loss='mean_absolute_error', optimizer='sgd') model.fit(X, y, nb_epoch=10000) model.predict(X) [1] http://keras.io http://keras.io
- avyfain 11y agoBut this is not nearly as clear for a beginner, or someone who doesn't know what Sequential, compile, fit and predict are doing.
- rolandnsharp 11y agoCould someone please translate this into JavaScript for me. Or link me to Neural Network resources for JavaScript. Thanks.
- ratsimihah 11y agoThe final "Line 36" in your explanation should be "Line 39."
- mjcohen 11y agoInteresting fact about the sigmoid function: If you scale it so its slope at the origin is the same as the cumulative error function, it differs from that function by at most 0.017.
- blazespin 11y agoThis is amazing stuff. Just want more!
- compostor42 11y agoCan HN recommend any good neural net tutorials? (Preferably in Python)
- vonnik 11y agoThis is an excellent course no matter which language you prefer: https://class.coursera.org/neuralnets-2012-001/lecture https://class.coursera.org/neuralnets-2012-001/lecture
- monstruoso 11y agoWhy do you say it is an excellent course? I started it and so far I found the professor to be uninspiring and the lectures lacking in preparation and detail. I much rather read a book.
- bakul 11y agoSome of you may like seeing the two level neural network code translated to kona (or k3)! nonlin:{:[y;x*1-x;1%1+_exp -x]} x:(0 0 1;0 1 1;1 0 1;1 1 1) y:+,0 0 1 1 syn0:-0.5+3 1 _draw 0 step:{L1:nonlin[x _mul syn0;0] L1err:y-L1 L1delta:L1err*nonlin[L1;1] syn0+:(+x)_mul L1delta;L1} do[10000;L1:step[x;y]] L1 (,0.009668795 ,0.007862982 ,0.9935916 ,0.9921171) k3 does this in 140ms, Kona in 2030ms on the same machine.
- robmcm 11y agoI don't get the point of these "x in n lines" posts. Fewer lines of code doesn't always equate to simple, comprehensible code.
- choudanu4 11y agoWell, I personally presumed that neural networks would require two or three orders of magnitude more of code. When its revealed that the code is actually much more manageable, individuals (such as myself) take the dive into the subject, knowing that it is something we can at least get through.
- bodokaiser 11y agoI done something similar some months ago. Maybe this helps someone though it should have same content: http://nbviewer.ipython.org/github/bodokaiser/data-science/blob/master/algorithms/perceptron.ipynb http://nbviewer.ipython.org/github/bodokaiser/data-science/b...
- toxik 11y agoI wonder if it'd work better with a leaky ReLU. Somebody wanna try? It's a _very_ simple mechanism. [1] Basically, instead of a sigmoid, let f(z·w) be the activation due to inputs z and weight w, then a(z) = z·w if z·w > 0 else 0.01 * z·w. Sigmoid for activation have several problems, most notably they suffer from vanishing gradient problems with bigger networks. [1] https://en.wikipedia.org/wiki/Rectifier_(neural_networks)#Leaky_ReLUs https://en.wikipedia.org/wiki/Rectifier_(neural_networks)#Le...
- shubhamjain 11y agoGreat work! I have been reading a practical guide to neural networks [1], and I am amazed that it is so comprehensible. I had assumed ML to be a lot more complicated. Although, I won't classify it as an easy craft but (maybe) it isn't so hard as it looks. [1]: http://neuralnetworksanddeeplearning.com/chap1.html http://neuralnetworksanddeeplearning.com/chap1.html
- arpgy 11y agoShouldn't the derivative of sigmod(x) be sigmoid(x)(1-sigmoid(x)) and not x(1-x). The code seems to implement the second.