10 ms·
Neural Networks from Scratch
- pplanel 5y agoCan't start in Android's Firefox.
- informationslob 5y agoI can.
- moffkalast 5y agoProbably can't start in Netscape Navigator either, the audacity.
- spoonsearch 5y agoVery nice, the color combination and the UI is so pleasing. The explanation is cool :)
- mdp2021 5y agoOn mine, the textboxes are broken - overlapping other areas and rendered with a heavy blur.
- aeg42x 5y agoAre you using mobile or a browser? And could you please post a screenshot? I'll see what I can do! Thank you!
- bnegreve 5y agoI don't think it is a good idea to describe neural networks as a large graph of neurons interacting with each other. It is not really helpful to understand what is going on inside. It is more useful to understand them as a series of transforms that bend and fold the input space, in order to place pairs of similar items close to each other. I would like to see people trying to illustrate that instead. It also has the benefit of making the connection with linear algebra much easier to understand.
- farresito 5y agoTotally agree with you. The article that opened my eyes was this[0] one. This[1] video is also very good. [0] https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/ https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/ [1] https://www.youtube.com/watch?v=e5xKayCBOeU https://www.youtube.com/watch?v=e5xKayCBOeU
- joefourier 5y agoYou really find large n-dimensional transforms easier to reason about and visualise, as opposed to layers of neurons with connections? You don’t find it much more intuitive to see it as a graph once you start adding recurrence, convolutions, sparsity, dropout, connections across multiple layers, etc., let alone coming up with new concepts? I think it’s useful to understand it in both ways, but our intuitions about transforms are largely useless when the number of dimensions is high enough.
- nerdponx 5y agoIt's good to have both perspectives. Ideally you learn the layers-of-transforms version alongside the styled graph-of-neurons version. If you had to only pick one, which one you learn would depend a lot on what kind of student you are and what your goals are. I think the layers-of-transforms version is "less wrong" in general, but probably harder to understand, so it's maybe better if you had to learn just one.
- farresito 5y agoNot the person you are answering to, but I think it's all about the level of abstraction you want to reason at. I didn't grok neural networks until I visualized the transformations that were happening in a very simple network. Once that made sense, I could start thinking in terms of layers.
- ravi-delia 5y agoI think understanding how neural networks work is easiest if you think of them as networks. Reasoning about why they work is a lot easier thinking about them as transformations. It's not like you're actually picturing all the parameters of a nontrivial network one way or the other.
- synergy20 5y agovery cool, nice UI, simplest tutorial but grasps the gist, perfect for starters to have a big picture before diving in the details.
- minihat 5y agoEach time I teach neural nets to an engineer, there's only a 50% chance they can write down the chain rule. Colah's blog on backprop used to be my favorite resource to leave them with (https://colah.github.io/posts/2015-08-Backprop https://colah.github.io/posts/2015-08-Backprop). The explanation of the calculus in this tool is equally fantastic. And the art is very cute. There are many ways to skin a cat, of course, but this is as good a tutorial as I've seen for getting you through backprop as fast as possible.
- jhgb 5y ago> there's only a 50% chance they can write down the chain rule I blame the common mathematical notation for that.
- ThinkingAgain 5y agoAfter reading the tutorial I was not sure why it was called backpropagation. Thanks for the Colah's blog link. I think the two links together explains the things beautifully. Backpropagation seems just like an optimization for the gradient descent calculation as per Colah's blog.
- shaan7 5y agoAny recommendations for a 101 book for neural nets for someone who is "just a programmer"? OP's tutorial is quite nice, but I love to read books and find it easier to learn from them.
- jacobcmarshall 5y agoDeep Learning with Python by Chollet is an excellent beginner resource if you are a hands-on learner. It starts off with some tutorials using the Keras library, and then gets into the math later on. By the end of the book, you create multiple different types of neural networks for identifying images, text, and more! I highly recommend it.
- shaan7 5y agoThanks for the recommendations folks! :)
- shimonabi 5y agoIf anyone is interested, here is a simple symbol recognizer using backpropagation I wrote in Python a while ago with the help of the book "Make your own network" by Tariq Rashid. Numpy is a great help with matrix calculations. https://www.youtube.com/watch?v=IAQyVmTDz0A https://www.youtube.com/watch?v=IAQyVmTDz0A
- sarathyweb 5y agoThe text is too small to read on my phone. I cannot zoom in either :(
- mLuby 5y agoWhile most of the random starting weights converged quickly, this one got stuck with a fairly incorrect worldview, so to speak:  Is it overfitting to say the same is true for humans, where a brain's starting weights and early experiences may make it much more difficult to achieve an accurate model?
- robomartin 5y agoFor those curious about the nabla (∇) or gradient symbol (not a Greek letter): https://en.wikipedia.org/wiki/Nabla_symbol https://en.wikipedia.org/wiki/Nabla_symbol
- andreyk 5y agoPleasantly surprised by this, not yet another blog post on this but rather a nice interactive lesson. Well done!
- aeg42x 5y agoHi everyone! I made this thing! I'm glad you all like it :) This is actually my first time using javascript, so if there's any issues please let me know and I'll do my best to fix them.
- windsignaling 5y ago"first time using javascript" Impressive. I think the first time I used Javascript I made a button.
- aeg42x 5y agoYeah if you go back to my first commits, the first month is mostly me getting very excited about drawing a square haha. I can’t recommend the PIXI graphics library enough, it was super easy to pick up.
- Pensacola 5y agoHi, nice site! But since you asked, here's an issue: the little "Click to increase or decrease weights" feature doesn't work in Firefox.
- aeg42x 5y agoThank you! It should be working now!
- kebsup 5y agoVery nice. I've have created very similar thing a few years ago, but yours is nicer. :D https://nnplayground.com https://nnplayground.com
- abraxas 5y agoFor those who want to take this a bit further I can't recommend cs231n enough. Especially when it was taught by Andrej Karpathy. I believe the lectures may still be up on YouTube. Andrej really has a knack for teaching.
- deleted 5y ago[deleted]