3 ms·
Have you ever considered visualizing artificial neural networks in training or in operation? Speaking purely from a selfish standpoint, it would be awesome to
by cs702 12y ago
Have you ever considered visualizing artificial neural networks in training or in operation?
Speaking purely from a selfish standpoint, it would be awesome to SEE what back-propagation "looks like" with different neuronal activation functions, or what feature learning by restricted Boltzmann machines "looks like," or how dropout causes networks to generalize better -- to name just a few possibilities.
If anyone can visualize neural network algorithms in a way that is intuitive and beautiful, it's you!
- Osmium 12y agoSeconded. I was recently trying to find a good resource to simply and concisely explain (or at least give a 'feel' for) artificial neural networks to someone and came up blank. Everything I found was either assumed too much domain-specific knowledge or was too long.
- reader5000 12y agoIs there a "feel" to neural networks? You define a bunch of hidden/latent variables connected in an array (or multiple arrays), then perform a heuristic search of the weight state space to minimize some error/energy function. Sometimes it works, often not.
- c0g 12y agoI love how you're getting downvoted when the truth is NNs work well _only_ when you have a team of specialists to select their hyperparameters. There's a rich vein of research for AutoML to automatically learn the hyps.
- chestervonwinch 12y agoI don't know if this is the type of thing you're looking for, BUT... I made this little web app for visualizing the prediction gradient for a binary classification problem with 2 dimensional inputs [0]. You basically add members of each class by clicking in the plane. It has some bells and whistles for viewing the decision gradient (the probability of any given point in space being of one class or the other) and how this changes after each training. You can can also do things like change the learning rate, edit weight values, and add noise to weights. In this toy problem, you can get more of a "feel" for the complexity capability of a network (by trying different clusterings of classes, etc...). Unfortunately, I hard coded it to have 2 hidden units. In retrospect, it would have been better to make the number of hidden units tunable as well so that one could visualize how a network with more non-linearities can draw increasingly complex decision boundaries. [0] http://www.math.fsu.edu/~mhancock/#!/software/web-apps/neural-net/ http://www.math.fsu.edu/~mhancock/#!/software/web-apps/neura...
- mbostock 12y agoI haven’t done much with neural networks, but Christopher Olah wrote a fantastic post on visualizing low-dimensional ones: http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/ http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/
- pospeselr 12y agoIf you're interested in visualizing RBMs or AutoEncoders in real time, you should check out my project VisualRBM: https://code.google.com/p/visual-rbm/wiki/Screenshots https://code.google.com/p/visual-rbm/wiki/Screenshots It basically lets you do exactly what you're looking for. I'm going to eventually add support for visualizing each layer of a DNN during back-propagation training, but that's several releases away.
- strebler 12y agoI have found this useful for helping visualize what's happening inside a neural net: http://cs.stanford.edu/people/karpathy/convnetjs/ http://cs.stanford.edu/people/karpathy/convnetjs/