4 ms·
Nice write up. I gave a presentation on DBNs for my Neural Networks class in Fall 2011. If you'd like references to the relevant papers and some more details on
by tansey 14y ago
Nice write up. I gave a presentation on DBNs for my Neural Networks class in Fall 2011. If you'd like references to the relevant papers and some more details on the algorithms and applications, here are the slides: https://docs.google.com/presentation/d/18vJ2mOmb-Cbqsk0aNoUMNKCjEaiWyryFAVBxM3nMFfY/edit?usp=sharing https://docs.google.com/presentation/d/18vJ2mOmb-Cbqsk0aNoUM...
- SatvikBeri 14y agoThis was a really fun read-thank you! If you don't mind answering, I've got a few questions: 1. "Vanishing gradients after 2-3 layers"-does this mean that the partial derivatives tend to be smaller on the higher layers, and therefore the network finds local minima that aren't very useful? 2. Step 3 (p 18) mentions that the outputs are not continuous variables, they're binary. What's the reasoning behind that?
- tansey 14y ago1. Basically. It means that the network has a hard time pulling itself in any direction since the weights in the deeper layers are never really adjusted by very much. 2. It's been a while since I read the paper, but I believe that the justification has to do with the proof of convergence of Gibbs sampling. I haven't tried using continuous values, so I can't give an intuition for what happens in those cases.