4 ms·
Deep Dive into Math Behind Deep Networks
- nafizh 8y agoI would like to see someone explain the math behind recurrent neural networks. Feed-forward neural networks are fairly straight-forward, and there are many, many blog posts explaining them already.
- deleted 8y ago[deleted]
- pmalynin 8y agoIf you unroll an RNN (which is what is done) then you have many copies of a single Feed Forward Network. Nothing fancy. The gradient gets accumulated.
- jing 8y agoI think this resource could be helpful: http://colah.github.io/posts/2015-08-Understanding-LSTMs/ http://colah.github.io/posts/2015-08-Understanding-LSTMs/ Essentially, RNNs and feed forward networks are very similar - RNNs are just "unrolled through time" and every timestep shares the same weights. The activations are slightly different as well, but the core concept is the same as feed forward networks; it's not a completely different concept or idea.
- mjfl 8y agoisn't it just gradient descent?
- ousta 8y agoyes just like a big chunk of what ML is.
- platz 8y agoBackpropagation section was just a list of formulas, and "it's because of the chain rule".
- m0zg 8y agoThat's not a "deep dive" that's the disappointingly barest minimum.