4 ms·
I'm curious, glancing at the code I can't tell immediately, but how does the DNN trainer (Deep Neural Network) differ from traditional back propagation? Does i
by radarsat1 11y ago
I'm curious, glancing at the code I can't tell immediately, but how does the DNN trainer (Deep Neural Network) differ from traditional back propagation? Does it train layer by layer using a kind of successive autoencoder approach? (If that makes any sense..)
I'm curious whether this would perform differently than e.g. Keras, on deeper networks.
- nrmn 11y agoNo they all use the same general principal of backpropagation to do the training. Different flavours of optimizers exist with different tweaks and additions to speed training up. Relevant file in project: https://github.com/tflearn/tflearn/blob/0.1.0/tflearn/optimizers.py https://github.com/tflearn/tflearn/blob/0.1.0/tflearn/optimi...
- radarsat1 11y agoSo it's not common to use a layer-by-layer training approach for deep nets? I thought that was one of the main things that made a huge difference and enabled the "deep" revolution. Anyways, isn't vanishing gradients still a problem? If so, how do people use these frameworks for deep nets? Otherwise, how is the problem resolved? I thought vanishing gradients was an issue for anything with more than 2 or 3 layers.
- make3 11y agoafaik this and Keras have identical mathematical methods, backpropagated sgd with some inertia scheme