3 ms·
It’d be interesting to compare this approach against a simpler baseline: setting a different (10 – 100 times higher?) learning rate for a fraction (10% ?) of n
by dchichkov 9y ago
It’d be interesting to compare this approach against a simpler baseline: setting a different (10 – 100 times higher?) learning rate for a fraction (10% ?) of neurons in an LSTM.
- fizx 9y agoInteresting... I mean knockout already does this (sort of), by setting the learning rate of 10% artificially low. I'm not sure if inverting the common thing is useful, but it might be fun to try.