3 ms·
> And around ~2012, a bunch of researchers have reported you don't even need 2nd-derivative information. You just have to initialize the neural net properly. T
by jules 13y ago
> And around ~2012, a bunch of researchers have reported you don't even need 2nd-derivative information. You just have to initialize the neural net properly.
This sounds very interesting. How do you property initialize the weights? Do you have a link to a paper about this?
- kkjkok 13y agoCheck out this paper: Practical recommendations for gradient-based training of deep architectures, Y. Bengio http://arxiv.org/abs/1206.5533 http://arxiv.org/abs/1206.5533 There is a section on weight initialization on page 15. In general, this paper has a lot of good information in one place.