3 ms·
It would be interesting to see neural networks that can grow layers as needed using this technique. Perhaps you could have the network occasionally monitor its
by dontreact 8y ago
It would be interesting to see neural networks that can grow layers as needed using this technique. Perhaps you could have the network occasionally monitor its own progress on a tune set somehow and adjust the model capacity based on the amount of over/underfitting observed.
- thrax 8y agoExactly my thoughts on this.. if you could detect when the network was getting saturated or over fitted, spawn new layers. Maybe this would allow starting with much simpler networks, operating on lower rez inputs, and then scaling then up as you feed them more data, instead of burning cycles slowing training a large network from scratch.