Exactly my thoughts on this.. if you could detect when the network was getting saturated or over fitted, spawn new layers. Maybe this would allow starting with

Exactly my thoughts on this.. if you could detect when the network was getting saturated or over fitted, spawn new layers. Maybe this would allow starting with much simpler networks, operating on lower rez inputs, and then scaling then up as you feed them more data, instead of burning cycles slowing training a large network from scratch.