Yes, unfortunately I have yet to beat that technique in wallclock convergence speed vs just using the larger network from the start. :'(((( Whoever figures out

Yes, unfortunately I have yet to beat that technique in wallclock convergence speed vs just using the larger network from the start. :'(((( Whoever figures out how to clearly and effectively do it consistently faster than a 'full network from the start' version will open up an entirely new subfield of neural network training efficiency. :'))))