5 ms·
I had success doubling the width of a layer by copying, adding epsilon noise and then fine tuning. Transform from [X] to [X X+epsilon]. In that case I was try
by hiddencost 3y ago
I had success doubling the width of a layer by copying, adding epsilon noise and then fine tuning.
Transform from [X] to [X X+epsilon].
In that case I was trying to fuse two signals that had fairly similar characteristics.
- tysam_and 3y agoYes, unfortunately I have yet to beat that technique in wallclock convergence speed vs just using the larger network from the start. :'(((( Whoever figures out how to clearly and effectively do it consistently faster than a 'full network from the start' version will open up an entirely new subfield of neural network training efficiency. :'))))