4 ms·
This does not seem to be a contradiction. Once you are in the right region of solution space training is expected to be faster and easier. Re-initialization cou
by longtom 7y ago
This does not seem to be a contradiction. Once you are in the right region of solution space training is expected to be faster and easier. Re-initialization could have a regularizing effect, explaining the better performance.
- Akababa 7y agoThey re-use the same initialization, so it appears that the initial weights are inherently coupled with the nonzero structure.