4 ms·
Independently of the scaling, I wonder if someone tried to initialize a deep neural network with a low-discrepancy sequence (also called quasi-random numbers) i
by nestorD 7y ago
Independently of the scaling, I wonder if someone tried to initialize a deep neural network with a low-discrepancy sequence (also called quasi-random numbers) instead of a uniform or gaussian distribution.
- p1esk 7y agoWhat would be the advantage of that?
- nestorD 7y agoBetter coverage of the space of weights (less clumps and holes). If I understand the lottery ticket hypothesis [0] correctly, this would lead to a better exploration of the space and thus better results. [0] https://arxiv.org/abs/1803.03635 https://arxiv.org/abs/1803.03635