An excellent paper which uses (some of the) results we have also found studying the weight matrices of neural networks...namely that they rarely undergo rank co

An excellent paper which uses (some of the) results we have also found studying the weight matrices of neural networks...namely that they rarely undergo rank collapse https://calculatedcontent.com/2018/09/21/rank-collapse-in-deep-learning/ https://calculatedcontent.com/2018/09/21/rank-collapse-in-de... But they miss something..the weight matrices also display power law behavior. https://calculatedcontent.com/2018/09/09/power-laws-in-deep-learning/ https://calculatedcontent.com/2018/09/09/power-laws-in-deep-... This is also important because it was suggested in the early 90s that Heavy Tailed Spin Glasses would have a single local mimima. This fact is the basis of my early suggestion that DNNs would exhibit a spin funnel