4 ms·
Is it just me or would this not drastically speed up calculation of the weights for neural nets?
by fleitz 11y ago
Is it just me or would this not drastically speed up calculation of the weights for neural nets?
- zzleeper 11y agoWell the Ax=b part is quite general as long as A has a special structure (diagonally dominant). Good luck coding it though, I'm still looking for an implementation of a low stretch spanning tree (needed in order for the solver to be really lineaR)
- espeed 11y ago"Using Petal-Decompositions to Build a Low Stretch Spanning Tree" (http://research.microsoft.com/pubs/163400/spanning-full1.pdf http://research.microsoft.com/pubs/163400/spanning-full1.pdf): Stretch: O(log n log log n) Time: O(m log n log log n)
- hiddencost 11y agoSVD is in fact used to compress dnns. Of course, like many things in ML, as soon as you make it faster, you just make the model bigger and turn the speed gains into accuracy gains.
- mike_mg 11y agoCould you please explain me how? I watched the lecture, but can't make connection. I understand how neural networks work and how they are optimized. I'm stupid. Thanks
- fleitz 11y agoHeat flow is akin to 'information' flow in a neural net because the weight of a node is the sum/average of the weights of all nodes it's connected to just like the heat of a node is the sum/average of the weights of all the nodes its connected to.