2 ms·
Deep Learning Meets Sparse Regularization: Signal Processing Perspective (2023)
- nerdponx 4y agoI haven't gone through this in detail yet, but I love articles like this. I've always wanted to see some tidier theory around neural networks to help guide my intuition when working with them. It's nice to get more insight into what the heck these emergent things we build actually look like, and how they manage to work so well. It's also interesting that neural networks with sparse neurons start to look a bit like decision tree outputs. I suppose the big difference is that e.g. gradient boosted trees are built greedily one tree at a time, while neural networks are built all at once.