Full pub is up on arXiv here: https://arxiv.org/pdf/2212.13881.pdf https://arxiv.org/pdf/2212.13881.pdf My 2c: The phys.org summary isn't great. The authors a

Full pub is up on arXiv here: https://arxiv.org/pdf/2212.13881.pdf https://arxiv.org/pdf/2212.13881.pdf My 2c: The phys.org summary isn't great. The authors are focused on a much narrower topic than simply "how NNs learn", they're trying to characterize the mechanism by which deep NNs develop 'features'. They identify a quantity which is definable for each layer of the NN (outer product of the input weight matrix), and posit that this quantity is proportional to the average derivative of the layer with respect to its inputs, where the average is taken over all training data. They (claim to, I haven't evaluated) prove this formally for the case of deep FNNs trained on gradient descent. They argue that, by treating this quantity as a measure of 'feature importance', it can be used to explain certain behaviors of NNs that are otherwise difficult to understand. Specifically they address: * Simple and spurious features (they argue that their proposal can identify when this has occurred) * "Lottery ticket" NNs (they argue that their proposal can help explain why pruning the connections of a fully connected NN improves its performance) * Grokking (they argue that their proposal can help explain why NNs can exhibit sudden performance improvements, even after training performance is 100%) Finally they propose a heuristic ML algorithm which updates their proposal directly during training, rather than the underlying weights, and shows that this achieves superior performance to some existing alternatives. Overall I would say that they have defined a nice tool for measuring NN feature importance, in terms of features that the NN itself is defining (not in terms of the original space). I can definitely see why this has a lot of value, and I'm especially intrigued by their comparisons of the NFM to testing performance in their 'grokking' case study. With that said, I'm not really active in the NN space, so it seems a little surprising that their result is really that novel. The quantity they define (outer product of the weight matrix) seems fairly intuitive as a way to rank the importance of inputs at any given layer of the NN, so I'm wondering if truly nobody else has ever done this before? Possibly the novelty is in their derivation of the proportionality, or in the analysis of this quantity over training iterations. I'd guess that their model proposal is totally new, and I'm curious to try it out on some test cases, it seems promising for cases where light-weight models are required. It also seems interesting to point out how both training performance AND the development of feature importance both jointly influence testing accuracy, but again I'm surprised that this is really novel. I also have to wonder how this extends to more complicated architectures with, eg. recursive elements; it's not discussed anywhere, but seems like it would be an important extension of this framework given where genAI is currently at (although first draft was pub'd in '22 so it's possible that this just wasn't as pressing when it was being written).