I haven't read the paper yet, but when people say things like "We don't know why neural networks work", what they mean is we don't understand why they generalis

I haven't read the paper yet, but when people say things like "We don't know why neural networks work", what they mean is we don't understand why they generalise so we'll, by most prior theory they are quite over parameterized, so while they should be able to fit the data - and it has been shown that typical architectures can fit random data - previous theory said the penalty for this is that it will not generalize well to new data from the same distribution. So generalization bounds on NNs are actually the key thing that people want from theory.