4 ms·

I have read this in several places and want to learn more. Do you have a reference handy?

• nephanth 2y ago
  This is something I saw a talk about a while ago. There are probably more recent papers on this topic, you might want to look browse the citations of this one https://arxiv.org/abs/2003.00307 https://arxiv.org/abs/2003.00307
• blt 2y ago
  [1] Was an empirical paper that inspired much theoretical follow-up. [2] Is one such follow-up, and the references therein should point to many of the other key works in the years between. [3] Introduces the neural tangent kernel (NTK), a theoretical tool used in much of this work. (Not everyone agrees that reliance on NTK is the right way towards long-term theoretical progress.) [4] Is a more recent paper I haven't read yet that goes into more detail on interpolation. Its authors were well known in more "clean" parts of ML theory (e.g. bandits) and recently began studying deep learning. --- [1] Understanding deep learning requires rethinking generalization. Zhang et al., arXiv, 2016. https://arxiv.org/abs/1611.03530 https://arxiv.org/abs/1611.03530 [2] Stochastic Mirror Descent on Overparameterized Nonlinear Models: Convergence, Implicit Regularization, and Generalization. Azizan et al., arXiv, 2019. https://arxiv.org/abs/1906.03830 https://arxiv.org/abs/1906.03830. [3] Neural Tangent Kernel: Convergence and Generalization in Neural Networks. Jacot et al., NeurIPS, 2018. https://proceedings.neurips.cc/paper/2018/hash/5a4be1fa34e62bb8a6ec6b91d2462f5a-Abstract.html https://proceedings.neurips.cc/paper/2018/hash/5a4be1fa34e62... [4] A Universal Law of Robustness via Isoperimetry. Bubeck et al., NeurIPS, 2021. https://proceedings.neurips.cc/paper/2021/hash/f197002b9a0853eca5e046d9ca4663d5-Abstract.html https://proceedings.neurips.cc/paper/2021/hash/f197002b9a085...
  • patrick451 2y ago
    Awesome, thank you!