4 ms·
Has anyone been able to reproduce double descent without trying to invert a seriously ill-posed linear system? Showing that a solver has numerical issues which
by johndough 3y ago
Has anyone been able to reproduce double descent without trying to invert a seriously ill-posed linear system? Showing that a solver has numerical issues which go away when enough parameters are introduced does not really prove anything. This looks much more like regularization though floating point rounding.
By the way, the linked Colab notebook is missing a "self." in front of the "lamb" in the unused regularization branch of the fit function.
- sebzim4500 3y agoOpenAI paper on double descent: https://arxiv.org/pdf/1912.02292.pdf https://arxiv.org/pdf/1912.02292.pdf They show it happens across a variety of architectures and real world datasets