> Up until now learning to replicate the input exactly was called overfitting and considered a bug, not a feature, but they showed a completely new way to wield

> Up until now learning to replicate the input exactly was called overfitting and considered a bug, not a feature, but they showed a completely new way to wield neural nets. Well, for that particular ting there was a predecessor of sort in Deep Image Priors from 2017. https://arxiv.org/abs/1711.10925 https://arxiv.org/abs/1711.10925 That was all about overfitting a neural net on a single image, which they used to get impressive inpainting, noise removal and superresolution results without any training at all (though of course it did not beat state of the art training-based approaches, even then.) I had a lot of fun playing around with it when it came. The idea is dead simple, within reach to implement yourself with no complex mathematical understanding.