I guess this boils down to something similar to the approximation of a continuous function by polynomials: given any finite number of values (x_i, y_i), you can

I guess this boils down to something similar to the approximation of a continuous function by polynomials: given any finite number of values (x_i, y_i), you can always find a polynomial passing through them. However, there are continuous functions which are nowhere differentiable and which have unbounded variation between x_i and x_{i+1} for each i. The "polynomials" are the "nice images" and the "continuous with unbounded variation" are the "noisy ones". Disclaimer: I am neither a lawyer not an expert on NN. Edit: I would like to be corrected if I the similitude is wrong.