3 ms·

But… he’s saying something here that is academically true: that neural networks can approximate any possible function, to any arbitrary degree of precision you

by grbsh 2y ago

But… he’s saying something here that is academically true: that neural networks can approximate any possible function, to any arbitrary degree of precision you require (given infinite capacity / depth). https://en.m.wikipedia.org/wiki/Universal_approximation_theorem https://en.m.wikipedia.org/wiki/Universal_approximation_theo... I will highlight one thing, which is that the theorem does not say anything about it being practical to learn this function, given available data or any specific optimization technique.

rossdavidh 2y ago
'the underlying “rules” that produce any distribution of data...' is clearly meant to convince the reader that it can produce something we would describe as a "rule", that is, a coherent and comprehensible regulating principle. This isn't just because he isn't being precise enough; he quite clearly wants the reader to understand this as neural networks being able to create a mental model of anything, in a manner similar to how a biological neural network would. It doesn't, it can't, and it won't in our lifetimes.