3 ms·
It can represent any arbitrary function. If you are using gradient descent, then you'll need the desired loss function to be differentiable with respect to the
by dual_basis 7y ago
It can represent any arbitrary function.
If you are using gradient descent, then you'll need the desired loss function to be differentiable with respect to the parameters, but that's a totally different matter.
- braised_babbage 7y agoNo, neural networks cannot "represent any arbitrary function". Find a theorem that you think states otherwise, and then read what the theorem actually states.
- dual_basis 7y agoYou're correct, the statement of the theorem refers to continuous functions. In practice, however, the input to neural networks is represented by floating point values, which is a discrete set. So pick whatever arbitrary function you would like, there is some continuous approximation to that function which is actually equal to it on every floating point value, and that function can be approximated arbitrarily closely by a neural network.