4 ms·
How does this relate to previous work that's described NNets in the limit of infinite width as GPs?
by byt143 8y ago
How does this relate to previous work that's described NNets in the limit of infinite width as GPs?
- duvenaud 8y agoOoo, great question. In both cases, we define a function using a sum over an infinite number of infinitesimal things. In the GP case, those things are random and independent of each other, so the central limit theorem applies and we get a simple Gaussian. In the ODE case, the infinitesimals are deterministic and depend on the previous ones in the sequence, so the final answer is deterministic and impossible to compute exactly in general. You could also use a GP to model the dynamics of an ODE, and this was done recently: [https://arxiv.org/abs/1803.04303 https://arxiv.org/abs/1803.04303] although the drawback was that they couldn't train the model by backpropagation.