17 ms·
This is very nice. Gaussian process regression didn't click for me until I thought of the data as a partially observed sample of size one from a stochastic proc
by clircle 5y ago
This is very nice. Gaussian process regression didn't click for me until I thought of the data as a partially observed sample of size one from a stochastic process.
Practically, I have a hard time using Gaussian process regression. I find regression with splines to be reasonable and fast, and I don't have to fret about the nugget parameter or the covariance function structure. But I admit GP regression has a beautiful theory.
But there is an equivalence between (smoothing) splines and certain types of Gaussian process models. [1]
[1] http://pages.stat.wisc.edu/~wahba/ftp1/oldie/kw70bayes.pdf http://pages.stat.wisc.edu/~wahba/ftp1/oldie/kw70bayes.pdf
- nestorD 5y agoFor me the good reason to use gaussian regression is the fact that you get an uncertainty on the output. The big downside is that it takes expert knowledge (to design a proper kernel) and a solid implementation (to avoid the various numerical problems they can produce) to apply them to practical problem. Most implementation either break down very quickly or are not flexible enough for my taste. I have a Rust implementation [0] which tries to help with the flexibility aspect but it is still very far from perfect. [0]: https://github.com/nestordemeure/friedrich https://github.com/nestordemeure/friedrich
- clircle 5y agoYep, uncertainty intervals are definitely easier to get with gp regression.
- singhrac 5y agoI've found it fairly easy to get high performance GPRs fitted with gpytorch; kernel design is another problem altogether. I think they take more tinkering than an equivalent neural network by way of the Gaussian process prior. In other words, we want a "smooth" function, but how smooth do we really want it?