4 ms·

Interactive Visualization of Gaussian Processes

145 points by sebg 5y ago

clircle 5y ago
This is very nice. Gaussian process regression didn't click for me until I thought of the data as a partially observed sample of size one from a stochastic process. Practically, I have a hard time using Gaussian process regression. I find regression with splines to be reasonable and fast, and I don't have to fret about the nugget parameter or the covariance function structure. But I admit GP regression has a beautiful theory. But there is an equivalence between (smoothing) splines and certain types of Gaussian process models. [1] [1] http://pages.stat.wisc.edu/~wahba/ftp1/oldie/kw70bayes.pdf http://pages.stat.wisc.edu/~wahba/ftp1/oldie/kw70bayes.pdf
- nestorD 5y ago
  For me the good reason to use gaussian regression is the fact that you get an uncertainty on the output. The big downside is that it takes expert knowledge (to design a proper kernel) and a solid implementation (to avoid the various numerical problems they can produce) to apply them to practical problem. Most implementation either break down very quickly or are not flexible enough for my taste. I have a Rust implementation [0] which tries to help with the flexibility aspect but it is still very far from perfect. [0]: https://github.com/nestordemeure/friedrich https://github.com/nestordemeure/friedrich
  - clircle 5y ago
    Yep, uncertainty intervals are definitely easier to get with gp regression.
  - singhrac 5y ago
    I've found it fairly easy to get high performance GPRs fitted with gpytorch; kernel design is another problem altogether. I think they take more tinkering than an equivalent neural network by way of the Gaussian process prior. In other words, we want a "smooth" function, but how smooth do we really want it?
emsal 5y ago
This is awesome! I really wonder if GPs will see an uptick in usage in the future, I've seen a lot of interest in them lately. Also, thanks for including the source so they can be seen in the browser dev tools, don't know if that gets done enough with interactive visualizations like this.
bernardv 5y ago
Nicely done. One can study equations all day long and not get the kind of insight a good interactive visualization provides in just a few minutes.
timeinput 5y ago
This is really nice. Have you given any thought to visualization of non-Gaussian processes / systems (think particle clouds, or systems that are mixes of Gaussian and non-Gaussian processes?)
trombonechamp 5y ago
In case anyone else wants to know how the Gaussian process is animated, here are the two links provided in the code: https://math.stackexchange.com/a/1930880/901583 https://math.stackexchange.com/a/1930880/901583 http://mlss.tuebingen.mpg.de/2013/Hennig_2013_Animating_Samples_from_Gaussian_Distributions.pdf http://mlss.tuebingen.mpg.de/2013/Hennig_2013_Animating_Samp...
ludwigschubert 5y ago
Well done, I particularly enjoy the rug-plot layout and the “marginalized” densities actually being in the margin of the diagram. \o/ Aligning axes across plots/small multiples is such an underused technique imho! (Have you considered writing the authors of the Distill article/making a PR to their article? Just wondering; your diagram is already nicely presented on your blog.)

145 points by sebg 5y ago

This is very nice. Gaussian process regression didn't click for me until I thought of the data as a partially observed sample of size one from a stochastic process. Practically, I have a hard time using Gaussian process regression. I find regression with splines to be reasonable and fast, and I don't have to fret about the nugget parameter or the covariance function structure. But I admit GP regression has a beautiful theory. But there is an equivalence between (smoothing) splines and certain types of Gaussian process models. [1] [1] http://pages.stat.wisc.edu/~wahba/ftp1/oldie/kw70bayes.pdf http://pages.stat.wisc.edu/~wahba/ftp1/oldie/kw70bayes.pdf

nestorD 5y ago
For me the good reason to use gaussian regression is the fact that you get an uncertainty on the output. The big downside is that it takes expert knowledge (to design a proper kernel) and a solid implementation (to avoid the various numerical problems they can produce) to apply them to practical problem. Most implementation either break down very quickly or are not flexible enough for my taste. I have a Rust implementation [0] which tries to help with the flexibility aspect but it is still very far from perfect. [0]: https://github.com/nestordemeure/friedrich https://github.com/nestordemeure/friedrich
- clircle 5y ago
  Yep, uncertainty intervals are definitely easier to get with gp regression.
- singhrac 5y ago
  I've found it fairly easy to get high performance GPRs fitted with gpytorch; kernel design is another problem altogether. I think they take more tinkering than an equivalent neural network by way of the Gaussian process prior. In other words, we want a "smooth" function, but how smooth do we really want it?

emsal 5y ago

This is awesome! I really wonder if GPs will see an uptick in usage in the future, I've seen a lot of interest in them lately. Also, thanks for including the source so they can be seen in the browser dev tools, don't know if that gets done enough with interactive visualizations like this.

bernardv 5y ago

Nicely done. One can study equations all day long and not get the kind of insight a good interactive visualization provides in just a few minutes.

timeinput 5y ago

This is really nice. Have you given any thought to visualization of non-Gaussian processes / systems (think particle clouds, or systems that are mixes of Gaussian and non-Gaussian processes?)

trombonechamp 5y ago

In case anyone else wants to know how the Gaussian process is animated, here are the two links provided in the code: https://math.stackexchange.com/a/1930880/901583 https://math.stackexchange.com/a/1930880/901583 http://mlss.tuebingen.mpg.de/2013/Hennig_2013_Animating_Samples_from_Gaussian_Distributions.pdf http://mlss.tuebingen.mpg.de/2013/Hennig_2013_Animating_Samp...

ludwigschubert 5y ago

Well done, I particularly enjoy the rug-plot layout and the “marginalized” densities actually being in the margin of the diagram. \o/ Aligning axes across plots/small multiples is such an underused technique imho! (Have you considered writing the authors of the Distill article/making a PR to their article? Just wondering; your diagram is already nicely presented on your blog.)