> Don't big alarm bells go off in your head when you hear N parameters for N data points? Gaussian process models typically also have hyperparameters that cont

> Don't big alarm bells go off in your head when you hear N parameters for N data points? Gaussian process models typically also have hyperparameters that control things like the smoothness of the learned surface. My hand-wavey sense of how GPs work is that you have a prior distribution over functions (think curves) and a hyperparameter controlling the smoothness/length-scale of those functions. You observe data, then find the posterior distribution over functions that make your data most likely - i.e. of all these possible functions, which ones have the best chance of actually generating the data you're observing? The shape of that posterior distribution depends on the set of N points you observe, and it will be different if you have different data. In that sense, then, the data are parameters, since the posterior depends exactly on the set of data. The analogy to traditional regression comes by having the hyperparameters (controlling the covariance function IIRC, i.e. the smoothness of the posterior curves), which allows the resulting posterior distribution to explain the data perfectly (overfitting, sharp curves) or not (generalizability, smooth curves). See for example slides 23-24 of this slide deck: https://www.cs.toronto.edu/~hinton/csc2515/notes/gp_slides_fall08.pdf https://www.cs.toronto.edu/~hinton/csc2515/notes/gp_slides_f... Rasmussen has some useful video tutorials of all this. It's been a few years since I've watched them but if you're interested I'd recommend them.