3 ms·
Compared to training the CNN, running a GP on the outputs seemed like a worthwhile tradeoff of time if it resulted in a discernible difference in validation err
by bhl 7y ago
Compared to training the CNN, running a GP on the outputs seemed like a worthwhile tradeoff of time if it resulted in a discernible difference in validation error; and the original authors of the paper we were following already had an open implementation.
I am open to suggestions if you know of any "post-processing" methods that would help to incorporate locality into the overall prediction.
- radarsat1 7y agoWhat I mean is, why GP instead of, say polynomial regression, or training a NN on the CNN outputs? I'm curious if the decision was due to some feature of GPs that you really needed, and could not easily get otherwise. I'm not doubting your choice, really, I'm just trying to get a feeling for what niche GPs fill (given their difficult computational requirements), as I have never found the opportunity to use them. > that would help to incorporate locality into the overall prediction. Not sure what you mean by this so I am sure I am missing something.
- bhl 7y agoWe used a Gaussian process on the outputs from a CNN, not separately. What I meant by locality is the idea that two points which are close to each other should also have predictions close to each other, assuming that there is some underlying Gaussian variable we can't see immediately from the data or from the CNN. One example could be housing prices: if your neighbor sold their house, controlled for other factors like square ft or the number of bedrooms, you should value your own house similarly. We assume that there is this underlying quality of the neighborhood, possibly given by distance between houses. The niche that GPs fill is broad. Two applications I've seen were their use in trying to find the missing MH370 flight, and another is in hyperparameter tuning of ML models.