3 ms·

I'm not statistically trained but am trying to study by myself. The idea that GP is the extension to the mixed linear models is very interesting. However I stil

I'm not statistically trained but am trying to study by myself. The idea that GP is the extension to the mixed linear models is very interesting. However I still struggle to understand the relationships here, i.e in mixed models the different slopes are reflecting the different factors (categories, groups etc), but how do they extend to the GP space? Also, it's easier to imagine the effects (or surrogates) of random variables such as Age, but a lot of the times in molecular biology there are independent variables that are not explainable (e.g gene expression level). Can they become intuitive with GPs?

itissid 5y ago
Here is an example say you are modeling cancer rates on a county level to discover anomalies and prima facie you don't have any hypothesis to believe that counties interact in a special way for this kind of cancer(there maybe inherent unknown interactions). All you have are data on counties(discrete) you can model the cancer rate T_i using a Poisson and a log link function: `T_i ~ Poisson(y_i)` and the link function: `y_i~ a_{unit[i]} + b_{unit[i]}*x_i` In a Mixed model without GPs you would assume a(simple) normal prior on each of the slopes and intercepts. Then you would say something like: `a_{unit[i]} ~ N(c, d)` and and similarly for b. You could also do a multivariate normal priors on all the a's and b's which can also help you pool across groups that are "similar", but you still don't know what those interactions are. But say we change the hypothesis, like say there is reason to believe that certain counties might interact for cancer rates strongly with a continuous variable like distance to each other then you could try and model the covariance of the prior as a function of the distance matrix between counties `b ~ MVN(c, K) ` where $K ~ \gamma (exp(-1*D_{i, j}^2))$ where D is the distance matrix between counties. That is what a GP helps with; More model structure. You essentially have a "distance" measure of some sort in this case. And this can be distance in any space like for modeling housing prices `b` could be a "locality" coefficient modeled on how similar it is to houses in a 10 block radius or something. ~~~~~~ This is the technical part. The intuitive part is essentially about seeing if your problem has information providing more structure. Problems benefitting from GPs are ones where you have lots of auxillary data your model can gain from like demographics, census data which is general and not collected as a part of the problem you are solving but you have an hypothesis upfront that needs to incorporate that data and structure. The other piece of intuition is that similar populations behave similarly than those that are very distinct. Like ages 15-25 will be more similar amongst themselves, so will the group 40-50, but not when you compare across groups. There is an exponential falloff between these groups of some similarity measure, but its continuous not discrete.
- dumb1224 5y ago
  Thanks very much for the detailed example. This particular description is very helpful >There is an exponential falloff between these groups of some similarity measure, but its continuous not discrete. That's a lot of food for thoughts for me to digest, however I still struggle to imagine the more refined structure that GPs can capture via the form of covariance function. More reading I guess. Thanks again :-D

itissid 5y ago

Here is an example say you are modeling cancer rates on a county level to discover anomalies and prima facie you don't have any hypothesis to believe that counties interact in a special way for this kind of cancer(there maybe inherent unknown interactions). All you have are data on counties(discrete) you can model the cancer rate T_i using a Poisson and a log link function: `T_i ~ Poisson(y_i)` and the link function: `y_i~ a_{unit[i]} + b_{unit[i]}*x_i` In a Mixed model without GPs you would assume a(simple) normal prior on each of the slopes and intercepts. Then you would say something like: `a_{unit[i]} ~ N(c, d)` and and similarly for b. You could also do a multivariate normal priors on all the a's and b's which can also help you pool across groups that are "similar", but you still don't know what those interactions are. But say we change the hypothesis, like say there is reason to believe that certain counties might interact for cancer rates strongly with a continuous variable like distance to each other then you could try and model the covariance of the prior as a function of the distance matrix between counties `b ~ MVN(c, K) ` where $K ~ \gamma (exp(-1*D_{i, j}^2))$ where D is the distance matrix between counties. That is what a GP helps with; More model structure. You essentially have a "distance" measure of some sort in this case. And this can be distance in any space like for modeling housing prices `b` could be a "locality" coefficient modeled on how similar it is to houses in a 10 block radius or something. ~~~~~~ This is the technical part. The intuitive part is essentially about seeing if your problem has information providing more structure. Problems benefitting from GPs are ones where you have lots of auxillary data your model can gain from like demographics, census data which is general and not collected as a part of the problem you are solving but you have an hypothesis upfront that needs to incorporate that data and structure. The other piece of intuition is that similar populations behave similarly than those that are very distinct. Like ages 15-25 will be more similar amongst themselves, so will the group 40-50, but not when you compare across groups. There is an exponential falloff between these groups of some similarity measure, but its continuous not discrete.

dumb1224 5y ago
Thanks very much for the detailed example. This particular description is very helpful >There is an exponential falloff between these groups of some similarity measure, but its continuous not discrete. That's a lot of food for thoughts for me to digest, however I still struggle to imagine the more refined structure that GPs can capture via the form of covariance function. More reading I guess. Thanks again :-D