3 ms·

What is your definition for "explanatory", in this context? I agree with your second sentence - the key question is indeed whether the model is explanatory rat

by mrow84 10y ago

What is your definition for "explanatory", in this context? I agree with your second sentence - the key question is indeed whether the model is explanatory rather than merely predictive. I offered a definition for explanatory as being when "the structure of your model reflects something about the structure of reality, and you can explore your model as though you were exploring reality", which isn't a terrible attempt, from my experience. At the risk of repeating myself ad nauseam, the relationship to model fitting is found in the presence or absence of additional assumptions required for finding your fit. The difference is between having a very low-dimensional model that fits the data and requires few if any extra assumptions to fit (fitting the model being equivalent to "validating your theory", in this context), or a very high-dimensional model that fits the data (making no claim to "theory"), but by definition requires extra assumptions to get the fit. In another sub-thread you said: > If you model a system using the smallest possible mathematical model you don't, from that act alone, understand how the system works. This is correct inasmuch as the understanding doesn't leap forth immediately, but if the model is a good representation of the data (an important if), then modelling a system in a parsimonious way possible does provide you with understanding, pretty much for free, by looking for systems with a similar structure and learning about their properties. As an example, if you have some random variable, and you realise that it might be modelled with a Poisson distribution, then (assuming you are correct) you immediately gain a lot of understanding, because there is an enormous amount of literature exploring the implications of such a model. This is what substantiates the link between model fitting and the explanatory vs. predictive question. If you can successfully fit a small model to a problem, without adding assumptions, then that model gives you understanding, by virtue of being a good representation of the data, and having structure. That is simply not the case with the high-dimensional models used in machine learning. I would be interested to see a counter example - a small model that fits a particular set of data well, but does not provide any explanatory power.

foobarqux 10y ago
Model fitting is only a useful analogy only in the very abstract sense of "is there something missing in my model". But usually people mean model fitting in the sense of choosing parameters in some restrictive model schema (i.e. selecting from a set of machine learning models), but those typically don't provide any meaningful understanding of the system, unless you apply a much weaker definition of "explanatory" or "understanding" than is typical. From my perspective, when you fit a linear regression I don't think you have any understanding of why the parameters are what they are, in the same way that I don't think a coder who tweaks constants in a buggy program until it works has any understanding why those constants are what they are. > I would be interested to see a counter example - a small model that fits a particular set of data well, but does not provide any explanatory power. As I said above, a linear regression typically doesn't lend itself to understanding. In fact, I would be interested in an example of a statistical model that does provide any meaningful explanation. Most ML and AI researchers don't even seem to pursue scientific understanding as a goal; predictive power is their measure of success. That is Chomsky's criticism.

by mrow84 10y ago

foobarqux 10y ago

Model fitting is only a useful analogy only in the very abstract sense of "is there something missing in my model". But usually people mean model fitting in the sense of choosing parameters in some restrictive model schema (i.e. selecting from a set of machine learning models), but those typically don't provide any meaningful understanding of the system, unless you apply a much weaker definition of "explanatory" or "understanding" than is typical. From my perspective, when you fit a linear regression I don't think you have any understanding of why the parameters are what they are, in the same way that I don't think a coder who tweaks constants in a buggy program until it works has any understanding why those constants are what they are. > I would be interested to see a counter example - a small model that fits a particular set of data well, but does not provide any explanatory power. As I said above, a linear regression typically doesn't lend itself to understanding. In fact, I would be interested in an example of a statistical model that does provide any meaningful explanation. Most ML and AI researchers don't even seem to pursue scientific understanding as a goal; predictive power is their measure of success. That is Chomsky's criticism.