3 ms·

One issue with this: calling the likelihood P(e|h) 'objective' ignores the fact that the choice of model is usually quite subjective, and often affects the post

by davmre 12y ago

One issue with this: calling the likelihood P(e|h) 'objective' ignores the fact that the choice of model is usually quite subjective, and often affects the posterior to a much larger extent than the choice of prior, especially if you're working with a lot of data.

lisper 12y ago
> the choice of model is usually quite subjective Actually it's not. That is one of the most remarkable things about the universe we live in: it's really hard to come up with compact models that are consistent with all the data. So you can eliminate the vast majority of prospective models by eliminating those that 1) aren't consistent with the data or 2) don't compress the data efficiently. What you are left with is invariably a very small number of potential models. The bigger problem is people's unwillingness to let go of their cognitive biases. For example, economists still (AFAIK -- I don't actually read the economics literature) make their careers publishing papers based on the efficient market hypothesis despite the fact that there is overwhelming evidence that markets are not efficient. There's no getting around the GIGO principle.
- davmre 12y ago
  There are domains, like particle physics, where models are quite strongly constrained by data. And then there are domains, like climate science, economics, political forecasting, genetics, sociology, etc., where the underlying phenomena are very complex, data are noisy, and quite a wide range of models can be made to fit the available data. The latter sort of situation is becoming more and more common, as machine learning and statistical modeling are increasingly being applied to human behavior and other such messy areas that fall well outside the realm of 'clean' physical laws. So I submit that subjective model choice is, in practice, a very important part of a statistician / scientist's art. (as is understanding the nonstatistical reasons why one might choose a 'worse' model over a better model, e.g., computational tractability).
  - lisper 12y ago
    "Subjective" means "based on or influenced by personal feelings, tastes, or opinions." Whether or not the choice of theories is subjective on this definition depends on whether or not you care about the truth. Consider: Proposition P: Anthropogenic climate change is a myth. Suppose I subjectively choose a Bayesian prior for proposition P to be very close to 1. I can turn the crank on that and come up with models that are consistent with all the available data. Of course, this will lead me to conclusions that some would consider bizarre ("conspiracy theory" is the pejorative term) but because my feelings, tastes and opinions are just as valid as anyone else's (qua opinions) there is no subjective basis for preferring or rejecting my conclusions. But, of course, there is an objective basis for rejecting my theory because it has more explanatory loose ends than a theory that doesn't have to invoke conspiracies. Perhaps you want to use subjectivity to choose among theories which are objectively "equivalent" in some sense, i.e. similar explanatory power, agreement with the data, etc. There's nothing preventing you from doing that, but what would be the point? The best you can say objectively is that there is no basis for preferring one theory over another and there's more work left to be done (c.f. string theory). You could pick one theory over another because you think it's "prettier" but so what? If there was anything actually to be gained then (again, by definition) you could have made the selection objectively, not subjectively.

by davmre 12y ago

lisper 12y ago

> the choice of model is usually quite subjective Actually it's not. That is one of the most remarkable things about the universe we live in: it's really hard to come up with compact models that are consistent with all the data. So you can eliminate the vast majority of prospective models by eliminating those that 1) aren't consistent with the data or 2) don't compress the data efficiently. What you are left with is invariably a very small number of potential models. The bigger problem is people's unwillingness to let go of their cognitive biases. For example, economists still (AFAIK -- I don't actually read the economics literature) make their careers publishing papers based on the efficient market hypothesis despite the fact that there is overwhelming evidence that markets are not efficient. There's no getting around the GIGO principle.

davmre 12y ago
There are domains, like particle physics, where models are quite strongly constrained by data. And then there are domains, like climate science, economics, political forecasting, genetics, sociology, etc., where the underlying phenomena are very complex, data are noisy, and quite a wide range of models can be made to fit the available data. The latter sort of situation is becoming more and more common, as machine learning and statistical modeling are increasingly being applied to human behavior and other such messy areas that fall well outside the realm of 'clean' physical laws. So I submit that subjective model choice is, in practice, a very important part of a statistician / scientist's art. (as is understanding the nonstatistical reasons why one might choose a 'worse' model over a better model, e.g., computational tractability).
- lisper 12y ago
  "Subjective" means "based on or influenced by personal feelings, tastes, or opinions." Whether or not the choice of theories is subjective on this definition depends on whether or not you care about the truth. Consider: Proposition P: Anthropogenic climate change is a myth. Suppose I subjectively choose a Bayesian prior for proposition P to be very close to 1. I can turn the crank on that and come up with models that are consistent with all the available data. Of course, this will lead me to conclusions that some would consider bizarre ("conspiracy theory" is the pejorative term) but because my feelings, tastes and opinions are just as valid as anyone else's (qua opinions) there is no subjective basis for preferring or rejecting my conclusions. But, of course, there is an objective basis for rejecting my theory because it has more explanatory loose ends than a theory that doesn't have to invoke conspiracies. Perhaps you want to use subjectivity to choose among theories which are objectively "equivalent" in some sense, i.e. similar explanatory power, agreement with the data, etc. There's nothing preventing you from doing that, but what would be the point? The best you can say objectively is that there is no basis for preferring one theory over another and there's more work left to be done (c.f. string theory). You could pick one theory over another because you think it's "prettier" but so what? If there was anything actually to be gained then (again, by definition) you could have made the selection objectively, not subjectively.