3 ms·
You figure out what the problem in the measurements might be and correct for that based on your hypothesis of what the problem might be -- then compare the new
by ezy 17y ago
You figure out what the problem in the measurements might be and correct for that based on your hypothesis of what the problem might be -- then compare the new results to the model predictions (and other data). The tacit assumption is that model should predict the data, if it doesn't either the data or the model is borked. If other data supports the model, then you're going to look at the data first -- especially if it's new data collected in a new way.
The satellite example is perfect, they didn't take the model output and add a factor based on that. No, they hypothesized that the satellite height was different and that was affecting the data, checked that, found ample evidence for it. Adjusted the data based on the height (not the model) and lo and behold, the data more closely matched the existing model. The likelihood that this kind of independent factor makes all the difference for a bad model is quite small.
Where the bias might come in is that one would have a tendency to only look for errors in data not fitting the model(s) or other data. But no one is suggesting that you readjust the data to match the model output -- what they may be saying is that if your data doesn't match the model -- you better be able to justify it.. and that starts with the data and proceeds to a new iteration of the model as you gain more confidence in the data.
- yummyfajitas 17y agoEverything you describe sounds fine, but in the aggregate it privileges the hypothesis. When data supports the model, it is accepted. When it contradicts the model, extra scrutiny is applied. In short, a higher standard of evidence is required when you disagree with the hypothesis. It's kind of like accepting p-values of 0.05 when you agree with the model, but 0.01 when you disagree. It creates a bias in favor of the model. When huge arbitrary amounts of new evidence are available (e.g., an experimental field), this won't matter. In fields without experiments (climate science, oceanography, economics), this effect can protect a bad hypothesis.