3 ms·

Physics can collect so much data that the generally accepted standard for results is five sigma. There are biases in physics, but it's much harder to delude you

by Rylinks 10y ago

Physics can collect so much data that the generally accepted standard for results is five sigma. There are biases in physics, but it's much harder to delude yourself into a five sigma result than a two sigma one--in the end the right answer becomes obvious.

privong 10y ago
> Physics can collect so much data that the generally accepted standard for results is five sigma. There are biases in physics, but it's much harder to delude yourself into a five sigma result than a two sigma one--in the end the right answer becomes obvious. That's in the case of clear-cut predictions from theory or empirical relations. But bias generally arises in a more subtle fashion than statistics on measurements, though. Even if you have a "5-sigma result", it still needs to be interpreted, and that's where biases can rear their heads. On possible manifestation (though not the only), is if one ascribes to a particular theory, one is generally biased into interpreting those results within the framework of that theory. And if one isn't careful, that can lead to diminishing the weight of evidence that conflicts with the theory (e.g., "it's 5-sigma, but those data aren't actually relevant to this issue"). As an example, one area I'm actively researching is the emission from molecules in galaxies. We see that some galaxies show enhancements in emission from a specific molecule (HCN). Everyone agrees that the enhancements exist but there's wide disagreement on what causes it. And much of the discussion around it parallels people's opinions/biases about which is more important for a galaxy: how many new stars it is forming versus how active the supermassive black hole is at the galaxy's center. The preferred interpretations for the extra HCN emission seems to correlate very well with people's prior notions* (which is not surprising). Of course, more data will eventually provide a concrete solution to this question, though what new data are collected is often chosen within an existing framework. * - I'm sure it's true for me, even though I try to mitigate it as much as possible. But we often have trouble recognizing our own biases.
- Rylinks 10y ago
  >That's in the case of clear-cut predictions from theory or empirical relations. But bias generally arises in a more subtle fashion than statistics on measurements, though. Even if you have a "5-sigma result", it still needs to be interpreted, and that's where biases can rear their heads. I agree, interpretation is the tricky thing. The difference is that in social sciences, both the result and the interpretation are suspect.

by Rylinks 10y ago

privong 10y ago

> Physics can collect so much data that the generally accepted standard for results is five sigma. There are biases in physics, but it's much harder to delude yourself into a five sigma result than a two sigma one--in the end the right answer becomes obvious. That's in the case of clear-cut predictions from theory or empirical relations. But bias generally arises in a more subtle fashion than statistics on measurements, though. Even if you have a "5-sigma result", it still needs to be interpreted, and that's where biases can rear their heads. On possible manifestation (though not the only), is if one ascribes to a particular theory, one is generally biased into interpreting those results within the framework of that theory. And if one isn't careful, that can lead to diminishing the weight of evidence that conflicts with the theory (e.g., "it's 5-sigma, but those data aren't actually relevant to this issue"). As an example, one area I'm actively researching is the emission from molecules in galaxies. We see that some galaxies show enhancements in emission from a specific molecule (HCN). Everyone agrees that the enhancements exist but there's wide disagreement on what causes it. And much of the discussion around it parallels people's opinions/biases about which is more important for a galaxy: how many new stars it is forming versus how active the supermassive black hole is at the galaxy's center. The preferred interpretations for the extra HCN emission seems to correlate very well with people's prior notions* (which is not surprising). Of course, more data will eventually provide a concrete solution to this question, though what new data are collected is often chosen within an existing framework. * - I'm sure it's true for me, even though I try to mitigate it as much as possible. But we often have trouble recognizing our own biases.

Rylinks 10y ago
>That's in the case of clear-cut predictions from theory or empirical relations. But bias generally arises in a more subtle fashion than statistics on measurements, though. Even if you have a "5-sigma result", it still needs to be interpreted, and that's where biases can rear their heads. I agree, interpretation is the tricky thing. The difference is that in social sciences, both the result and the interpretation are suspect.