4 ms·

The two main rationales are that systematic uncertainties are historically under-estimated and that we are looking in many channels (more than 1000), so it woul

The two main rationales are that systematic uncertainties are historically under-estimated and that we are looking in many channels (more than 1000), so it would not be too hard to find a 3 or 4 sigma anomaly. The second part is the so-called "Look Elsewhere Effect." If you hit 5 sigma, you are fairly safe from either of these effects ruining your "discovery."

_delirium 14y ago
> The second part is the so-called "Look Elsewhere Effect." Also more generally known as the "multiple testing" problem, fwiw (not sure why it has a different name in physics, unless I'm missing a subtlety). It's a major problem in "big data" also, where people just data-dredge thousands of possible parameter choices and pairwise correlations, and then report the p<0.01 results that came up, even though you'd expect several false positives just by chance with that methodology.