imh's explanation is good, but it's also worth considering a very common example of this. Consider two groups of people where one group engages in some activity

imh's explanation is good, but it's also worth considering a very common example of this. Consider two groups of people where one group engages in some activity and another does not. Measure a dozen things about those people. Then report only on the one that positively (or negatively) correlates with group membership as if it was the only thing you measured. This generates an unrealistically high statistical significance (unrealististically low p-value) because the question should not be, "What are the odds these two things are correlated by chance" but "What are the odds that any of of these things is correlated by chance with this other thing?" The significance of the result is far below the reported p-value from an analysis that assumes only one thing was investigated. You frequently see incorrect p-values of this kind even when the researchers acknowledge they tested for multiple things.