3 ms·

Potentially OT. Why do people write >"While suicide rates for females increased after the show’s release, the increase was not statistically significant." ins

Potentially OT. Why do people write >"While suicide rates for females increased after the show’s release, the increase was not statistically significant." instead of >"Suicide rates for females did not increase significantly after the show's release." ? I have seen this quite often and it always seems a bit misleading to me. Edit: I should have written the 2nd version as "No statistically significant increase in [...] was found". My point is not about the difference between "significance" and "statistical significance" but about mentioning an effect, while also saying the effect is not statistically significant. If it's not statistically significant, why mention it?

Matticus_Rex 7y ago
Significance and statistical significance are two different things, and it's good for them to be distinguished.
tikhonj 7y ago
This is a recurring issue because "significant" means very different things in different contexts. The first sentence reads as "rates increased, but in a way that is not distinguishable from noise", while the second reads as "rates increased, but not to an extent that matters".
- lazugod 7y ago
  The second reads as though there wasn’t even an insignificant increase, which is very different.
roywiggins 7y ago
They observed an increase, but the statistical test they used couldn't distinguish it from noise with sufficient confidence to call it "statistically significant." Statistical significance and a significant increase aren't the same. A statistically significant increase might be totally insignificant, and for a small enough study, a subjectively "significant" increase might be statistically insignificant.
clircle 7y ago
Because those statements are not equivalent.
gitpusher 7y ago
The former version is more specific and therefore accurate.
acchow 7y ago
These mean different things. You can have a 1% increase (which is not a "significant" increase) but with a large enough sample size you can be quite confident in it (statistically significant).
deleted 7y ago
[deleted]
deleted 7y ago
[deleted]
ISL 7y ago
The former is technically correct and much more descriptive. The latter is potentially-misleading. One might think that the rate did not increase at all, when in fact it did, or that it did not increase "a lot" (the colloquial understanding of significant). "Statistically-significant" is a keyword with technical meaning. The former could mean that the rate increased by 2±2 units. A more-precise study might show that the rate did increase, or it might not. All we know from the former is that the rate increased, but that the rate was not statistically-significant.
dbatten 7y ago
IMO, your proposed second option leaves ambiguity as to whether "significant" means "statistically significant" or "materially significant." The original option is clear on that point. However, I would argue that the original text is also misleading, because if the result wasn't statistically significant, there's no evidence that rates for females increased. If anything, the first option could perhaps say "While suicide rates for females IN THE STUDY increased after the show's release, the increase was not statistically significant, so we have no evidence that it's a real effect."
- Majromax 7y ago
  > if the result wasn't statistically significant, there's no evidence that rates for females increased. That's not quite right. A conclusion of no statistical significance is not a conclusion that there is no evidence to support the hypothesis, but that there is insufficient evidence. The study failed to disprove the null hypothesis, but "failure to disprove" does not prove it to be true either. A more powerful study (with a larger sample size, or one that includes this incident in a meta-study of similar media events) might be able to use this data to support a positive conclusion.
ec109685 7y ago
Why not just say: Suicide rates for females did not change in a statistically significant way.
froindt 7y ago
I think it's to combat the colloquial definition of "significant" with the specific statistical definition. All polls and samplings have an amount of error (e.g. Biden leading in the polls at 30%, 3% margin of error). If Biden polls at 32% next week, there was an increase. Colloquially some would say that's a significant increase (after all, his pie grew by 6% compared to the prior week!), but it's very likely to do with chance. Unless you survey the entire population, you don't know the true proportion in which something occurs. A change from 30% to 35% is quite unlikely to be random chance.

Matticus_Rex 7y ago

Significance and statistical significance are two different things, and it's good for them to be distinguished.

tikhonj 7y ago

This is a recurring issue because "significant" means very different things in different contexts. The first sentence reads as "rates increased, but in a way that is not distinguishable from noise", while the second reads as "rates increased, but not to an extent that matters".

lazugod 7y ago
The second reads as though there wasn’t even an insignificant increase, which is very different.

roywiggins 7y ago

They observed an increase, but the statistical test they used couldn't distinguish it from noise with sufficient confidence to call it "statistically significant." Statistical significance and a significant increase aren't the same. A statistically significant increase might be totally insignificant, and for a small enough study, a subjectively "significant" increase might be statistically insignificant.

clircle 7y ago

Because those statements are not equivalent.

gitpusher 7y ago

The former version is more specific and therefore accurate.

acchow 7y ago

These mean different things. You can have a 1% increase (which is not a "significant" increase) but with a large enough sample size you can be quite confident in it (statistically significant).

deleted 7y ago

[deleted]

deleted 7y ago

[deleted]

ISL 7y ago

The former is technically correct and much more descriptive. The latter is potentially-misleading. One might think that the rate did not increase at all, when in fact it did, or that it did not increase "a lot" (the colloquial understanding of significant). "Statistically-significant" is a keyword with technical meaning. The former could mean that the rate increased by 2±2 units. A more-precise study might show that the rate did increase, or it might not. All we know from the former is that the rate increased, but that the rate was not statistically-significant.

dbatten 7y ago

IMO, your proposed second option leaves ambiguity as to whether "significant" means "statistically significant" or "materially significant." The original option is clear on that point. However, I would argue that the original text is also misleading, because if the result wasn't statistically significant, there's no evidence that rates for females increased. If anything, the first option could perhaps say "While suicide rates for females IN THE STUDY increased after the show's release, the increase was not statistically significant, so we have no evidence that it's a real effect."

Majromax 7y ago
> if the result wasn't statistically significant, there's no evidence that rates for females increased. That's not quite right. A conclusion of no statistical significance is not a conclusion that there is no evidence to support the hypothesis, but that there is insufficient evidence. The study failed to disprove the null hypothesis, but "failure to disprove" does not prove it to be true either. A more powerful study (with a larger sample size, or one that includes this incident in a meta-study of similar media events) might be able to use this data to support a positive conclusion.

ec109685 7y ago

Why not just say: Suicide rates for females did not change in a statistically significant way.

froindt 7y ago

I think it's to combat the colloquial definition of "significant" with the specific statistical definition. All polls and samplings have an amount of error (e.g. Biden leading in the polls at 30%, 3% margin of error). If Biden polls at 32% next week, there was an increase. Colloquially some would say that's a significant increase (after all, his pie grew by 6% compared to the prior week!), but it's very likely to do with chance. Unless you survey the entire population, you don't know the true proportion in which something occurs. A change from 30% to 35% is quite unlikely to be random chance.