3 ms·

> They found that out of 100,000 non-drinkers, 914 would develop an alcohol-related health problem such as cancer or suffer an injury. > But an extra four peopl

by 6nf 5y ago

> They found that out of 100,000 non-drinkers, 914 would develop an alcohol-related health problem such as cancer or suffer an injury. > But an extra four people would be affected if they drank one alcoholic drink a day. So 914 vs 918 out of 100,000? That's gotta be below statistical significance.

inglor_cz 5y ago
Journalists are often not great at understanding statistics. That is how we got so many misleading news. 914 to 918 per 100,000 is a tiny change. It might be caused by subtle differences in the two groups that have nothing to do with alcohol. If anything, the study shows that having <= 1 drink a day is basically the same as being a teetotaler. Two drinks a day, OTOH, are visibly worse for health (914 to 977).
- kqr 5y ago
  Just saying it's a tiny change because "it feels that way" doesn't make it so. When ranting against misleading news, I think your approach of argument from assertiveness is slightly dishonest. A bit more rigorous would be to compare maybe the Beta distribution with a=915 to a=919 and b=100,000 in both cases. Or Poisson distributions with lambda 914/100,000 and 918/100,000. I don't know whether Beta or Poisson are better fits. I don't even know how I'd find out. Abd of course, that's still a fairly handwavy approach and I'd love for someone to show me how it's done for real. But just saying it's noise without quantifying it seems to me exactly as misleading and irresponsible as the original news, only biased in the other direction.
  - inglor_cz 5y ago
    In case of humans and health, the biggest variable is age. Biological aging is strongly connected to higher risk of serious diseases. (See Makeham-Gompertz law; this is the Gompertz component.) If you have groups of 100 000 people, but there are just a few slightly older people in the other group, the incidence of age-related diseases is expected to be higher. Then there are subtle things such as "how many people in group A vs group B live next to a motorway or suffer from higher levels of ambient noise". Not to mention the cultural differences (religious fasting or absence thereof etc.) Picking up huge groups of people that are perfectly equivalent is very, very hard. But I am not a professional either. I studied algebra, not statistics, we only had a year-long course in the basics. Maybe I was too handwavy.
- B1FF_PSUVM 5y ago
  > Journalists are often not great at understanding statistics. Perhaps, as comrade Upton Sinclair put it, "It is difficult to get a man to understand something, when his salary depends on his not understanding it."
kqr 5y ago
You're probably not, but the way you phrased this suggests you might be confusing statistical significance with effect size. You can reject a null hypothesis with statistical significance even if the effect size of the alternate hypothesis is small -- you just need a big sample.
- ascar 5y ago
  Using a simple t-test calculation and assuming normal distribution and an equal split in both groups, the sample size per group needs to be 20 million to be in the 90% confidence interval. It's not even yet in the 95% interval at that point. You reach the "scientific default" 95% confidence at 40million per group (that's 80million total). At a sample size of 20million that's 182.800 cases in the non-alcoholic group and 183.600 in the alcoholic once a day group. At that scale I wouldn't trust my data enough to believe the reporting of once-a-day and not-at-all is actually accurate and other completly unrelated unaccounted effects do not outweigh my testesd criteria. If I haven't done any major errors calculating this (which might be, because it's back of the napkin math with a t-test calculator or my assumptions are wrong), I doubt the sample size and accuracy of the measure is high enough to make a statistical significant claim about these groups. Even if it actually is a somewhat accurate result, it means having one drink per day increases health issues by as little as 0.5%. The risk of addicition and starting to drink much more probably heavily outweighs that. Regarding OPs statement, I think it's fair to assume what is usually meant is "that has to be statistically insignificant for any sample size this sutdy probably had".
dagw 5y ago
Depends what you are trying to show. What it does show is that they didn't find any health benefits from drinking a glass a day, which was something lots of other studies where purporting to find around that time. The actual conclusion from the actual paper is "the level of consumption that minimises health loss is zero"

by 6nf 5y ago

inglor_cz 5y ago

Journalists are often not great at understanding statistics. That is how we got so many misleading news. 914 to 918 per 100,000 is a tiny change. It might be caused by subtle differences in the two groups that have nothing to do with alcohol. If anything, the study shows that having <= 1 drink a day is basically the same as being a teetotaler. Two drinks a day, OTOH, are visibly worse for health (914 to 977).

kqr 5y ago
Just saying it's a tiny change because "it feels that way" doesn't make it so. When ranting against misleading news, I think your approach of argument from assertiveness is slightly dishonest. A bit more rigorous would be to compare maybe the Beta distribution with a=915 to a=919 and b=100,000 in both cases. Or Poisson distributions with lambda 914/100,000 and 918/100,000. I don't know whether Beta or Poisson are better fits. I don't even know how I'd find out. Abd of course, that's still a fairly handwavy approach and I'd love for someone to show me how it's done for real. But just saying it's noise without quantifying it seems to me exactly as misleading and irresponsible as the original news, only biased in the other direction.
- inglor_cz 5y ago
  In case of humans and health, the biggest variable is age. Biological aging is strongly connected to higher risk of serious diseases. (See Makeham-Gompertz law; this is the Gompertz component.) If you have groups of 100 000 people, but there are just a few slightly older people in the other group, the incidence of age-related diseases is expected to be higher. Then there are subtle things such as "how many people in group A vs group B live next to a motorway or suffer from higher levels of ambient noise". Not to mention the cultural differences (religious fasting or absence thereof etc.) Picking up huge groups of people that are perfectly equivalent is very, very hard. But I am not a professional either. I studied algebra, not statistics, we only had a year-long course in the basics. Maybe I was too handwavy.
B1FF_PSUVM 5y ago
> Journalists are often not great at understanding statistics. Perhaps, as comrade Upton Sinclair put it, "It is difficult to get a man to understand something, when his salary depends on his not understanding it."

kqr 5y ago

You're probably not, but the way you phrased this suggests you might be confusing statistical significance with effect size. You can reject a null hypothesis with statistical significance even if the effect size of the alternate hypothesis is small -- you just need a big sample.

ascar 5y ago
Using a simple t-test calculation and assuming normal distribution and an equal split in both groups, the sample size per group needs to be 20 million to be in the 90% confidence interval. It's not even yet in the 95% interval at that point. You reach the "scientific default" 95% confidence at 40million per group (that's 80million total). At a sample size of 20million that's 182.800 cases in the non-alcoholic group and 183.600 in the alcoholic once a day group. At that scale I wouldn't trust my data enough to believe the reporting of once-a-day and not-at-all is actually accurate and other completly unrelated unaccounted effects do not outweigh my testesd criteria. If I haven't done any major errors calculating this (which might be, because it's back of the napkin math with a t-test calculator or my assumptions are wrong), I doubt the sample size and accuracy of the measure is high enough to make a statistical significant claim about these groups. Even if it actually is a somewhat accurate result, it means having one drink per day increases health issues by as little as 0.5%. The risk of addicition and starting to drink much more probably heavily outweighs that. Regarding OPs statement, I think it's fair to assume what is usually meant is "that has to be statistically insignificant for any sample size this sutdy probably had".

dagw 5y ago

Depends what you are trying to show. What it does show is that they didn't find any health benefits from drinking a glass a day, which was something lots of other studies where purporting to find around that time. The actual conclusion from the actual paper is "the level of consumption that minimises health loss is zero"