4 ms·

> 7% higher risk in white women (HR = 1.07, 95% CI: 0.99–1.16) The confidence interval clearly includes 1.0. I'm tired of all these studies being uncritically

> 7% higher risk in white women (HR = 1.07, 95% CI: 0.99–1.16) The confidence interval clearly includes 1.0. I'm tired of all these studies being uncritically reported on in the press with no mention of the giant error bars. IMO medical researchers need to raise their standards. 95% is way too low. Also, journalists should be ashamed at the state of science and medical reporting. It's ironic that you can read articles blaming tech for the spread of fake news and then turn the page to the health section and read garbage like this article.

icegreentea2 7y ago
Won't argue about the validity 95% confidence interval here, but the 95% CI of HR of women as a whole excludes 1, and for African Americans clearly also excludes one. The overall conclusion is still sound (if you accept the validity of 95% CI as a measurement).
- modeless 7y ago
  Only for dye, not for straighteners, and only for "Nonprofessional application of semipermanent dye to others", and black women. If you count up the specific claims made in the article, a majority are not actually supported by the study. And unless the study was preregistered for the cohorts they analyzed, the statistical analysis is suspect even at the 95% level which I still contend is too low.
  - gus_massa 7y ago
    It's interesting to see the supplementary tables (that are free). In the supplementary table 1, classified by menopausal status, they report 55 results. The intervals that exclude 1.00 are: 1.30 (1.04, 1.62) 1.35 (1.08, 1.69) 1.41 (1.10, 1.79) 1.26 (1.02, 1.55) 1.51 (1.17, 1.94) 0.68 (0.47, 0.99) (Some are very correlated.) Anyway, they get 6 interesting results, slightly more than the expected number of false positive cases with a 95% CI interval with 55 cases. Moreover, in the last case, 1.00 is excluded because the interval is too low, i.e. in this study in the premenopausal women that applied hair dye to other person between 1 and 3 times a year got less cancer that the women that didn't. This is probably just noise, but note that this case is not highlighted in the abstract in spite they highlight a few cases where the 95% interval includes 1.00. In the supplementary table 2, classified by estrogen reception, they report also 55 results. The intervals that exclude 1.00 are: 0.78 (0.63, 0.97) 0.82 (0.70,0.95) Both are below 1.00, and neither is highlighted in the abstract. This number is close to the expected number of false positive cases with a 95% CI interval with 55 cases. I guess there is another table in the paper where they classify the women black or white, and perhaps even more classifications. So the evidence looks weak and close to the noise level.
    - Knufen 7y ago
      Always remember, all articles are accessible through the Hub of Science Edit: Forgot e
- xiphias2 7y ago
  You can't reuse the same data to get 95% CI for all women and for white woman by deducing that all women are excluded but white women included. They have to calculate a confidence for the event that white women are affected AND all women are not.
rand_r 7y ago
Here’s a question I had when I learned this stuff in stats class: What does a 95% confidence interval actually mean? I could never get a clear answer.
- hombre_fatal 7y ago
  It's the age of information. Putting "95% confidence interval" into Google gives back all sorts of examples. What part of it are you having trouble with? Here's a pick at random: https://www.youtube.com/watch?v=mizNAmeAT5w https://www.youtube.com/watch?v=mizNAmeAT5w ("What Does a 95% Confidence Interval Mean?")
- bonoboTP 7y ago
  Not sure if the answer will be clear, but that's inherent in the weird and counter-intuitive nature of the concept itself (which I consider as a major disadvantage of it). It is an interval that was obtained using a confidence interval generation procedure, for which the following is true: for any true value of the quantity you're estimating, the procedure yields an interval that contains that correct value 95% of the times, when run on different random samples collected conditioned on that true value. You want to estimate some quantity, let's say the cancer risk ratio of hair-dyers to non-hair dyers. You make some (noisy) experimental measurements on people. Then, let's say someone hands you a confidence interval generating procedure. You feed it with your measurement data and it spits out an interval. To claim that this interval is a 95% confidence interval is actually a claim about the confidence interval generating procedure, not about the particular interval. And that claim is that for any true cancer risk ratio value, if we ran lots of experiments and fed all the results to this procedure, then 95% of the resulting intervals would contain the correct corresponding value. So in more detail: Assume the true ratio is, say, 1.05. Then if we repeated our noisy experiment in a 1.05-ratio world and applied the confidence interval generating procedure to the results, we would expect to get 95% of those intervals containing the number 1.05 and 5% of them not containing it. If we assume the ratio is 1.2, the same thing should hold. For this analysis to work, we need an idealized measurement model that tells us what the distribution of measurements looks like given any particular underlying value of the quantity of interest (say the cancer ratio). Then using this model and the description of a given confidence interval generating procedure, we may be able to prove mathematically that it indeed has the above property. Most of the time, statisticians use standards procedures, for which this property has been proved and the proof is widely known. They rarely invent new such procedures. So, in summary: we don't know what the true value is in reality. But we can prove that the confidence interval generating procedure has the above explained property. Then, we will call the interval that this procedure gives us on our real measurement data a 95% confidence interval. What it definitely does not mean is that the true value is in the interval with 95% probability, and that's the most common misunderstanding. I'm not sure if this helps at all. These things are really counter-intuitive and twisted against our natural way of thinking.

icegreentea2 7y ago

Won't argue about the validity 95% confidence interval here, but the 95% CI of HR of women as a whole excludes 1, and for African Americans clearly also excludes one. The overall conclusion is still sound (if you accept the validity of 95% CI as a measurement).

modeless 7y ago
Only for dye, not for straighteners, and only for "Nonprofessional application of semipermanent dye to others", and black women. If you count up the specific claims made in the article, a majority are not actually supported by the study. And unless the study was preregistered for the cohorts they analyzed, the statistical analysis is suspect even at the 95% level which I still contend is too low.
- gus_massa 7y ago
  It's interesting to see the supplementary tables (that are free). In the supplementary table 1, classified by menopausal status, they report 55 results. The intervals that exclude 1.00 are: 1.30 (1.04, 1.62) 1.35 (1.08, 1.69) 1.41 (1.10, 1.79) 1.26 (1.02, 1.55) 1.51 (1.17, 1.94) 0.68 (0.47, 0.99) (Some are very correlated.) Anyway, they get 6 interesting results, slightly more than the expected number of false positive cases with a 95% CI interval with 55 cases. Moreover, in the last case, 1.00 is excluded because the interval is too low, i.e. in this study in the premenopausal women that applied hair dye to other person between 1 and 3 times a year got less cancer that the women that didn't. This is probably just noise, but note that this case is not highlighted in the abstract in spite they highlight a few cases where the 95% interval includes 1.00. In the supplementary table 2, classified by estrogen reception, they report also 55 results. The intervals that exclude 1.00 are: 0.78 (0.63, 0.97) 0.82 (0.70,0.95) Both are below 1.00, and neither is highlighted in the abstract. This number is close to the expected number of false positive cases with a 95% CI interval with 55 cases. I guess there is another table in the paper where they classify the women black or white, and perhaps even more classifications. So the evidence looks weak and close to the noise level.
  - Knufen 7y ago
    Always remember, all articles are accessible through the Hub of Science Edit: Forgot e
xiphias2 7y ago
You can't reuse the same data to get 95% CI for all women and for white woman by deducing that all women are excluded but white women included. They have to calculate a confidence for the event that white women are affected AND all women are not.

rand_r 7y ago

Here’s a question I had when I learned this stuff in stats class: What does a 95% confidence interval actually mean? I could never get a clear answer.

hombre_fatal 7y ago
It's the age of information. Putting "95% confidence interval" into Google gives back all sorts of examples. What part of it are you having trouble with? Here's a pick at random: https://www.youtube.com/watch?v=mizNAmeAT5w https://www.youtube.com/watch?v=mizNAmeAT5w ("What Does a 95% Confidence Interval Mean?")
bonoboTP 7y ago
Not sure if the answer will be clear, but that's inherent in the weird and counter-intuitive nature of the concept itself (which I consider as a major disadvantage of it). It is an interval that was obtained using a confidence interval generation procedure, for which the following is true: for any true value of the quantity you're estimating, the procedure yields an interval that contains that correct value 95% of the times, when run on different random samples collected conditioned on that true value. You want to estimate some quantity, let's say the cancer risk ratio of hair-dyers to non-hair dyers. You make some (noisy) experimental measurements on people. Then, let's say someone hands you a confidence interval generating procedure. You feed it with your measurement data and it spits out an interval. To claim that this interval is a 95% confidence interval is actually a claim about the confidence interval generating procedure, not about the particular interval. And that claim is that for any true cancer risk ratio value, if we ran lots of experiments and fed all the results to this procedure, then 95% of the resulting intervals would contain the correct corresponding value. So in more detail: Assume the true ratio is, say, 1.05. Then if we repeated our noisy experiment in a 1.05-ratio world and applied the confidence interval generating procedure to the results, we would expect to get 95% of those intervals containing the number 1.05 and 5% of them not containing it. If we assume the ratio is 1.2, the same thing should hold. For this analysis to work, we need an idealized measurement model that tells us what the distribution of measurements looks like given any particular underlying value of the quantity of interest (say the cancer ratio). Then using this model and the description of a given confidence interval generating procedure, we may be able to prove mathematically that it indeed has the above property. Most of the time, statisticians use standards procedures, for which this property has been proved and the proof is widely known. They rarely invent new such procedures. So, in summary: we don't know what the true value is in reality. But we can prove that the confidence interval generating procedure has the above explained property. Then, we will call the interval that this procedure gives us on our real measurement data a 95% confidence interval. What it definitely does not mean is that the true value is in the interval with 95% probability, and that's the most common misunderstanding. I'm not sure if this helps at all. These things are really counter-intuitive and twisted against our natural way of thinking.