5 ms·

0.5% difference seems a little questionable to draw conclusions. Agreed.

UncleMeat 8y ago
Why? With a sufficient sample size you can demonstrate even low power effects. The magnitude of the difference does not prevent good scientific analysis at all.
- whatshisface 8y ago
  The discussion behind the discussion here is that nobody ever reports confidence intervals... When you don't have them you just have to guess what they are based on what's usual for the field. We're having this entire discussion here but unless the error bars show up it all ought to conclude nothing.
  - SamReidHughes 8y ago
    Knowing there are over 500k patients in the sample, you can calculate the error bars.
    - amelius 8y ago
      Also a p-value would be nice.
      - klodolph 8y ago
        Click on the link to the actual study, you will see P values right there in the abstract.
    - klodolph 8y ago
      Well, ideally you would know more than just the number of patients. The roughest estimate would be to compute the expected variance of a binomial distribution with N=120e3 and P=0.12, which gives a standard deviation of 0.00094, so a difference of 0.5% is well over 5 standard deviations out. Not quite good enough for particle physics, but you're damn right it's good enough for medicine. Now you just have to look at group sizes, possible confounding factors, etc...
      - deleted 8y ago
        [deleted]
  - cowsandmilk 8y ago
    > nobody ever reports confidence intervals But the article[0], in PNAS, has confidence intervals. [0] http://www.pnas.org/content/early/2018/07/31/1800097115 http://www.pnas.org/content/early/2018/07/31/1800097115
    - whatshisface 8y ago
      They don't report it in their abstract, which is all the majority of HN has access to. As a result any conclusions reached will be unfounded. (Unless someone gets the paper and tells us what it said, which is what the linked article's authors were supposed to be doing...) Edit: The thing is that in the abstract no form of statistical indicator was reported. That plus its absence from the linked article means that unless someone here can get access to the paper our discussion is in vain.
      - klodolph 8y ago
        As much as people like to talk about how P values are not useful and debate what values we should use, I don't think there's an especially compelling reason to choose confidence intervals over P values, or complain that one is appropriate and the other is not. They both measure the relationship between statistical likelihood and parameter values. P values fix the parameter values and measure the statistical likelihood, confidence intervals fix the statistical likelihood and measure the parameter values. So if you ask for confidence intervals and they give you a P-value, well you have a confidence interval it just won't be a nice round number like 95% or 99%. It's kind of like complaining that you ordered 1 kg of water but were delivered 1 l, and insisting that everyone measure water by weight rather than volume. Don't read too much into that analogy, but the relationship between P values and confidence intervals is of a broadly similar nature.
random3 8y ago
0.5% is not irrelevant when you talk about deaths. Statistically if you consider the sample size and significance (P = 0.001) it actually looks very non questionable. https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2593255 https://jamanetwork.com/journals/jamainternalmedicine/fullar...

Why? With a sufficient sample size you can demonstrate even low power effects. The magnitude of the difference does not prevent good scientific analysis at all.

whatshisface 8y ago
The discussion behind the discussion here is that nobody ever reports confidence intervals... When you don't have them you just have to guess what they are based on what's usual for the field. We're having this entire discussion here but unless the error bars show up it all ought to conclude nothing.
- SamReidHughes 8y ago
  Knowing there are over 500k patients in the sample, you can calculate the error bars.
  - amelius 8y ago
    Also a p-value would be nice.
    - klodolph 8y ago
      Click on the link to the actual study, you will see P values right there in the abstract.
  - klodolph 8y ago
    Well, ideally you would know more than just the number of patients. The roughest estimate would be to compute the expected variance of a binomial distribution with N=120e3 and P=0.12, which gives a standard deviation of 0.00094, so a difference of 0.5% is well over 5 standard deviations out. Not quite good enough for particle physics, but you're damn right it's good enough for medicine. Now you just have to look at group sizes, possible confounding factors, etc...
    - deleted 8y ago
      [deleted]
- cowsandmilk 8y ago
  > nobody ever reports confidence intervals But the article[0], in PNAS, has confidence intervals. [0] http://www.pnas.org/content/early/2018/07/31/1800097115 http://www.pnas.org/content/early/2018/07/31/1800097115
  - whatshisface 8y ago
    They don't report it in their abstract, which is all the majority of HN has access to. As a result any conclusions reached will be unfounded. (Unless someone gets the paper and tells us what it said, which is what the linked article's authors were supposed to be doing...) Edit: The thing is that in the abstract no form of statistical indicator was reported. That plus its absence from the linked article means that unless someone here can get access to the paper our discussion is in vain.
    - klodolph 8y ago
      As much as people like to talk about how P values are not useful and debate what values we should use, I don't think there's an especially compelling reason to choose confidence intervals over P values, or complain that one is appropriate and the other is not. They both measure the relationship between statistical likelihood and parameter values. P values fix the parameter values and measure the statistical likelihood, confidence intervals fix the statistical likelihood and measure the parameter values. So if you ask for confidence intervals and they give you a P-value, well you have a confidence interval it just won't be a nice round number like 95% or 99%. It's kind of like complaining that you ordered 1 kg of water but were delivered 1 l, and insisting that everyone measure water by weight rather than volume. Don't read too much into that analogy, but the relationship between P values and confidence intervals is of a broadly similar nature.

random3 8y ago

0.5% is not irrelevant when you talk about deaths. Statistically if you consider the sample size and significance (P = 0.001) it actually looks very non questionable. https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2593255 https://jamanetwork.com/journals/jamainternalmedicine/fullar...