5 ms·

The comment insinuates that the study did not control for age. The study controlled for age. It's not terrible to not make a good point, but it's not a good po

The comment insinuates that the study did not control for age. The study controlled for age. It's not terrible to not make a good point, but it's not a good point. The not good points should not be ranked highly.

roenxi 2mo ago
But after the study controls for age the starkness of the effect more or less disappears (eyeballing Fig 1, the 2nd graph). It turns out that there is a profession with a lowest risk of getting Alzheimer's (duh, one had to exist) and it looks like a slight outlier (it's the lowest, we'd expect that one to look like a slight outlier). As outliers go it isn't that impressive, it looks like it is part of a fairly boring distribution. We just plotted 400 data points on the graph, the 1/400 chance weird data point is going to look a little weird. We're actually just reading a lot into the fact that the 2nd data point in ambulance drivers which isn't all that convincing. Not to knock the study, I'm sure it is a great study. Lots of science done. They do good graphs and all that. But the study appears to be telling me that Alzheimer's is random with respect to occupation and ideally don't spend so much time on the road if you want to live to see old age.
- RIMR 2mo ago
  > But after the study controls for age the starkness of the effect more or less disappears (eyeballing Fig 1, the 2nd graph). Adjusting for age, Alzheimers deaths were 1/100 for taxi drivers, and 1/60 for the general public. That's extremely significant. However you feel about the graphs means nothing compared to the actual numbers presented here. > I'm sure it is a great study. Lots of science done. They do good graphs and all that. Your choice to be dismissively anti-science helps to explain the bad point you're making.
  - bluGill 2mo ago
    > That's extremely significant. What makes you think that is significant at all? Anytime you take a lot of small subsets of a large random sample you expect to find outliers that look a lot like this. That is https://xkcd.com/882/ https://xkcd.com/882/ seems to better fix the data just as well.
    - fc417fc802 2mo ago
      Outliers, sure. Exactly like this though? That depends. Since this is a population then assuming a gaussian (which could well be wrong for a variety of reasons I'm just guessing here) what's sigma? If 1/100 is within ~3 sigma then sure. Otherwise no.
      - bluGill 2mo ago
        It needs more statistical analysis than I have the background to do. However all signs look to another variation of the replication crisis, which is more an more my default assumption when I see an observational study.
        fc417fc802 2mo ago
        You're handwaving. The xkcd point only applies if you're within ~3 sigma. This isn't a complicated statistical analysis it's just asking how far out the outlier is versus the standard deviation and size of the dataset. To be clear I'm not claiming in one direction or the other. I feel that if you're going to attempt to discredit a piece of work it's on you to do such a basic check. Really the whole point of that xkcd was to viscerally illustrate the practical effect of binary misclassification, or alternatively to make the point that a p of 0.05 isn't necessarily as rigorous as you might expect. Neither of those things directly applies here.
        bluGill 2mo ago
        The article doesn't give enough information to answer your question. It isn't on me it is on the reporter's who shouldn't allow a handwave like that to pass without question. There are far to many cases of people hacking statistics in the world to accept that.
      - roenxi 2mo ago
        It is an outcome from over 400 data points. We're expecting to see events outside a 3 sigma boundary. In fact, 3 sigma is basically the 1:400 boundary. Strictly it is 1 in 370 [0] for anyone who takes Wikipedia at face value. In the study they seem put the P value of the taxi driver outcome at <0.01 (in Table 2, although I admit I haven't read the thing especially closely). For 400 data points, we expect there to be around 4 values with a P value less than <0.01. [0] https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule#Table_of_numerical_values https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rul...
  - dekhn 2mo ago
    Comparing 1/100 and 1/60 doesn't mean anything in terms of statistical significance (you would need to know more). They aren't being anti-science and it's low-quality to suggest they are.
nylonstrung 2mo ago
It's getting hard to find any study posted without a kneejerk "correlation does not equal causation" take as if that's not something researchers have to control for in order to even conduct the study in the first place, let alone publish
- mapt 2mo ago
  The institutional media departments, journalists, and headline writers add back in the insinuated causation and effect size as a routine matter. That way we can't find fault with the researchers for their clickbait study.
- jmull 2mo ago
  > kneejerk "correlation does not equal causation" The article (and apparently the researchers too) advances the idea there's a causal link here, even though one is not shown by the research. They go so far as to generally prescribe spatial reasoning training as a way to reduce Alzheimers risk. You might be tired of hearing "correlation does not equal causation", but it needs to be said here.
  - Forgeties79 2mo ago
    On hackernews I really don't think it needs to be said. Call me arrogant/too generous to this community, but I think most of the users here know the phrase and its meaning. Seeing 8000 variants of it doesn't contribute, and since it costs nothing to say "I bet they made a mistake" while elevating your own intellect people default to it far too much.
    - jmull 2mo ago
      Read the comment I replied to. It claims the researches must have somehow controlled for correlation (whatever that’s supposed to mean). Which they haven’t. There are a bunch of other comments arguing there’s something beyond correlation shown here. But it hasn’t. The study just demonstrates correlation. The misunderstanding here is widespread so it worth calling out. It’s actually kind of funny that people are more sick of the mistake being pointed out over and over than of the mistake being made over and over.

roenxi 2mo ago

But after the study controls for age the starkness of the effect more or less disappears (eyeballing Fig 1, the 2nd graph). It turns out that there is a profession with a lowest risk of getting Alzheimer's (duh, one had to exist) and it looks like a slight outlier (it's the lowest, we'd expect that one to look like a slight outlier). As outliers go it isn't that impressive, it looks like it is part of a fairly boring distribution. We just plotted 400 data points on the graph, the 1/400 chance weird data point is going to look a little weird. We're actually just reading a lot into the fact that the 2nd data point in ambulance drivers which isn't all that convincing. Not to knock the study, I'm sure it is a great study. Lots of science done. They do good graphs and all that. But the study appears to be telling me that Alzheimer's is random with respect to occupation and ideally don't spend so much time on the road if you want to live to see old age.

RIMR 2mo ago
> But after the study controls for age the starkness of the effect more or less disappears (eyeballing Fig 1, the 2nd graph). Adjusting for age, Alzheimers deaths were 1/100 for taxi drivers, and 1/60 for the general public. That's extremely significant. However you feel about the graphs means nothing compared to the actual numbers presented here. > I'm sure it is a great study. Lots of science done. They do good graphs and all that. Your choice to be dismissively anti-science helps to explain the bad point you're making.
- bluGill 2mo ago
  > That's extremely significant. What makes you think that is significant at all? Anytime you take a lot of small subsets of a large random sample you expect to find outliers that look a lot like this. That is https://xkcd.com/882/ https://xkcd.com/882/ seems to better fix the data just as well.
  - fc417fc802 2mo ago
    Outliers, sure. Exactly like this though? That depends. Since this is a population then assuming a gaussian (which could well be wrong for a variety of reasons I'm just guessing here) what's sigma? If 1/100 is within ~3 sigma then sure. Otherwise no.
    - bluGill 2mo ago
      It needs more statistical analysis than I have the background to do. However all signs look to another variation of the replication crisis, which is more an more my default assumption when I see an observational study.
      - fc417fc802 2mo ago
        You're handwaving. The xkcd point only applies if you're within ~3 sigma. This isn't a complicated statistical analysis it's just asking how far out the outlier is versus the standard deviation and size of the dataset. To be clear I'm not claiming in one direction or the other. I feel that if you're going to attempt to discredit a piece of work it's on you to do such a basic check. Really the whole point of that xkcd was to viscerally illustrate the practical effect of binary misclassification, or alternatively to make the point that a p of 0.05 isn't necessarily as rigorous as you might expect. Neither of those things directly applies here.
        bluGill 2mo ago
        The article doesn't give enough information to answer your question. It isn't on me it is on the reporter's who shouldn't allow a handwave like that to pass without question. There are far to many cases of people hacking statistics in the world to accept that.
    - roenxi 2mo ago
      It is an outcome from over 400 data points. We're expecting to see events outside a 3 sigma boundary. In fact, 3 sigma is basically the 1:400 boundary. Strictly it is 1 in 370 [0] for anyone who takes Wikipedia at face value. In the study they seem put the P value of the taxi driver outcome at <0.01 (in Table 2, although I admit I haven't read the thing especially closely). For 400 data points, we expect there to be around 4 values with a P value less than <0.01. [0] https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule#Table_of_numerical_values https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rul...
- dekhn 2mo ago
  Comparing 1/100 and 1/60 doesn't mean anything in terms of statistical significance (you would need to know more). They aren't being anti-science and it's low-quality to suggest they are.

nylonstrung 2mo ago

It's getting hard to find any study posted without a kneejerk "correlation does not equal causation" take as if that's not something researchers have to control for in order to even conduct the study in the first place, let alone publish

mapt 2mo ago
The institutional media departments, journalists, and headline writers add back in the insinuated causation and effect size as a routine matter. That way we can't find fault with the researchers for their clickbait study.
jmull 2mo ago
> kneejerk "correlation does not equal causation" The article (and apparently the researchers too) advances the idea there's a causal link here, even though one is not shown by the research. They go so far as to generally prescribe spatial reasoning training as a way to reduce Alzheimers risk. You might be tired of hearing "correlation does not equal causation", but it needs to be said here.
- Forgeties79 2mo ago
  On hackernews I really don't think it needs to be said. Call me arrogant/too generous to this community, but I think most of the users here know the phrase and its meaning. Seeing 8000 variants of it doesn't contribute, and since it costs nothing to say "I bet they made a mistake" while elevating your own intellect people default to it far too much.
  - jmull 2mo ago
    Read the comment I replied to. It claims the researches must have somehow controlled for correlation (whatever that’s supposed to mean). Which they haven’t. There are a bunch of other comments arguing there’s something beyond correlation shown here. But it hasn’t. The study just demonstrates correlation. The misunderstanding here is widespread so it worth calling out. It’s actually kind of funny that people are more sick of the mistake being pointed out over and over than of the mistake being made over and over.