6 ms·

This is a fascinating share, if not for the results—"None of the measures of intellectual engagement was linked to the speed of decline of people's mental abili

This is a fascinating share, if not for the results—"None of the measures of intellectual engagement was linked to the speed of decline of people's mental abilities over time"—but for the study itself. "Many people also dropped out of the study. Only 96 of the 498 people recruited took part in the last round of testing. The researchers lost touch with 13 people, while 57 people died and 332 declined to take part." Do studies like this often reach the front page of HN?

amelius 8y ago
You can ask the same question about the NHS.
- jodrellblank 8y ago
  You mean "Does the NHS often reach the front page of HN?"? .. I don't get it, why would you ask that?
  - amelius 8y ago
    Comment was about the research reaching the news section of the NHS.
    - arctangent 8y ago
      www.nhs.uk/news exists specifically to debunk false health claims (I'm tech lead for this site)
      - BossHogg 8y ago
        Thought this was a great site. Then I looked for the rss feed. Still good but...
        arctangent 8y ago
        We syndicate our content in a different way: https://developer.api.nhs.uk/ https://developer.api.nhs.uk/
joe_the_user 8y ago
Wow, a rate of drop-out like that would certainly make me doubtful of the results of the study. It seems reasonably likely that people dropped-out in a fashion that would be weighted in one way or another.
- make3 8y ago
  for sure.. but I expect the real results to be worse (though this is purely speculative on my part), as I can only imagine people with steeper intellectual decline being more likely to drop out
  - ojbyrne 8y ago
    Its mentioned in the paper: "However, recruitment to and retention in cognitive ageing studies is also affected by self selection to volunteer among better educated individuals who are ageing well, which is also true of our study. Thus, the findings might be biased towards those individuals with higher cognitive performance."
    - mirimir 8y ago
      Thanks for finding that. I was about to read the paper, looking for something like that.
  - RyanOD 8y ago
    I immediately thought the same. Then I thought about for a bit longer and quickly came up with an equal number of reasons why participants experiencing more rapid mental decline might stick it out. Really no way to know.
    - gweinberg 8y ago
      Maybe, but given the huge dropout rate I don't see how we can conclude anything from this study. Of course, that would also be true if it had reached the opposite conclusion.
  - 8y ago
ojbyrne 8y ago
This is a cohort study conducted over 15 years. You can't really criticize it only based on those numbers, at least not without looking at similar numbers from other cohort studies with the same length. I tend to ascribe some value to "It was published in the peer-reviewed British Medical Journal." Additional thought: Especially because it is essentially supporting the null hypothesis.
- flatline 8y ago
  There is nothing more frustrating than doing things you are bad at, regularly, especially if you are bound to get somewhat worse as time goes on. To your point, even if that were the reason for a significant number of departures, it would not go against the conclusion.
  - ido 8y ago
    Sounds like a use case ripe for game developers to jump on to.
- joe_the_user 8y ago
  You can't really criticize it only based on those numbers, at least not without looking at similar numbers from other cohort studies with the same length. Well, one can't criticize the quality of the study without looking how well other studies did. But even if all studies involve a massive drop-out rate, the drop-out rate still seems like something to look at in evaluating the accuracy of the study. I mean my concern, looking from the outside, isn't relative accuracy, but absolute accuracy. Not "is it as good as other studies?" but "it is good enough that I can rely on it?"
  - mjevans 8y ago
    15 -years- is a LONG time for a study. I'm not sure about the dropout rates, but it is a real statistical issue. I'm just not sure there's anything that can reasonably be done about it. You'd have to have something like federally mandated participation / continued participation. That's also very unlikely to pass ethics boards. BTW, it would also be an issue if the subjects were compensated to the point that they differed from the actual population at large. Fundamentally having more respect for doing things right, the right way, over the /long/ term, including the STEM fields, would be a 'good thing' for any society, but particularly western ones (where often the bad/evil antagonist is based on science or a cold lack of emotions; but the protagonist is based mostly on emotion and feeling instead of logic).
docbrown 8y ago
>The researchers lost touch with 13 people, while 57 people died and 332 declined to take part." This is one of the major downsides to longitudinal studies— people are unpredictable.
- RyanOD 8y ago
  I wonder if we'll see that change with the proliferation of communication across so many mediums. Between mail, email, texting, voice, social media, etc. staying in touch with a cohort might be much easier and more sustainable.
  - kgwgk 8y ago
    They lost touch with 2.6% of the participants. There is not much room for improvement there.
mirimir 8y ago
Maybe they dropped out because they were failing the tests of mental abilities, and were too frustrated and embarrassed to continue. But one would think that the authors had explored that. Edit: Yes, they did. See ojbyrne's comment.

amelius 8y ago

You can ask the same question about the NHS.

jodrellblank 8y ago
You mean "Does the NHS often reach the front page of HN?"? .. I don't get it, why would you ask that?
- amelius 8y ago
  Comment was about the research reaching the news section of the NHS.
  - arctangent 8y ago
    www.nhs.uk/news exists specifically to debunk false health claims (I'm tech lead for this site)
    - BossHogg 8y ago
      Thought this was a great site. Then I looked for the rss feed. Still good but...
      - arctangent 8y ago
        We syndicate our content in a different way: https://developer.api.nhs.uk/ https://developer.api.nhs.uk/

joe_the_user 8y ago

Wow, a rate of drop-out like that would certainly make me doubtful of the results of the study. It seems reasonably likely that people dropped-out in a fashion that would be weighted in one way or another.

make3 8y ago
for sure.. but I expect the real results to be worse (though this is purely speculative on my part), as I can only imagine people with steeper intellectual decline being more likely to drop out
- ojbyrne 8y ago
  Its mentioned in the paper: "However, recruitment to and retention in cognitive ageing studies is also affected by self selection to volunteer among better educated individuals who are ageing well, which is also true of our study. Thus, the findings might be biased towards those individuals with higher cognitive performance."
  - mirimir 8y ago
    Thanks for finding that. I was about to read the paper, looking for something like that.
- RyanOD 8y ago
  I immediately thought the same. Then I thought about for a bit longer and quickly came up with an equal number of reasons why participants experiencing more rapid mental decline might stick it out. Really no way to know.
  - gweinberg 8y ago
    Maybe, but given the huge dropout rate I don't see how we can conclude anything from this study. Of course, that would also be true if it had reached the opposite conclusion.
- 8y ago

ojbyrne 8y ago

This is a cohort study conducted over 15 years. You can't really criticize it only based on those numbers, at least not without looking at similar numbers from other cohort studies with the same length. I tend to ascribe some value to "It was published in the peer-reviewed British Medical Journal." Additional thought: Especially because it is essentially supporting the null hypothesis.

flatline 8y ago
There is nothing more frustrating than doing things you are bad at, regularly, especially if you are bound to get somewhat worse as time goes on. To your point, even if that were the reason for a significant number of departures, it would not go against the conclusion.
- ido 8y ago
  Sounds like a use case ripe for game developers to jump on to.
joe_the_user 8y ago
You can't really criticize it only based on those numbers, at least not without looking at similar numbers from other cohort studies with the same length. Well, one can't criticize the quality of the study without looking how well other studies did. But even if all studies involve a massive drop-out rate, the drop-out rate still seems like something to look at in evaluating the accuracy of the study. I mean my concern, looking from the outside, isn't relative accuracy, but absolute accuracy. Not "is it as good as other studies?" but "it is good enough that I can rely on it?"
- mjevans 8y ago
  15 -years- is a LONG time for a study. I'm not sure about the dropout rates, but it is a real statistical issue. I'm just not sure there's anything that can reasonably be done about it. You'd have to have something like federally mandated participation / continued participation. That's also very unlikely to pass ethics boards. BTW, it would also be an issue if the subjects were compensated to the point that they differed from the actual population at large. Fundamentally having more respect for doing things right, the right way, over the /long/ term, including the STEM fields, would be a 'good thing' for any society, but particularly western ones (where often the bad/evil antagonist is based on science or a cold lack of emotions; but the protagonist is based mostly on emotion and feeling instead of logic).

docbrown 8y ago

>The researchers lost touch with 13 people, while 57 people died and 332 declined to take part." This is one of the major downsides to longitudinal studies— people are unpredictable.

RyanOD 8y ago
I wonder if we'll see that change with the proliferation of communication across so many mediums. Between mail, email, texting, voice, social media, etc. staying in touch with a cohort might be much easier and more sustainable.
- kgwgk 8y ago
  They lost touch with 2.6% of the participants. There is not much room for improvement there.

mirimir 8y ago

Maybe they dropped out because they were failing the tests of mental abilities, and were too frustrated and embarrassed to continue. But one would think that the authors had explored that. Edit: Yes, they did. See ojbyrne's comment.