4 ms·

To summarize the paper more technically: In a large cohort of patients with cognitive decline symptoms, it predicted with 90% accuracy which patients are deter

by Rhapso 2y ago

To summarize the paper more technically: In a large cohort of patients with cognitive decline symptoms, it predicted with 90% accuracy which patients are determined by specialists to have Alzheimer's. This is in contrast to a 60% correct diagnosis rate by general practitioners. The test has both 90%+ specificity and sensitivity. The proposal is that this could improve the overall diagnostic rate by GPs by giving them a blood test to use when they are suspicious of Alzheimer's.

nrr 2y ago
"The test has both 90%+ specificity and sensitivity." This by itself feels like a big deal, even if we strip away the Alzheimer's context. Do we have a lot of blood tests that are both sensitive and specific to this degree?
- tptacek 2y ago
  Seems important as a clinical substitute/adjunct for existing diagnostics (especially because they suck for other reasons), but the base rate for Alzheimers is just 10%, so like, you couldn't really run this on a whim, right?
  - nrr 2y ago
    I mean, part of the reason why I ask this question is that specificity and sensitivity tend to be inversely related: as a test becomes more sensitive, it tends to become less specific, say. (While I'm here, in case folks are lacking the statistics background: specificity and sensitivity refer to the probabilities that a test will return with a negative test result in the absence of a condition or a positive test result in the presence of a condition respectively.) Most of the "promising" tests I've seen tend to be something like 80% sensitive and 20% specific (testing for Lyme disease comes to mind), which makes them no better than flipping a weighted coin. The fact that this Alzheimer's test beats that feels like a big deal on the probability merits alone, and I can't think of basically any other tests that do that. (Serum cardiac troponins maybe?)
    - Rhapso 2y ago
      They don't offer any tests for "general population" situations, only "people we think might already have dementia". This is a really strong filter up-front. I suspect the test would be high sensitivity very low specificity in the general population, but that is an intuition not backed by any data.
      - nrr 2y ago
        Aha, yeah, restricting your population could certainly have that effect on the specificity. Thanks for clarifying.
        tptacek 2y ago
        For what it's worth, I'm not sure specificity/sensitivity is the limiting factor here (for doing broad population surveys with this test); it's the low base rate of Alzheimers relative to the specificity. At 90% true positive and 10% base rate, I think? (I suck at math) a positive test has like a coin flip chance of being right?
        nrr 2y ago
        We're in the realm of probability, which is mysterious sometimes even to people who have a math background. I didn't fully grok it until I studied intensely for my first actuarial exams. (: This kind of thing is almost always a weighted coin toss: with sensitivity or specificity alone, you only have two possible outcomes (present/relevant, absent/irrelevant), and the thing that changes is the probability distribution of those outcomes. Combining the two gets you the full four: present and relevant; present and irrelevant; absent and relevant; and absent and irrelevant. Taking the uniform distribution, they're all 25% likely, but the idea is to find a probability distribution that makes the "present and relevant" and "absent and relevant" outcomes more likely. Since I myself don't work in a clinical setting, I simply hadn't considered that the clinician would want to exercise discretion in pre-screening for specificity before ordering the test in order to get there. Oops.
  - Rhapso 2y ago
    > you couldn't really run this on a whim, right? yes, it is meant for the context of "a doctor thinks this patient may have Alzheimer's". They don't even test it in a "general population" situation. Its not presented as a screening test.

by Rhapso 2y ago

nrr 2y ago

"The test has both 90%+ specificity and sensitivity." This by itself feels like a big deal, even if we strip away the Alzheimer's context. Do we have a lot of blood tests that are both sensitive and specific to this degree?

tptacek 2y ago
Seems important as a clinical substitute/adjunct for existing diagnostics (especially because they suck for other reasons), but the base rate for Alzheimers is just 10%, so like, you couldn't really run this on a whim, right?
- nrr 2y ago
  I mean, part of the reason why I ask this question is that specificity and sensitivity tend to be inversely related: as a test becomes more sensitive, it tends to become less specific, say. (While I'm here, in case folks are lacking the statistics background: specificity and sensitivity refer to the probabilities that a test will return with a negative test result in the absence of a condition or a positive test result in the presence of a condition respectively.) Most of the "promising" tests I've seen tend to be something like 80% sensitive and 20% specific (testing for Lyme disease comes to mind), which makes them no better than flipping a weighted coin. The fact that this Alzheimer's test beats that feels like a big deal on the probability merits alone, and I can't think of basically any other tests that do that. (Serum cardiac troponins maybe?)
  - Rhapso 2y ago
    They don't offer any tests for "general population" situations, only "people we think might already have dementia". This is a really strong filter up-front. I suspect the test would be high sensitivity very low specificity in the general population, but that is an intuition not backed by any data.
    - nrr 2y ago
      Aha, yeah, restricting your population could certainly have that effect on the specificity. Thanks for clarifying.
      - tptacek 2y ago
        For what it's worth, I'm not sure specificity/sensitivity is the limiting factor here (for doing broad population surveys with this test); it's the low base rate of Alzheimers relative to the specificity. At 90% true positive and 10% base rate, I think? (I suck at math) a positive test has like a coin flip chance of being right?
        nrr 2y ago
        We're in the realm of probability, which is mysterious sometimes even to people who have a math background. I didn't fully grok it until I studied intensely for my first actuarial exams. (: This kind of thing is almost always a weighted coin toss: with sensitivity or specificity alone, you only have two possible outcomes (present/relevant, absent/irrelevant), and the thing that changes is the probability distribution of those outcomes. Combining the two gets you the full four: present and relevant; present and irrelevant; absent and relevant; and absent and irrelevant. Taking the uniform distribution, they're all 25% likely, but the idea is to find a probability distribution that makes the "present and relevant" and "absent and relevant" outcomes more likely. Since I myself don't work in a clinical setting, I simply hadn't considered that the clinician would want to exercise discretion in pre-screening for specificity before ordering the test in order to get there. Oops.
- Rhapso 2y ago
  > you couldn't really run this on a whim, right? yes, it is meant for the context of "a doctor thinks this patient may have Alzheimer's". They don't even test it in a "general population" situation. Its not presented as a screening test.