3 ms·

> If we gave this test to random people 75% of the people who test positive are healthy. So this clearly isn't a useful test as a first screening Unless I'm m

> If we gave this test to random people 75% of the people who test positive are healthy. So this clearly isn't a useful test as a first screening Unless I'm misreading what you're saying, I think you have Bayes's rule confused in that statement. If 75% of positive results are false, that's actually right in line with what you might want for a first screening. Of course, you want to know not just P(X|Y), but also P(Y|X) -- you really need the joint distribution to determine how useful the test is. For a rapid screening, you generally want a low false negative, and you're willing to compromise by having a high false positive rate as a result. The real thing you care about is P(test positive | have MDD), not P(have MDD | test positive). You don't care about the latter (within reason - obviously you could just have everyone test positive, but you're also trying to maximize the former probability subject to some constraint - usually time/money). Psychiatric conditions are notoriously difficult to test/evaluate/treat because they are oftentimes heterogeneous, so I don't know what kind of numbers we should be looking for in this case, but I just wanted to clarify that having a majority of people who test positive ending up being healthy actually doesn't necessarily rule a test out as being a useful rapid-response screening, in the general case.

mjb 15y ago
One issue of concern is the possible cost of these false positives. If doctors are very careful to communicate to patients that 75% of positive results will only require a second test, then that's OK. Sadly, that doesn't seem to be what happens in many cases. The communication given to the patient is 'this is a test for disease X', and the patient rightfully interprets a positive result as 'I have X'. The problem comes in because 'I have depression' could lead the patient to take risky or dangerous action. Sadly, misunderstanding on the side of doctors (who, at least the ones I know well, aren't given a strong education in statistics at university) might also lead them to make poor decisions. As long as everybody is honest about the meaning of the test, and understands the differences between P(Y), P(X), P(Y|X) and P(X|Y) then that's great. Sadly, there is good evidence that this is not always the case. For an example more along HN's normal lines, consider hard drive monitoring data from SMART. Several public studies of SMART attributes in modern drives have shown that positive predictive value of SMART data as a drive failure indicator is very poor. Many of the drives which are replaced because they are failing SMART are actually just fine, but a lot of money, labor and downtime is spent on the incorrect assumption that P(healthy|SMART failure) is much smaller than P(unhealthy|SMART failure).
- pwf 15y ago
  The bigger issue to me is that there isn't a second test. You can't just send the samples off to the lab and let them culture for a few days. What's the point of making someone take this blood test if they still have to fill out the mental health questionnaire anyways?
  - gte910h 15y ago
    That's why the low false negative rate is important. If you score a negative you don't have depression.

mjb 15y ago

One issue of concern is the possible cost of these false positives. If doctors are very careful to communicate to patients that 75% of positive results will only require a second test, then that's OK. Sadly, that doesn't seem to be what happens in many cases. The communication given to the patient is 'this is a test for disease X', and the patient rightfully interprets a positive result as 'I have X'. The problem comes in because 'I have depression' could lead the patient to take risky or dangerous action. Sadly, misunderstanding on the side of doctors (who, at least the ones I know well, aren't given a strong education in statistics at university) might also lead them to make poor decisions. As long as everybody is honest about the meaning of the test, and understands the differences between P(Y), P(X), P(Y|X) and P(X|Y) then that's great. Sadly, there is good evidence that this is not always the case. For an example more along HN's normal lines, consider hard drive monitoring data from SMART. Several public studies of SMART attributes in modern drives have shown that positive predictive value of SMART data as a drive failure indicator is very poor. Many of the drives which are replaced because they are failing SMART are actually just fine, but a lot of money, labor and downtime is spent on the incorrect assumption that P(healthy|SMART failure) is much smaller than P(unhealthy|SMART failure).

pwf 15y ago
The bigger issue to me is that there isn't a second test. You can't just send the samples off to the lab and let them culture for a few days. What's the point of making someone take this blood test if they still have to fill out the mental health questionnaire anyways?
- gte910h 15y ago
  That's why the low false negative rate is important. If you score a negative you don't have depression.