2 ms·

What's interesting to me is that the three big criteria tests are rated on don't take this into account at all. In a population of 1,000,000: TP=99 TN=989901

by Apes 5y ago

What's interesting to me is that the three big criteria tests are rated on don't take this into account at all. In a population of 1,000,000: TP=99 TN=989901 FP=9999 FN=1 Accuracy = TN + TP / (TP+TN+FP+FN) = 99% Sensitivity = TP / (TP + FN ) = 99% Specificity = TN / (TN + FP ) = 99% Looks like a great test across the board! However, the actual chance someone with a positive test result has the disease: TP / ( TP + FP ) = ~1% So why isn't this ratio part of the major test criteria?

killjoywashere 5y ago
PPV is absolutely something that's tracked. In fact, if you read the instructions for use (equivalent to "package insert") for lab diagnostics, you'll find the manufacturer's truth tables and can calculate the PPV if it's not in there. FDA keeps them online, so you don't even need to buy the diagnostic. Look the up.
majormajor 5y ago
I'm not sure 99% is considered the gold standard you think it is. Sure, a lot of our tests can't even do that well, but we know that 99% isn't a lot for large populations. We even have jargon like "five nines", "six sigma" around the fact that 99% isn't a lot.
thaumasiotes 5y ago
> So why isn't this ratio part of the major test criteria? Because it has nothing to do with the test. Sensitivity and specificity are facts about the test, and conceptually they will remain whatever they are if you start testing a different population. Whether doing a test has any value is a fact about the population ("olympic athletes are never anemic, so there's no point in testing them for anemia") that has nothing to do with the test. Why would you evaluate the test based on that?
colinmhayes 5y ago
> So why isn't this ratio part of the major test criteria? I think it's because this number depends on the percent of the population that is positive which can change after the trial is conducted.

by Apes 5y ago

killjoywashere 5y ago

PPV is absolutely something that's tracked. In fact, if you read the instructions for use (equivalent to "package insert") for lab diagnostics, you'll find the manufacturer's truth tables and can calculate the PPV if it's not in there. FDA keeps them online, so you don't even need to buy the diagnostic. Look the up.

majormajor 5y ago

I'm not sure 99% is considered the gold standard you think it is. Sure, a lot of our tests can't even do that well, but we know that 99% isn't a lot for large populations. We even have jargon like "five nines", "six sigma" around the fact that 99% isn't a lot.

thaumasiotes 5y ago

> So why isn't this ratio part of the major test criteria? Because it has nothing to do with the test. Sensitivity and specificity are facts about the test, and conceptually they will remain whatever they are if you start testing a different population. Whether doing a test has any value is a fact about the population ("olympic athletes are never anemic, so there's no point in testing them for anemia") that has nothing to do with the test. Why would you evaluate the test based on that?

colinmhayes 5y ago

> So why isn't this ratio part of the major test criteria? I think it's because this number depends on the percent of the population that is positive which can change after the trial is conducted.