5 ms·
There's a version of this that I was once told and now repeat to everyone else: When you have a condition that affects only 1% of the population, a test that r
by idrios 5y ago
There's a version of this that I was once told and now repeat to everyone else:
When you have a condition that affects only 1% of the population, a test that reads negative 100% of the time will be 99% accurate.
- thaumasiotes 5y agoBut tests are not generally described in terms of accuracy. They're described in terms of sensitivity and specificity, because those quantities are both "accuracy" but they're not closely related to each other. A test that is always negative has 0% sensitivity and 100% specificity. Nobody thinks that sounds good.
- cortesoft 5y agoThat is how tests are described in scientific circles, but not with the general public. An overall `accuracy` number is usually given.
- amelius 5y agoPerhaps multiply sensitivity by specificity then, to get a single number that the public can use?
- thaumasiotes 5y agoThat's not a number the public can use. What would it be supposed to mean?
- amelius 5y agoA fuzzy AND of sensitivity and specificity. Another option is to take the minimum (part of the Zadeh operators). https://en.wikipedia.org/wiki/Fuzzy_logic https://en.wikipedia.org/wiki/Fuzzy_logic
- cortesoft 5y agoThe point isn’t that you can’t create a single number that can represent the effectiveness, just that you can’t get a single number that the public will understand. People have enough trouble with simple percentages.
- lordnacho 5y agoYou can combine them in an F1 score. Seems like a reasonable way to get a single number, though there might be some drawbacks I haven't thought of.
- thaumasiotes 5y agoIt looks like an F1 score is calculated from precision and recall. Sensitivity and recall are the same thing, but precision and specificity are different things. Precision is not a good way to describe a test because, as I mentioned elsewhere, it relies on information about the population being tested.
- MiddleEndian 5y agoMy favorite is the 50% test case. If you flip a coin as a test and ascribe heads to true, 50% of the time it will describe what's happening accurately. So if you want to know "Has the Sun exploded?" you can flip a coin, assign heads to "Yes the Sun has exploded", and 50% of the time it will be correct. You flip a coin, you get heads. You decide to flip the coin again, and you get heads again! What are the odds the Sun has exploded?