26 ms·

Can someone describe the core reason the second test is a false positive is not 1%? Why does the outside world (number of people being tested) affect my own tes

Can someone describe the core reason the second test is a false positive is not 1%? Why does the outside world (number of people being tested) affect my own test error probability?

andreareina 5y ago
It's the same reason that the first test doesn't get you 1%, but the testing pool has changed. Instead of testing 10000 people without and 1 person with, you're testing 100 people without (and the same person with). It's a Bayesian update. Every (independent) positive result increases your confidence of having the condition by 100x. At first you have no information so you assume the base rate, 1:10000. At the first test, that goes to 100:10000. Second test gets you to 10000:10000 or even odds. Third positive result finally gets you to 1000000:10000, or a 99% confidence of having the condition.
MrJohz 5y ago
A maths teacher once told me that if I'm struggling to work through a logical problem in my head, I should look at the extreme numbers, because that often gives a clearer picture of what's happening. This has helped me a lot, and is how I would approach this: Let's say we've got a test that: * If you've got cancer, detects it 100% of the time (to make the maths easy) * If you haven't got cancer, gives you a (true) negative 99% of the time, and a (false) positive 1% of the time. For starters, consider a population of 100 people, all of whom have cancer. Our magical test always finds these cases, so we have an 100% positive rate, and a 0% false positive rate. (None of these can be false positives because we know they all really do have cancer.) Now consider a group of 200 people, 50% of whom have cancer, and 50% of whom don't. The group who have cancer will still all get detected, but if the 100 people who don't have cancer, one of them will be marked as having cancer even though they don't. In total, that's 101 people who will get a positive result. So our overall positivity rate is 101/200 (~50%), but our false positivity rate is the number of false positives divided by the number of all positives, so 1/101 (~1%) Now consider the case where we have 101 people, of whom only one has cancer. If we run our test a third time, we of course detect the person with cancer, but of the remaining 100 who don't have cancer, we will detect a second person. That's two positive cases in 101 people, so our positivity rate is ~2%. However, of those two positive cases we know only one really has cancer - the false positivity rate is 1/2, or 50%. So that's three situations where the only thing we've varied is the proportion of the population who have the condition we're scanning for. (Technically we also changed how big the population was in the first place but that was just to make the maths easier - it doesn't affect the results at all.) However, we get a different rate of false positives in every test, because the number of false positives is dependent on the ratio between the number of people with cancer and the number of people without. It's also worth noting that this obviously doesn't just apply to cancer, but also other situations. Most relevant to us in the last couple of years is probably Covid tests - you might have seen over the summers some scepticism about how valuable it is to do lots and lots of tests, and that's because during those times, the number of affected people was lower, and so these tests were more likely to produce less useful results. However, in winter, even relatively poor tests become more useful, because there are more true positives, and therefore a lower rate of false positives.
lostcolony 5y ago
First, it assumes that a prior false positive does not increase the likelihood of a false positive on testing again (that is, a retest's result is independent from the first test's result). Given that, from the first test, we have 100 people who were false positives, and 1 person who is truly positive. We would expect that 1 truly positive person to be positive again (we have a 99% chance of it from the test, after all), and 1 of the 100 who were false positive before to also test positive this second time (that 1% inaccuracy rate). Given 1 true positive, and 1 false positive, in that sample, you would expect a positive rate to mean you have a 50% chance of being the true positive. If we did not have information from the first test, yes, it would be ~1% chance of a false positive. We have nearly a 0% chance of having a true positive, so of the 101 people, and a 99% accurate test, we would expect one to come out positive, despite not having the disease. But, because we have that first test, from a much larger sample size, we likely have one person who is a true positive, and so could expect two people to test positive, one with the disease, and one without (the 1/100 of the remaining), giving us a 50% chance a positive result = has the disease.

Can someone describe the core reason the second test is a false positive is not 1%? Why does the outside world (number of people being tested) affect my own test error probability?

andreareina 5y ago

It's the same reason that the first test doesn't get you 1%, but the testing pool has changed. Instead of testing 10000 people without and 1 person with, you're testing 100 people without (and the same person with). It's a Bayesian update. Every (independent) positive result increases your confidence of having the condition by 100x. At first you have no information so you assume the base rate, 1:10000. At the first test, that goes to 100:10000. Second test gets you to 10000:10000 or even odds. Third positive result finally gets you to 1000000:10000, or a 99% confidence of having the condition.

MrJohz 5y ago

A maths teacher once told me that if I'm struggling to work through a logical problem in my head, I should look at the extreme numbers, because that often gives a clearer picture of what's happening. This has helped me a lot, and is how I would approach this: Let's say we've got a test that: * If you've got cancer, detects it 100% of the time (to make the maths easy) * If you haven't got cancer, gives you a (true) negative 99% of the time, and a (false) positive 1% of the time. For starters, consider a population of 100 people, all of whom have cancer. Our magical test always finds these cases, so we have an 100% positive rate, and a 0% false positive rate. (None of these can be false positives because we know they all really do have cancer.) Now consider a group of 200 people, 50% of whom have cancer, and 50% of whom don't. The group who have cancer will still all get detected, but if the 100 people who don't have cancer, one of them will be marked as having cancer even though they don't. In total, that's 101 people who will get a positive result. So our overall positivity rate is 101/200 (~50%), but our false positivity rate is the number of false positives divided by the number of all positives, so 1/101 (~1%) Now consider the case where we have 101 people, of whom only one has cancer. If we run our test a third time, we of course detect the person with cancer, but of the remaining 100 who don't have cancer, we will detect a second person. That's two positive cases in 101 people, so our positivity rate is ~2%. However, of those two positive cases we know only one really has cancer - the false positivity rate is 1/2, or 50%. So that's three situations where the only thing we've varied is the proportion of the population who have the condition we're scanning for. (Technically we also changed how big the population was in the first place but that was just to make the maths easier - it doesn't affect the results at all.) However, we get a different rate of false positives in every test, because the number of false positives is dependent on the ratio between the number of people with cancer and the number of people without. It's also worth noting that this obviously doesn't just apply to cancer, but also other situations. Most relevant to us in the last couple of years is probably Covid tests - you might have seen over the summers some scepticism about how valuable it is to do lots and lots of tests, and that's because during those times, the number of affected people was lower, and so these tests were more likely to produce less useful results. However, in winter, even relatively poor tests become more useful, because there are more true positives, and therefore a lower rate of false positives.

lostcolony 5y ago

First, it assumes that a prior false positive does not increase the likelihood of a false positive on testing again (that is, a retest's result is independent from the first test's result). Given that, from the first test, we have 100 people who were false positives, and 1 person who is truly positive. We would expect that 1 truly positive person to be positive again (we have a 99% chance of it from the test, after all), and 1 of the 100 who were false positive before to also test positive this second time (that 1% inaccuracy rate). Given 1 true positive, and 1 false positive, in that sample, you would expect a positive rate to mean you have a 50% chance of being the true positive. If we did not have information from the first test, yes, it would be ~1% chance of a false positive. We have nearly a 0% chance of having a true positive, so of the 101 people, and a 99% accurate test, we would expect one to come out positive, despite not having the disease. But, because we have that first test, from a much larger sample size, we likely have one person who is a true positive, and so could expect two people to test positive, one with the disease, and one without (the 1/100 of the remaining), giving us a 50% chance a positive result = has the disease.