5 ms·

20% false positives is not "just fine". That is a huge error.

skissane 6y ago
I agree with you, but in psychiatry and clinical psychology, a 20% false positive rate is considered "acceptable". Consider for example the Cochrane review of autism diagnostic instruments in preschoolers [1]. It found that on average (in research contexts), the main diagnostic test used for ASD, the ADOS[2], had a specificity of 0.80 (in other words, a 20% false positive rate). The Cochrane authors considered that acceptable. Now, professionals may consider that acceptable – but how many parents are told that the test used to diagnose their child has a 20% false positive rate? Added to that, there is evidence that the tests are actually less accurate in non-research contexts (the standards for administering them are certainly lower), which may make the false positive rate in non-research contexts even higher (on that topic see [3]). [1] https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD009044.pub2/full https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD... [2] actually the original ADOS has now been superseded by a newer version, the ADOS-2, but the Cochrane reviewers couldn't find any studies of the ADOS-2 that met their inclusion criteria. In any case, I doubt the accuracy of the new version is substantially different from the original one. [3] https://link.springer.com/article/10.1007/s00787-018-1143-y https://link.springer.com/article/10.1007/s00787-018-1143-y
anbende 6y ago
These are screening tools. They are not meant to make definitive diagnoses for the most part. They are used for epidemiology and flagging clients for further assessment. There's always a balance to be made between sensitivity (picking up on real positives) and specificity (not producing false positives). The balance for these instruments is at about 80-85%. If a person is depressed, there's an 80-85% chance they will be correctly classified. If a person is not depressed, there is an 80-85% chance they will be correct classified. Raise the threshold and you'll be more specific but less sensitive. Lower the threshold and you'll be less specific but more sensitive. It's just how the game works and why questionnaires are only first-pass tools.

I agree with you, but in psychiatry and clinical psychology, a 20% false positive rate is considered "acceptable". Consider for example the Cochrane review of autism diagnostic instruments in preschoolers [1]. It found that on average (in research contexts), the main diagnostic test used for ASD, the ADOS[2], had a specificity of 0.80 (in other words, a 20% false positive rate). The Cochrane authors considered that acceptable. Now, professionals may consider that acceptable – but how many parents are told that the test used to diagnose their child has a 20% false positive rate? Added to that, there is evidence that the tests are actually less accurate in non-research contexts (the standards for administering them are certainly lower), which may make the false positive rate in non-research contexts even higher (on that topic see [3]). [1] https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD009044.pub2/full https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD... [2] actually the original ADOS has now been superseded by a newer version, the ADOS-2, but the Cochrane reviewers couldn't find any studies of the ADOS-2 that met their inclusion criteria. In any case, I doubt the accuracy of the new version is substantially different from the original one. [3] https://link.springer.com/article/10.1007/s00787-018-1143-y https://link.springer.com/article/10.1007/s00787-018-1143-y

anbende 6y ago

These are screening tools. They are not meant to make definitive diagnoses for the most part. They are used for epidemiology and flagging clients for further assessment. There's always a balance to be made between sensitivity (picking up on real positives) and specificity (not producing false positives). The balance for these instruments is at about 80-85%. If a person is depressed, there's an 80-85% chance they will be correctly classified. If a person is not depressed, there is an 80-85% chance they will be correct classified. Raise the threshold and you'll be more specific but less sensitive. Lower the threshold and you'll be less specific but more sensitive. It's just how the game works and why questionnaires are only first-pass tools.