There is another issue I've read about, which is that outside the context of a medical diagnosis, even a very accurate test would still be relatively unhelpful

There is another issue I've read about, which is that outside the context of a medical diagnosis, even a very accurate test would still be relatively unhelpful because it can be measuring natural variability. (E.g., testing some metric associated with prostate cancer in young men/women — prostate cancer only appears in older men, so testing young people is mostly useless.) Whereas if you're already potentially sick and go to a doctor, the relevant tests then have more predictive power.