It seems like the problem is the entire premise. Suppose that older people are more likely to get cancer, so older people with cancer are more represented in t

It seems like the problem is the entire premise. Suppose that older people are more likely to get cancer, so older people with cancer are more represented in the training data. Then we discover that it has a 20% false negative rate for older people but a 35% false negative rate for younger people, i.e. it has a better handle on what cancer looks like in older people. This is fairly intrinsic, the only real way to fix it is to provide it with more samples of images of younger people with cancer, which may not be available. But you can also "fix" it by raising the false negative rate for older people to 35%. Doing this is idiotic and should not be done, but it allows people to claim that the model is now "fair", and so they will have the incentive to do things like that as long as we continue to demand "fairness" above other considerations. Moreover, you're likely to see similar effects with all kinds of other groups. Maybe farm workers are more likely to get skin cancer because they spend more time in the sun, and you could see the same kind of disparity in its ability to detect cancer in farm workers because the effects of doing farm work also have other long-term physical correlates that show up in the image. Probably nobody is studying this because farm workers are not a protected class, but it's just a thing that happens whenever you divide the population into distinct groups. That doesn't mean there is inherently anything to be done about it. It's a facet of what data is available.