I’m suspicious that there’s another factor in play: images being correctly labeled in the training set. Even from high-end hospitals, my personal experience lea

I’m suspicious that there’s another factor in play: images being correctly labeled in the training set. Even from high-end hospitals, my personal experience leads me to believe that radiologists make both major types of error on a regular basis: calling out abnormalities in an image that are entirely irrelevant and missing actual relevant problems in an image that are subsequently easily seen by a doctor who is aware of what seems to be actually wrong. To top it off, sure many patients end up being diagnosed with whatever the radiologist saw, and no one ever confirms that the diagnosis was really correct. (And how could they? A lot of conditions are hard to diagnose!) So the training data is probably hugely biased, and the models will learn to predict the training labels as opposed to any magically correct “ground truth”. And internally detecting demographics and producing an output biased by demographics may well result in a better match to the training data than a perfect, unbiased output would be.