3 ms·

I think you're oversimplifying the issue. It's not important that the model cannot distinguish between people of different demographics, it's important that the

I think you're oversimplifying the issue. It's not important that the model cannot distinguish between people of different demographics, it's important that the model does not use demographic information in place of actual diagnosis for the sake of better accuracy. That the model can determine biological sex from X-rays wouldn't be an issue if it never shortcuts the diagnostic process by using biological sex in place of meaningful diagnostic data. I would not like a model to ignore a melanoma in my chest scan because it can deduce that I was born male and my risk of breast cancer is quite low. The idea of penalising a model which takes such biological shortcuts (because its subgroup accuracy gets worse) seems like a good solution, and it's cool that the approach works in TFA.

carbocation 2y ago
I believe that imbuing what the model does to make a prediction with the idea of "shortcuts" is oversimplifying a more complex issue. I don't think it's helpful to describe a model that can distinguish demographics as taking "shortcuts". I think that adds a layer of jargon that we then need to cut through to understand what is going on. There are plenty of tools that have been developed to enforce that models perform in a manner that is unbiased across some dimension (e.g., sex, or hospital, etc). (For example unsupervised domain adaptation.) I think that splitting the field with jargon makes it more difficult to follow the breadth of the field.