35 ms·
If the classifier is being used for medical diagnosis, race, age and sex may be important factors. Ignoring them would amount to systematically ruining innocent
by yorwba 9y ago
If the classifier is being used for medical diagnosis, race, age and sex may be important factors. Ignoring them would amount to systematically ruining innocent hopeful life without any wrongdoing.
The problem usually isn't injustice caused by model errors, because humans make unjust errors all the time; using a model that's better than humans is still a win even if the model isn't perfect. Likewise for medical diagnosis, if a model is better than the average doctor, it should replace them.
The actual problem is that humans mistake the model prediction for something it is not. E.g. if you train a model on the data of people accused of a crime to predict whether they will be convicted, you're modeling P(convicted | accused). Even if you train this model to perfection (zero test errors), you would at best reproduce the existing legal system; which is a win if the model is cheaper. But people tend to mistake the model output for P(should be convicted | accused), which assumes that the legal system is perfect. Or worse, they think of it as P(should be convicted), without accusation.
The solution to your "go in the field and run your classifier on everyone" thought experiment is not to hide race, sex and age from the model. If the model had no access to any features at all, it would just predict the baseline conviction rate, much higher than the baseline crime rate. Using the model output as evidence of a crime would be just as silly, because that's not what the model was trained to do.
The solution is to not even try to predict something for which you don't have any data. Modeling can never make a process more accurate, it can only make an accurate process faster and cheaper (and slightly less accurate). Sometimes the accurate process is to wait for a few years and see what happens, e.g. for cancer mortality or recidivism rates. Sometimes the accurate process is to ask a lot of experts.
Once you have a model for the accurate process that is better than the cheaper process you'd otherwise use, then you can replace it by the model. Instead of a single human judge or a jury, have a model trained on the decisions of multiple judges and juries deliberating much longer than usual. Instead of a single doctor, have a model that has seen the medical histories of thousands of cancer patients and knows exactly which treatments worked.