> How do you de-bias datasets, when the raw data (coming from the real world) starts biased to begin with? I can think of at least one trick that might work. B

> How do you de-bias datasets, when the raw data (coming from the real world) starts biased to begin with? I can think of at least one trick that might work. Build a classifier capable of identifying race from the data, and then reverse its gradient to remove its signals. This would deracialize a dataset, so that there is no detectable latent information in it. However, whether or not my particular solution works is somewhat beside the point. It's an engineering challenge to figure out how to do this, but that isn't an argument that ML is fundamentally bad for these purposes. > For example, there are already attempts to use expert systems in the courts - to "scientifically" determine flight risk (and thus bail), and even chance of re-offence (and thus punishments). Those are trained on datasets of convictions. But we know that both conviction rates, and arrest rates, are themselves indicative of bias - e.g. some people get arrested more often for a crime, for which other people might get a verbal warning from police. I totally agree that this is a problem. But it seems to me like the solution is more effort on fixing the biases, not reverting to human judgment. I think it's always important to make sure we understand what the baseline is we're comparing to. Humans are racist. Humans created the biases in these datasets. These algorithmic tools just bring those biases to the surface and make them quantifiable. It seems to me that that represents an incredible opportunity to actually fix the biases in a measurable way.