2 ms·
Machine learning systems are good at finding the unconscious biases of large populations. Any change to force notions of political correctness on top of that i
by btilly 9y ago
Machine learning systems are good at finding the unconscious biases of large populations. Any change to force notions of political correctness on top of that is very complicated, and makes the algorithm work worse for its intended purpose. Literally, failing to accept the discrimination inherent in the data loses you money directly or indirectly.
This is well-known, and not Google's fault. Getting offended at Google for a reality they didn't create and can't control seems silly to me. If you don't like the result, work to change the incentives. Or work to change people's biases.
This kind of problem is not new. For example insurance companies have long known that where you live affects how much they are likely to pay out. If they base insurance rates on the data, the result is that your zip code becomes a bigger determinant of insurance rates than your driving record. Which results in very large effective racial discrimination. There are laws limiting that, for example prop 109 in California. However it is an eternal struggle because, in fact, competing insurance companies have a good motivation to have the cost of insurance reflect their best estimate of the expected cost of insuring you, and their cost really is a lot higher if you're in a "black neighborhood".