3 ms·

> Some people expect that fighting algorithmic racism is going to come with some sort of trade-off. Um, that's because we know it comes with trade-offs once yo

> Some people expect that fighting algorithmic racism is going to come with some sort of trade-off. Um, that's because we know it comes with trade-offs once you have the most optimal algorithm. See for instance https://arxiv.org/pdf/1610.02413.pdf https://arxiv.org/pdf/1610.02413.pdf. If your best performing algorithm is "racist" (for some definition of racist") you are mathematically forced to make tradeoffs if you want to eliminate that "racism". Of course, defining "racism" itself gets extremely tricky because many definitions of racism are mutually contradictory (https://arxiv.org/pdf/1609.05807.pdf https://arxiv.org/pdf/1609.05807.pdf).

ma2rten 8y ago
Not necessarily. In the case of word vectors we are using unsupervised learning to identify patterns in a large corpus of data to improve the learning. This is a completely different issue than your credit score example, which is supervised learning. Not all patterns are equally useful. By removing those unuseful patterns we might make less mistakes (for example giving negative sentiment to a Mexican restaurant review) and free up capacity in the word vectors to store more useful patterns. I would expect baking other real-world assumptions into your word vectors unrelated to bias could also be helpful.
dan-robertson 8y ago
> If your best performing algorithm is racist There are two ways to look at this: 1. Racism makes the algorithm good so we should make the algorithm less racist (at a cost to its performance) or decide we want to allow systematic racism. 2. The metric for how good the algorithm is (ie training data) encourages it to be racist and therefore correcting the bias in the algorithm may decrease its performance on the training data but may not affect its performance in the real world, or decrease its performance in the “performance + meets legal requirements” metric.

ma2rten 8y ago

Not necessarily. In the case of word vectors we are using unsupervised learning to identify patterns in a large corpus of data to improve the learning. This is a completely different issue than your credit score example, which is supervised learning. Not all patterns are equally useful. By removing those unuseful patterns we might make less mistakes (for example giving negative sentiment to a Mexican restaurant review) and free up capacity in the word vectors to store more useful patterns. I would expect baking other real-world assumptions into your word vectors unrelated to bias could also be helpful.

dan-robertson 8y ago

> If your best performing algorithm is racist There are two ways to look at this: 1. Racism makes the algorithm good so we should make the algorithm less racist (at a cost to its performance) or decide we want to allow systematic racism. 2. The metric for how good the algorithm is (ie training data) encourages it to be racist and therefore correcting the bias in the algorithm may decrease its performance on the training data but may not affect its performance in the real world, or decrease its performance in the “performance + meets legal requirements” metric.