2 ms·

They almost certainly will not encode any explicit biases of the algorithm authors (e.g. “she’s pregnant, can we get rid of her?”), which is already a great ste

They almost certainly will not encode any explicit biases of the algorithm authors (e.g. “she’s pregnant, can we get rid of her?”), which is already a great step forward, as explicit bias is still sadly far too common in the workplace. When implicit biases find their way into algorithms, it is usually a specific edge case that’s a function of the training data (e.g. facial recognition preforming poorly on darker skinned people due to training set imbalance). The biggest case of implicit bias in algorithms I’m aware of is when the training set is the behavior of society as a whole, as with search engines: for example, women search less for C-level jobs and so are less likely to be shown postings for C-level jobs when doing job searches (thereby perpetuating a vicious cycle), or websites depicting Black teens are more likely to show them in a criminal context than White teens, so searches for “black teenager” show pictures of criminals whereas searches for “white teenager” do not. These implicit biases are not encoded by the algorithm developer but rather by the dataset the algorithm is applied to. In the case of automated firing, I don’t see how implicit bias can creep in if the metrics are strictly work-related (e.g. fraction of on-time deliveries). For the record, I do not agree with automating performance metrics, since they cannot account for nuance (e.g. delivery drivers assigned to gated communities have issues opening the gate and thus deliver fewer on-time packages, which is not accounted for in the algorithm.) However, this is not a form of demographic bias, explicit or implicit.

jeromegv 5y ago
Where I think you are wrong here is that you fail to consider the fact that the same programmer must decide which dataset/training data they will use for the AI. So implicit biais is not just external to the programmer, it's definitely caused by the programmer. And implicit biais goes way beyond their training dataset. Did they put a clause "if pregnant, then...", likely not. However, what conditions did they use for testing this algorithm and ensure if it works? Testing is really much part of the process as well.
- MontyCarloHall 5y ago
  > So implicit biais is not just external to the programmer, it's definitely caused by the programmer. I agree in the case of things like facial recognition, but in the case of search results, Google can’t exactly choose whether websites it indexes present “Black teens” in a poor light and “White teens” in a good light. Perfectly unbiased algorithms trained on data produced by society will always exactly reflect the unfortunate biases of society. (Ironically, attempting to correct for society’s biases in the algorithm would by definition bias the algorithm itself!) This is (likely) not the case with Amazon drivers, since the training data are (presumably) strictly job performance-related, e.g. fraction of on-time deliveries, or number of times a worker showed up late. And even if these happen to correlate to demographics (e.g. if Black workers lived further from the warehouse than White workers [or vice-versa] and were thus more likely to show up to work late), tracking employees based on their on-time performance is not bias on Amazon’s part, since a White worker who showed up late would be just as likely to be fired as a Black worker who showed up late. The fact that one demographic (by no fault of its own) happens to show up late more often is not bias on the part of Amazon.

jeromegv 5y ago

Where I think you are wrong here is that you fail to consider the fact that the same programmer must decide which dataset/training data they will use for the AI. So implicit biais is not just external to the programmer, it's definitely caused by the programmer. And implicit biais goes way beyond their training dataset. Did they put a clause "if pregnant, then...", likely not. However, what conditions did they use for testing this algorithm and ensure if it works? Testing is really much part of the process as well.

MontyCarloHall 5y ago
> So implicit biais is not just external to the programmer, it's definitely caused by the programmer. I agree in the case of things like facial recognition, but in the case of search results, Google can’t exactly choose whether websites it indexes present “Black teens” in a poor light and “White teens” in a good light. Perfectly unbiased algorithms trained on data produced by society will always exactly reflect the unfortunate biases of society. (Ironically, attempting to correct for society’s biases in the algorithm would by definition bias the algorithm itself!) This is (likely) not the case with Amazon drivers, since the training data are (presumably) strictly job performance-related, e.g. fraction of on-time deliveries, or number of times a worker showed up late. And even if these happen to correlate to demographics (e.g. if Black workers lived further from the warehouse than White workers [or vice-versa] and were thus more likely to show up to work late), tracking employees based on their on-time performance is not bias on Amazon’s part, since a White worker who showed up late would be just as likely to be fired as a Black worker who showed up late. The fact that one demographic (by no fault of its own) happens to show up late more often is not bias on the part of Amazon.