3 ms·

This is a valuable lesson in what AI can do for society: act as a mirror to show us systemic biases that exist that people try to pretend don't exist. I think

This is a valuable lesson in what AI can do for society: act as a mirror to show us systemic biases that exist that people try to pretend don't exist. I think most here would understand systemic bias if reflected in a distributed system. Imagine a classifier where, through no explicit intent of the designer of the classifier, always said that some user login was malicious or some binary was malware or the like, even when it was not. And all those users or binaries had something in common that was apparent, and were all being misclassified. That would be systemic bias, and nobody would blink an eye about a call for it to be corrected. Here we see systemic racial bias, probably not because of individual racial animus on the part of some Google engineer, but because society has systemic racial bias and the classifier is just reflecting that because it was trained on public data. We should look at incidents like this differently: not as yet another chance to argue about what Google should or shouldn't do, but as a chance to see that a socially-neutral AI that does not have bias built into it is learning racial bias from society.

pfannkuchen 3y ago
> the classifier is just reflecting that because it was trained on public data Are you suggesting that the cause here is that there are a lot of racist images on the internet visually depicting black people as gorillas? I assume that is not the understanding of most people here who are questioning what makes this racist, because if that is indeed the cause then I agree it is obviously racist. Is it the cause though? I would be interested to see some evidence or rationale, because I don’t think I have ever seen an image like this outside of perhaps a handful of very old hand drawn propaganda images (which presumably would not affect the classifier much due to being overwhelmed by the vast number of images of real people and real gorillas).
jl2718 3y ago
There is a rigorous definition of bias, and that isn’t it. That is called ‘variance’. It doesn’t matter if there is structure to the variance. It doesn’t matter if the model is consistently wrong for one type of input. It is not bias unless it has been specifically and intentionally programmed to predict something other than the result from the maximum likelihood estimator given the data. A trickier type of bias is in the dataset itself, but once again, this would require a specific effort and intention to produce that bias by modifying the collection or filtering procedure. Nothing as much has been alleged, so you have provided no evidence of bias, ‘systemic’ or otherwise. Come up with a different word. Maybe “non-uniform variance”.

pfannkuchen 3y ago

> the classifier is just reflecting that because it was trained on public data Are you suggesting that the cause here is that there are a lot of racist images on the internet visually depicting black people as gorillas? I assume that is not the understanding of most people here who are questioning what makes this racist, because if that is indeed the cause then I agree it is obviously racist. Is it the cause though? I would be interested to see some evidence or rationale, because I don’t think I have ever seen an image like this outside of perhaps a handful of very old hand drawn propaganda images (which presumably would not affect the classifier much due to being overwhelmed by the vast number of images of real people and real gorillas).

jl2718 3y ago

There is a rigorous definition of bias, and that isn’t it. That is called ‘variance’. It doesn’t matter if there is structure to the variance. It doesn’t matter if the model is consistently wrong for one type of input. It is not bias unless it has been specifically and intentionally programmed to predict something other than the result from the maximum likelihood estimator given the data. A trickier type of bias is in the dataset itself, but once again, this would require a specific effort and intention to produce that bias by modifying the collection or filtering procedure. Nothing as much has been alleged, so you have provided no evidence of bias, ‘systemic’ or otherwise. Come up with a different word. Maybe “non-uniform variance”.