3 ms·

I did not make a blanket statement that all ML algos are racist against Asians. My point was to convey that if the training data do not accurately represent the

by muraiki 10y ago

yummyfajitas 10y ago
I did not make a blanket statement that all ML algos are racist against Asians. My point was to convey that if the training data do not accurately represent the population, the algorithm will carry the biases of that inaccurate representation: the algorithm is only as good as the data it is trained upon (for supervised learning). But that's not true. Inadequate training data will cause variance, not bias. This variance can have any direction. In a linear prediction scenario (i.e. any go/no go decision process), which exactly what Cathy O'Neil is criticizing, the effect could just as easily be "more loans for black people" as "less loans for black people". In a multidimensional predictor, it does directly decrease accuracy but that's all. "Black guy -> gorilla" is just as likely as "yummyfajitas -> tank". Also, realistically, if we want to think about your black guy -> gorilla example, most likely if insufficient training data was the problem, then it was the number of gorillas that was too few. Google has vastly more black humans in their training set than gorillas - a quick google search suggests there are only 100,000 gorillas in the world, most of whom are probably unphotographed. As a sibling comment stated, humans are deeply involved in all aspects of designing and interpreting an algorithm's results. To the extent that we are flawed, so will our algorithms be. Sure, algorithms are flawed. But this does not imply that bias must be present proportionally to ours. Insofar as we do good statistics, human bias is mitigated and can even go away. Science works. As a classical example, consider Morton's analysis of human skulls. Morton was a super racist guy trying to prove how white people are superior. But he did an unbiased analysis and accurately measured skull volume - filling a skull with buckshot is a pretty unbiased process. http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1001071 http://journals.plos.org/plosbiology/article?id=10.1371/jour... I am frankly confused as to what is so controversial about the statement that algorithms created by humans are not free from human biases. What's controversial is that this is the idea of "original sin" but applied to science. And most of the proposed mechanisms are simply mathematically wrong. Also many of the examples cited by proponents of this original sin concept are also wrong.
- muraiki 10y ago
  I appreciate your contribution to this discussion and will think about it further. Thanks.
- thaw13579 10y ago
  Thanks for the interesting article. Although, from my reading, it seems to support the importance of both data handling and statistics in avoiding bias. That is, a bias in the dataset (or which subset is analyzed) would subsequently bias the statistical results (Box 1 and 2). Regarding big data and ML, I guess the question is then what kind of biases exist in data collection, something that seems dependent on the details and history each particular dataset.

by muraiki 10y ago

yummyfajitas 10y ago

I did not make a blanket statement that all ML algos are racist against Asians. My point was to convey that if the training data do not accurately represent the population, the algorithm will carry the biases of that inaccurate representation: the algorithm is only as good as the data it is trained upon (for supervised learning). But that's not true. Inadequate training data will cause variance, not bias. This variance can have any direction. In a linear prediction scenario (i.e. any go/no go decision process), which exactly what Cathy O'Neil is criticizing, the effect could just as easily be "more loans for black people" as "less loans for black people". In a multidimensional predictor, it does directly decrease accuracy but that's all. "Black guy -> gorilla" is just as likely as "yummyfajitas -> tank". Also, realistically, if we want to think about your black guy -> gorilla example, most likely if insufficient training data was the problem, then it was the number of gorillas that was too few. Google has vastly more black humans in their training set than gorillas - a quick google search suggests there are only 100,000 gorillas in the world, most of whom are probably unphotographed. As a sibling comment stated, humans are deeply involved in all aspects of designing and interpreting an algorithm's results. To the extent that we are flawed, so will our algorithms be. Sure, algorithms are flawed. But this does not imply that bias must be present proportionally to ours. Insofar as we do good statistics, human bias is mitigated and can even go away. Science works. As a classical example, consider Morton's analysis of human skulls. Morton was a super racist guy trying to prove how white people are superior. But he did an unbiased analysis and accurately measured skull volume - filling a skull with buckshot is a pretty unbiased process. http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1001071 http://journals.plos.org/plosbiology/article?id=10.1371/jour... I am frankly confused as to what is so controversial about the statement that algorithms created by humans are not free from human biases. What's controversial is that this is the idea of "original sin" but applied to science. And most of the proposed mechanisms are simply mathematically wrong. Also many of the examples cited by proponents of this original sin concept are also wrong.

muraiki 10y ago
I appreciate your contribution to this discussion and will think about it further. Thanks.
thaw13579 10y ago
Thanks for the interesting article. Although, from my reading, it seems to support the importance of both data handling and statistics in avoiding bias. That is, a bias in the dataset (or which subset is analyzed) would subsequently bias the statistical results (Box 1 and 2). Regarding big data and ML, I guess the question is then what kind of biases exist in data collection, something that seems dependent on the details and history each particular dataset.