3 ms·
Am I reading this correctly, they are saying people are bad at doing on-the-fly statistical analysis to conclude whether a system is biased? For example in one
by andy99 1y ago
Am I reading this correctly, they are saying people are bad at doing on-the-fly statistical analysis to conclude whether a system is biased?
For example in one case they showed data where sad faces were “mostly” black and asked people if they detect “bias”. Even if you saw more sad black people than white, would you reject the null hypothesis that it’s unbiased?
This unfortunately seems typical of the often very superficial “count the races” work that people claim is bias research.
- kmacdough 1y agoThis seems to ignore most of the experiments in this study. Note that also studied much more extreme distributions including only happy/white and sad/black. Even in these cases of extreme bias, the bias went unnoticed. You hyper-focus on only one of dozens of experiments in this study for your criticism. Very straw-man.
- card_zero 1y agoAre you sure that the bias went unnoticed? The article says "most participants in their experiments only started to notice bias when the AI showed biased performance", which I understand to mean they noticed the bias in the experiment you're talking about. Have I got that wrong? Is it written wrong? Do we even have the means to check?