5 ms·

Not sure what the author meant there but we do have a problem of offloading the ethical burden of our systems onto the algorithms themselves and it should be ta

Not sure what the author meant there but we do have a problem of offloading the ethical burden of our systems onto the algorithms themselves and it should be taken more seriously. Like when Google shipped an AI for moderating hate speech which turned out to be biased against minorities [1]. Google would have been sued if that bias had been coded in procedurally, but because they used "AI" their response was basically "we'll take another look at the training data". The damage caused is the same in both cases though, and it shouldn't take a team of independent researchers to prevent Google from shipping racist products. [1] https://fortune.com/2019/08/16/google-jigsaw-perspective-racial-bias/ https://fortune.com/2019/08/16/google-jigsaw-perspective-rac...

ekianjo 6y ago
> The damage caused is the same in both cases though, and it shouldn't take a team of independent researchers to prevent Google from shipping racist products. Your comment reads like "company X should not be shipping products with issues affecting users" - as long as it was not intentional and they take corrective actions, things are working as well as they can.
- coddle-hark 6y ago
  With that logic we shouldn't need clinical trials for drugs or food additives either. Just ship it, if someone gets hurt we'll sort it out later! At this point it's well known that unfair biases can easily creep into these kind of systems. I don't think it's unreasonable to require products that could potentially cause much harm (e.g. online censoring, scoring CVs, vetting loan applications) to be tested for fairness before letting them loose on the public.
smegma2 6y ago
Forgive me for only reading the abstract. > The results show evidence of systematic racial bias in all datasets, as classifiers trained on them tend to predict that tweets written in African-American English are abusive at substantially higher rates. Wouldn't we need to know the true rate of abusive tweets by race in order to conclude this? Is this addressed at all, e.g. with human labelled data, or am I mistaken in thinking this is necessary?
- SpicyLemonZest 6y ago
  Your core idea is true, but such analyses are typically not done because the answer isn't considered relevant. An advocate would argue that the concept of "abusive" is culturally relative, so we must draw the line of abusiveness in a way that avoids disparate impact.

ekianjo 6y ago

> The damage caused is the same in both cases though, and it shouldn't take a team of independent researchers to prevent Google from shipping racist products. Your comment reads like "company X should not be shipping products with issues affecting users" - as long as it was not intentional and they take corrective actions, things are working as well as they can.

coddle-hark 6y ago
With that logic we shouldn't need clinical trials for drugs or food additives either. Just ship it, if someone gets hurt we'll sort it out later! At this point it's well known that unfair biases can easily creep into these kind of systems. I don't think it's unreasonable to require products that could potentially cause much harm (e.g. online censoring, scoring CVs, vetting loan applications) to be tested for fairness before letting them loose on the public.

smegma2 6y ago

Forgive me for only reading the abstract. > The results show evidence of systematic racial bias in all datasets, as classifiers trained on them tend to predict that tweets written in African-American English are abusive at substantially higher rates. Wouldn't we need to know the true rate of abusive tweets by race in order to conclude this? Is this addressed at all, e.g. with human labelled data, or am I mistaken in thinking this is necessary?

SpicyLemonZest 6y ago
Your core idea is true, but such analyses are typically not done because the answer isn't considered relevant. An advocate would argue that the concept of "abusive" is culturally relative, so we must draw the line of abusiveness in a way that avoids disparate impact.