4 ms·

Are there people programming their "algorithms" with flags that say "if black, fire them"???

TonyTrapp 5y ago
That's not even necessary. Some of those -isms creep in implicitly, because e.g. certain performance metrics assumed by the algorithm may be statistically less likely to be fulfilled by women, or stuff like that.
- NalNezumi 5y ago
  Doesn't this make it an intractable problem though? Attribute X is more common in group Y first require a population based study, which can be tampered left and right and doesn't always stay the same over time. (commonality of the attribute can shift with generations) Besides if the goal is "make an algorithm that only base its judgment on population studies" all human decision making algorithms will be the same, and you can endlessly cry wolf.
lostcolony 5y ago
Probably not, but so what? It happens all the time, because the non-racist logic still affects people unequally due to racial reasons. Here's the top result from "racist algorithms" in Google - https://www.nature.com/articles/d41586-019-03228-6 https://www.nature.com/articles/d41586-019-03228-6 TL;DR: One of the factors for referring people to treatment was doctor visits over the last year. Makes sense; unwell people go to the doctor, so those going to the doctor more often are more likely to be unwell. But, of course, due to historical segregation, generational wealth, etc, black people on average tended to go to the doctor less. So even if they exhibited the same risks, symptoms, etc, as white people, because they saw a doctor about it less often the algorithm was less likely to refer them for treatment.
scottLobster 5y ago
No, by they might filter for traits like less likely to commit a crime. Given that African Americans commit roughly half of all crime in America (last time I read the FBI data), such an algorithm would get really racist, really fast if left to itself.
- pseudalopex 5y ago
  Black people made up 26.6% of 2019 arrests according to FBI data.[1] And arrests and crimes aren't the same. Few wage thieves are arrested for example. [1] https://ucr.fbi.gov/crime-in-the-u.s/2019/crime-in-the-u.s.-2019/topic-pages/tables/table-43 https://ucr.fbi.gov/crime-in-the-u.s/2019/crime-in-the-u.s.-...
AlexandrB 5y ago
I mean, kind of. If you develop an algorithm and the user you're picturing in your mind looks like you, talks like you, and lives like you do you're embedding all kind of gender-related, racial, and socioeconomic biases. The most famous example being automatic soap dispensers that don't detect black hands[1]. Now combine that fact with the demographics of Silicon Valley companies and you get a recipe for products and algorithms that are "fair" only if you match a set of implicit criteria. None of this is intentionally malevolent, just a product of hubris and "move fast and break things". [1] https://reporter.rit.edu/tech/bigotry-encoded-racial-bias-technology https://reporter.rit.edu/tech/bigotry-encoded-racial-bias-te...
- TheFreim 5y ago
  Doesn't having someone who doesn't speak like you, doesn't share your culture, etc make working together worse and therefore not as good of a match? If I went to China and tried to get a job I wouldn't fault them for choosing someone or equal qualification who's also Chinese since by that very fact they'd be a better team fit. Does that make sense?

That's not even necessary. Some of those -isms creep in implicitly, because e.g. certain performance metrics assumed by the algorithm may be statistically less likely to be fulfilled by women, or stuff like that.

NalNezumi 5y ago
Doesn't this make it an intractable problem though? Attribute X is more common in group Y first require a population based study, which can be tampered left and right and doesn't always stay the same over time. (commonality of the attribute can shift with generations) Besides if the goal is "make an algorithm that only base its judgment on population studies" all human decision making algorithms will be the same, and you can endlessly cry wolf.

lostcolony 5y ago

Probably not, but so what? It happens all the time, because the non-racist logic still affects people unequally due to racial reasons. Here's the top result from "racist algorithms" in Google - https://www.nature.com/articles/d41586-019-03228-6 https://www.nature.com/articles/d41586-019-03228-6 TL;DR: One of the factors for referring people to treatment was doctor visits over the last year. Makes sense; unwell people go to the doctor, so those going to the doctor more often are more likely to be unwell. But, of course, due to historical segregation, generational wealth, etc, black people on average tended to go to the doctor less. So even if they exhibited the same risks, symptoms, etc, as white people, because they saw a doctor about it less often the algorithm was less likely to refer them for treatment.

scottLobster 5y ago

No, by they might filter for traits like less likely to commit a crime. Given that African Americans commit roughly half of all crime in America (last time I read the FBI data), such an algorithm would get really racist, really fast if left to itself.

pseudalopex 5y ago
Black people made up 26.6% of 2019 arrests according to FBI data.[1] And arrests and crimes aren't the same. Few wage thieves are arrested for example. [1] https://ucr.fbi.gov/crime-in-the-u.s/2019/crime-in-the-u.s.-2019/topic-pages/tables/table-43 https://ucr.fbi.gov/crime-in-the-u.s/2019/crime-in-the-u.s.-...

AlexandrB 5y ago

I mean, kind of. If you develop an algorithm and the user you're picturing in your mind looks like you, talks like you, and lives like you do you're embedding all kind of gender-related, racial, and socioeconomic biases. The most famous example being automatic soap dispensers that don't detect black hands[1]. Now combine that fact with the demographics of Silicon Valley companies and you get a recipe for products and algorithms that are "fair" only if you match a set of implicit criteria. None of this is intentionally malevolent, just a product of hubris and "move fast and break things". [1] https://reporter.rit.edu/tech/bigotry-encoded-racial-bias-technology https://reporter.rit.edu/tech/bigotry-encoded-racial-bias-te...

TheFreim 5y ago
Doesn't having someone who doesn't speak like you, doesn't share your culture, etc make working together worse and therefore not as good of a match? If I went to China and tried to get a job I wouldn't fault them for choosing someone or equal qualification who's also Chinese since by that very fact they'd be a better team fit. Does that make sense?