4 ms·

I'm not convinced the article established the computer program "definitely is" racially biased. The claim was thrown out there and then not substantiated. I g

by Rapzid 12y ago

I'm not convinced the article established the computer program "definitely is" racially biased. The claim was thrown out there and then not substantiated. I guess the article needed to be racially charged..

pron 12y ago
This is because you (like manny others) are unfamiliar with what racism means. Racism (when applied to processes rather than individuals) means practices -- intentional or not -- that result in an unequal distribution of power among races. If a computer program perpetuates or increases the unequal distribution of power among races, then that program is racist. The same definition, BTW, applies to sexism, where that refers to unequal distribution of power between the sexes.
- yellowapple 12y ago
  But what about that program is actually racist? The first paragraph establishes that the program bases its decisions on indicators of financial responsibility. It then makes a decision based (apparently) on race alone. This would require race to be a recorded indicator of financial responsibility as decided by the programmer (or his/her employer). Had race not been recorded at all by the computer, then it would not have made a "racist" decision (for it would be based entirely on individual merit, rather than any particular connections with others of a specific demographic), and this article would therefore be turned over onto its head and rendered entirely moot.
  - pron 12y ago
    This is a bit of a philosophical question, and I don't think you should take it too literally, but here's what I think. If race is one of the variables, then due to correlations created by society, race might be learned to be a negative signal, which is racist. If it isn't one of the variables, it might be still inferred. For example, if your purchasing history is an input, there is probably a correlation between purchase of cosmetics for blacks and chances of incarceration, which, in turn affects your chances of loan repayment. The correlation between being black and buying, say, hair-care products for blacks is direct and causal (and completely non racist -- there is no effect on power), so this could be an almost perfect, and neutral, proxy for race, and, in fact, might be the best and simplest correlation the program could come up with.
forgottenpass 12y ago
It certainly made me roll my eyes, but I've read enough Slate to know what they mean. The software makes decisions on metrics casually correlated with race. Basically it's the codification of institutional racism. Some contemporary political trends refer to "institutional racism" as just "racism," which causes other readers/listeners to hear "prejudice/bigotry racism." By skipping a few steps in the causality chain and playing with definitions it is possible to make eye-grabbing statements. "Computers are racially biased" sounds WAY more sinister than the (not much) longer explanation. The eye-popping version is at best a hook to get people to care about an issue. At worst the in-group signaling about tangential issues can cause something worth broadly considering to be thoughtlessly bundled up into existing narratives and political groups. Then nobody will care about the issue itself, but instead whatever their bandwagon has pre-determined to be the opinion to have. FWIW: I think there is a lot to be considered about unintended consequences of removing humans from decision loops. After reading the intro to that Slate article, I'm pretty sure it isn't happening there and closed the tab.

by Rapzid 12y ago

pron 12y ago

This is because you (like manny others) are unfamiliar with what racism means. Racism (when applied to processes rather than individuals) means practices -- intentional or not -- that result in an unequal distribution of power among races. If a computer program perpetuates or increases the unequal distribution of power among races, then that program is racist. The same definition, BTW, applies to sexism, where that refers to unequal distribution of power between the sexes.

yellowapple 12y ago
But what about that program is actually racist? The first paragraph establishes that the program bases its decisions on indicators of financial responsibility. It then makes a decision based (apparently) on race alone. This would require race to be a recorded indicator of financial responsibility as decided by the programmer (or his/her employer). Had race not been recorded at all by the computer, then it would not have made a "racist" decision (for it would be based entirely on individual merit, rather than any particular connections with others of a specific demographic), and this article would therefore be turned over onto its head and rendered entirely moot.
- pron 12y ago
  This is a bit of a philosophical question, and I don't think you should take it too literally, but here's what I think. If race is one of the variables, then due to correlations created by society, race might be learned to be a negative signal, which is racist. If it isn't one of the variables, it might be still inferred. For example, if your purchasing history is an input, there is probably a correlation between purchase of cosmetics for blacks and chances of incarceration, which, in turn affects your chances of loan repayment. The correlation between being black and buying, say, hair-care products for blacks is direct and causal (and completely non racist -- there is no effect on power), so this could be an almost perfect, and neutral, proxy for race, and, in fact, might be the best and simplest correlation the program could come up with.

forgottenpass 12y ago

It certainly made me roll my eyes, but I've read enough Slate to know what they mean. The software makes decisions on metrics casually correlated with race. Basically it's the codification of institutional racism. Some contemporary political trends refer to "institutional racism" as just "racism," which causes other readers/listeners to hear "prejudice/bigotry racism." By skipping a few steps in the causality chain and playing with definitions it is possible to make eye-grabbing statements. "Computers are racially biased" sounds WAY more sinister than the (not much) longer explanation. The eye-popping version is at best a hook to get people to care about an issue. At worst the in-group signaling about tangential issues can cause something worth broadly considering to be thoughtlessly bundled up into existing narratives and political groups. Then nobody will care about the issue itself, but instead whatever their bandwagon has pre-determined to be the opinion to have. FWIW: I think there is a lot to be considered about unintended consequences of removing humans from decision loops. After reading the intro to that Slate article, I'm pretty sure it isn't happening there and closed the tab.