4 ms·

No, they're not[0]. Correlation is not causation, and this article makes many such correlations. It also appeals heavily to fairness. Business is not about f

No, they're not[0]. Correlation is not causation, and this article makes many such correlations. It also appeals heavily to fairness. Business is not about fairness, it is about profit. I don't see the potential profits in racism - if it is possible to provide services to people profitably, it will be done. Arbitrary clusterings such as zip codes, while certainly statistically significant, do not translate to intent by algorithms or algorithm makers. Fairness is somewhat subjective, and I think it's fair for companies to pursue profits such as "same day shipping", even if that cannot be simultaneously rolled out to all markets. If it is profitable, eventually, it will be. [0] Invoking Betteridge's Law of Headlines ( https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline... )

lallysingh 9y ago
"At the center of Northpointe’s argument was an essentialist fallacy: Because persons that police classified as Afro-Americans were re-arrested more often in the training dataset, they claimed, COMPAS is justified in predicting that other persons classified as Afro-Americans by police—even in a different city, state, and time period—are more likely to be re-arrested. The cycling of classification into data then back into classification echoes W.E.B. Dubois’ 1923 definition, “the black man is the man who has to ride Jim Crow in Georgia.”" Sounds like the answer is yes.
- Ygg2 9y ago
  Then I would not say they are building new infrastructure of racism. They are getting garbage input and returning garbage output.
  - Cthulhu_ 9y ago
    The accusation here is that they're maintaining the status quo, in a way, instead of working on making things better.
    - Ygg2 9y ago
      True, but the research they are doing - "improving" language to reduce bias, reeks to me of 1984's Doublespeak. What are the chances this won't be co-opted by the other side, to enhance divisions?
      - mkane848 9y ago
        You're right, let's just do nothing and let the problem get even worse as we push blame onto an algorithm rather than deeply ingrained societal issues
        Ygg2 9y ago
        I think you are misinterpreting my statement. I never said blame algorithm. I just said, what happens if algorithm (for eliminating some thought patterns using language inference) is used by the other side?
  - aninhumer 9y ago
    The problem which the quote is pointing to here, is that then people use the old garbage to justify the new reprocessed garbage and vice versa, and it becomes harder for people to realise that it's garbage.
- merlincorey 9y ago
  How does that answer the question as an exclusive yes without any room for no?
throwawayReply 9y ago
> Correlation is not causation That's particularly ironic, since a lot of ML can treat it that way.
jacobinofnone 9y ago
"... do not translate to intent ..." intention is orthogonal to structural bias from the last section of the article: "infrastructure beats intent"
shaki-dora 9y ago
What a tired, lazy cliché... Why don’t you engage with the content of the article: But when COMPAS’ prediction was inaccurate (either predicting re-arrest when none happened, or not predicting an actual re-arrest) it routinely underestimated the probability of white recidivism and over-estimated the probability of black recidivism. In other words, it contained a bias hidden from the perspective of one set of statistics, but plainly visible in another.. Edit, because you noticed how unhelpful “ha ha, Betteridge” by itself is: intent may not be necessary for discrimination, nor can you deny intent when you continue to use methods shown to produce discriminatory outcomes. A profit motive is similarly useless as a defense of discrimination. Maybe young, white, heterosexual female employees are better for sales, because you operate in a xeno-, homo-, gymno-, gerontophobic neigbourhood. But it’s no excuse to limit your job ads to that demographic. One reason is that such logic perpetuates its own causes: if the police is more likely to stop&frisk non-whites, you will have fewer whites in your arrest statistics, even if they commit the same number of crimes. To then turn around and use those statistics to justify your racist police procedures is both morally and scientifically bankrupt.
- merlincorey 9y ago
  I feel like intent is necessary for racial discrimination. To discriminate between racial discrimination and mere ignorance or ineptitude one must look at the intent.
  - aalleavitch 9y ago
    Why would that matter if the outcome is the same? People are being disadvantaged solely on the basis of race, whether this is being done intentionally or not it's harmful and needs to be rectified.
  - shaki-dora 9y ago
    Why should intent matter? These are not criminal cases, but civil. If an employer is discriminating against a group because of unconscious biases, or using methods that lead to Discriminstory outcomes, there are real people who were harmed. Civil law isn’t about morality. It’s about righting a wrong. If you die because a mechanic negligently forgot to reconnect your car’s brake lines, you (and your family) will be harmed. The question for the courts then becomes shifting that harm to the party responsible, as far as that is possible, and your family will likely be awarded damages.
perfmode 9y ago
You use scientific buzzwords, but you’re coming into the conversation with unscientific assumptions.
sgift 9y ago
> Business is not about fairness, it is about profit. I don't see the potential profits in racism - if it is possible to provide services to people profitably, it will be done. That assumes companies are completely logical, singular entities which all follow the same motivation without deviation. As companies are the sum of the humans working there with all their personal problems, faults, opinions and attitudes (plus the organizational framework itself) that's rather unlikely. It also assumes companies always know beforehand what the best strategy is to make more profit (just look at VW for a recent example where that didn't pan out).
EliRivers 9y ago
Business is not about fairness, it is about profit. Yet historically, a number of businesses in various parts chose not to serve certain sections of the population, and sometimes still do (off the top of my head, wedding cake companies choosing not to sell to gay couples).

lallysingh 9y ago

"At the center of Northpointe’s argument was an essentialist fallacy: Because persons that police classified as Afro-Americans were re-arrested more often in the training dataset, they claimed, COMPAS is justified in predicting that other persons classified as Afro-Americans by police—even in a different city, state, and time period—are more likely to be re-arrested. The cycling of classification into data then back into classification echoes W.E.B. Dubois’ 1923 definition, “the black man is the man who has to ride Jim Crow in Georgia.”" Sounds like the answer is yes.

Ygg2 9y ago
Then I would not say they are building new infrastructure of racism. They are getting garbage input and returning garbage output.
- Cthulhu_ 9y ago
  The accusation here is that they're maintaining the status quo, in a way, instead of working on making things better.
  - Ygg2 9y ago
    True, but the research they are doing - "improving" language to reduce bias, reeks to me of 1984's Doublespeak. What are the chances this won't be co-opted by the other side, to enhance divisions?
    - mkane848 9y ago
      You're right, let's just do nothing and let the problem get even worse as we push blame onto an algorithm rather than deeply ingrained societal issues
      - Ygg2 9y ago
        I think you are misinterpreting my statement. I never said blame algorithm. I just said, what happens if algorithm (for eliminating some thought patterns using language inference) is used by the other side?
- aninhumer 9y ago
  The problem which the quote is pointing to here, is that then people use the old garbage to justify the new reprocessed garbage and vice versa, and it becomes harder for people to realise that it's garbage.
merlincorey 9y ago
How does that answer the question as an exclusive yes without any room for no?

throwawayReply 9y ago

> Correlation is not causation That's particularly ironic, since a lot of ML can treat it that way.

jacobinofnone 9y ago

"... do not translate to intent ..." intention is orthogonal to structural bias from the last section of the article: "infrastructure beats intent"

shaki-dora 9y ago

What a tired, lazy cliché... Why don’t you engage with the content of the article: But when COMPAS’ prediction was inaccurate (either predicting re-arrest when none happened, or not predicting an actual re-arrest) it routinely underestimated the probability of white recidivism and over-estimated the probability of black recidivism. In other words, it contained a bias hidden from the perspective of one set of statistics, but plainly visible in another.. Edit, because you noticed how unhelpful “ha ha, Betteridge” by itself is: intent may not be necessary for discrimination, nor can you deny intent when you continue to use methods shown to produce discriminatory outcomes. A profit motive is similarly useless as a defense of discrimination. Maybe young, white, heterosexual female employees are better for sales, because you operate in a xeno-, homo-, gymno-, gerontophobic neigbourhood. But it’s no excuse to limit your job ads to that demographic. One reason is that such logic perpetuates its own causes: if the police is more likely to stop&frisk non-whites, you will have fewer whites in your arrest statistics, even if they commit the same number of crimes. To then turn around and use those statistics to justify your racist police procedures is both morally and scientifically bankrupt.

merlincorey 9y ago
I feel like intent is necessary for racial discrimination. To discriminate between racial discrimination and mere ignorance or ineptitude one must look at the intent.
- aalleavitch 9y ago
  Why would that matter if the outcome is the same? People are being disadvantaged solely on the basis of race, whether this is being done intentionally or not it's harmful and needs to be rectified.
- shaki-dora 9y ago
  Why should intent matter? These are not criminal cases, but civil. If an employer is discriminating against a group because of unconscious biases, or using methods that lead to Discriminstory outcomes, there are real people who were harmed. Civil law isn’t about morality. It’s about righting a wrong. If you die because a mechanic negligently forgot to reconnect your car’s brake lines, you (and your family) will be harmed. The question for the courts then becomes shifting that harm to the party responsible, as far as that is possible, and your family will likely be awarded damages.

perfmode 9y ago

You use scientific buzzwords, but you’re coming into the conversation with unscientific assumptions.

sgift 9y ago

> Business is not about fairness, it is about profit. I don't see the potential profits in racism - if it is possible to provide services to people profitably, it will be done. That assumes companies are completely logical, singular entities which all follow the same motivation without deviation. As companies are the sum of the humans working there with all their personal problems, faults, opinions and attitudes (plus the organizational framework itself) that's rather unlikely. It also assumes companies always know beforehand what the best strategy is to make more profit (just look at VW for a recent example where that didn't pan out).

EliRivers 9y ago

Business is not about fairness, it is about profit. Yet historically, a number of businesses in various parts chose not to serve certain sections of the population, and sometimes still do (off the top of my head, wedding cake companies choosing not to sell to gay couples).