4 ms·

I'm confused and slightly alarmed by the insistence on doing everything possible to scrub the prospect of human agency from matters that affected by and that ef

by bsanr2 5y ago

I'm confused and slightly alarmed by the insistence on doing everything possible to scrub the prospect of human agency from matters that affected by and that effect humans. If we see some social ill, shouldn't be combing the systems that are related to thei existence for the flaws that let those ills happen, instead of just pointing at the system and saying, "Well, a human didn't make that specific decision, so it must be objective and fair." It's high-tech ventriloquism.

SpicyLemonZest 5y ago
I think you're misunderstanding the concern. As seen in stories like the x-ray race detection thing (https://www.wired.com/story/these-algorithms-look-x-rays-detect-your-race/ https://www.wired.com/story/these-algorithms-look-x-rays-det...), there's a lot of people in the AI field who believe that any correlation with race in a model is a "bias" which should be corrected. If a radiology AI model stumbles across some unknown condition like sickle cell anemia with a strong racial correlation, I think we should learn about it, and I worry that under the kinds of practices the source article suggests it would just be bias adjusted away.
- bsanr2 5y ago
  You're misunderstanding the article. It's talking about a situation where an AI model replicated the unscientific classification of people by race. That means that, far from uncovering race-correlated health issues, it could perpetuate unscientific and damaging associations that could put people's health at risk: for example, throwing out disease diagnoses commonly associated with white people that show up for a black person. Of course, the central issue is that we wouldn't know why the model came to any portion of its conclusions. Additionally, you're privileging possible advantages over concrete and known issues. That's the opposite of risk mitigation.
  - SpicyLemonZest 5y ago
    Respectfully, you're the one who's misunderstood, and in a way that really kinda proves my point. There's nothing "unscientific" about classifying people by race, and the entire study methodology depended on the researchers doing so - they took some data sets which had been pre-classified by race, and trained the AI on those data sets, in order to prove that it could reproduce the known classification based only on the medical scans. You can read their preprint (https://arxiv.org/ftp/arxiv/papers/2107/2107.10356.pdf https://arxiv.org/ftp/arxiv/papers/2107/2107.10356.pdf) for full details.
    - bsanr2 5y ago
      Race is a socially-constructed attribute, not a biological one. To purport to be able to base medical diagnoses on race is already an issue, and already causes much suffering. Specifically, diagnoses are missed when doctors believe that people of certain races are unable to develop certain diseases. Moving from individuals, who can learn about the history of racism in medicine and make positive changes to their outlook, to AI models, which can only produce output based on past (biased) data, is a nightmare scenario.

by bsanr2 5y ago

SpicyLemonZest 5y ago

I think you're misunderstanding the concern. As seen in stories like the x-ray race detection thing (https://www.wired.com/story/these-algorithms-look-x-rays-detect-your-race/ https://www.wired.com/story/these-algorithms-look-x-rays-det...), there's a lot of people in the AI field who believe that any correlation with race in a model is a "bias" which should be corrected. If a radiology AI model stumbles across some unknown condition like sickle cell anemia with a strong racial correlation, I think we should learn about it, and I worry that under the kinds of practices the source article suggests it would just be bias adjusted away.

bsanr2 5y ago
You're misunderstanding the article. It's talking about a situation where an AI model replicated the unscientific classification of people by race. That means that, far from uncovering race-correlated health issues, it could perpetuate unscientific and damaging associations that could put people's health at risk: for example, throwing out disease diagnoses commonly associated with white people that show up for a black person. Of course, the central issue is that we wouldn't know why the model came to any portion of its conclusions. Additionally, you're privileging possible advantages over concrete and known issues. That's the opposite of risk mitigation.
- SpicyLemonZest 5y ago
  Respectfully, you're the one who's misunderstood, and in a way that really kinda proves my point. There's nothing "unscientific" about classifying people by race, and the entire study methodology depended on the researchers doing so - they took some data sets which had been pre-classified by race, and trained the AI on those data sets, in order to prove that it could reproduce the known classification based only on the medical scans. You can read their preprint (https://arxiv.org/ftp/arxiv/papers/2107/2107.10356.pdf https://arxiv.org/ftp/arxiv/papers/2107/2107.10356.pdf) for full details.
  - bsanr2 5y ago
    Race is a socially-constructed attribute, not a biological one. To purport to be able to base medical diagnoses on race is already an issue, and already causes much suffering. Specifically, diagnoses are missed when doctors believe that people of certain races are unable to develop certain diseases. Moving from individuals, who can learn about the history of racism in medicine and make positive changes to their outlook, to AI models, which can only produce output based on past (biased) data, is a nightmare scenario.