4 ms·

I just skimmed both sources you list and Ctrl-F "bias" seems to indicate they assume unbiased noise. Which might be okay depending on the application, but for

I just skimmed both sources you list and Ctrl-F "bias" seems to indicate they assume unbiased noise. Which might be okay depending on the application, but for many data collection processes will not be okay. Imagine if one group of people's wealth is systematically under-estimated (aka "measured") when training an insurance policy AI. The algorithm will then correctly learn the bias in the dataset. None of this is magic. Data collection is hard. Spotting your own biases before they make it into your dataset is a blind-spot exercise.

skrebbel 6y ago
I'm not trying to refute the problem of biased AI data, but I do want to understand it better. You mention an "insurance policy AI". The rest of this thread seems to be about image recognition. What's an insurance policy AI and how does it work? Is it a thing or a hypothetical future thing? Can bias effects have equally bad effects in image recognition? I know about the story of the photo software that categorized a man's holiday photos under "gorillas" because it was trained only on white people. This is terribly insulting, but less terrible than an insurance company unfairly overcharging you or refusing you service. I guess what I'm saying is I can't come up with biased image recognition AIs having unfair outcomes that actually affect people deeply, and I'm likely missing lots of terrible examples, and I'd love if someone can educate me. EDIT: before people erupt in outrage, I'm not saying that a computer telling you that you're a gorilla isn't terrible, but in the end it's an indictment of the software, not you.
- whynaut 6y ago
  If that software can’t distinguish a black man from a gorilla, i’m sure it’ll have trouble distinguishing two black men at least some of the time.
- lanstin 6y ago
  Pulling people for extra checks in a security screening use case (from an image get an emotional state rating). Not hitting pedestrians in a self driving car. Also a lot of image scanning applications are being sold as “upload a photo of a prospective employee and a get a trust worthiness rating”.

skrebbel 6y ago

I'm not trying to refute the problem of biased AI data, but I do want to understand it better. You mention an "insurance policy AI". The rest of this thread seems to be about image recognition. What's an insurance policy AI and how does it work? Is it a thing or a hypothetical future thing? Can bias effects have equally bad effects in image recognition? I know about the story of the photo software that categorized a man's holiday photos under "gorillas" because it was trained only on white people. This is terribly insulting, but less terrible than an insurance company unfairly overcharging you or refusing you service. I guess what I'm saying is I can't come up with biased image recognition AIs having unfair outcomes that actually affect people deeply, and I'm likely missing lots of terrible examples, and I'd love if someone can educate me. EDIT: before people erupt in outrage, I'm not saying that a computer telling you that you're a gorilla isn't terrible, but in the end it's an indictment of the software, not you.

whynaut 6y ago
If that software can’t distinguish a black man from a gorilla, i’m sure it’ll have trouble distinguishing two black men at least some of the time.
lanstin 6y ago
Pulling people for extra checks in a security screening use case (from an image get an emotional state rating). Not hitting pedestrians in a self driving car. Also a lot of image scanning applications are being sold as “upload a photo of a prospective employee and a get a trust worthiness rating”.