4 ms·

When talking about something like crime rates by race, your statement incorrectly assumes society enforces laws fairly/equally. For example, based on the number

When talking about something like crime rates by race, your statement incorrectly assumes society enforces laws fairly/equally. For example, based on the numbers alone, black people are much more likely to commit drug-related crimes than white people. However, studies show that black people are stopped a lot more often and have unequal sentencing outcomes. It's important to remember that AI doesn't have context, and just because it's using "data" to make decisions doesn't mean the decisions are unbiased - the underlying data may be biased.

geofft 8y ago
One of the things I'm rather worried about here is AI's ability to learn on the most unlikely (to human cognition) features. Take the image recognizer that finds "sheep" by seeing pastures where it thinks sheep might graze http://aiweirdness.com/post/172894792687/when-algorithms-surprise-us http://aiweirdness.com/post/172894792687/when-algorithms-sur... , or take any of the adversarial perturbing examples like https://blog.openai.com/robust-adversarial-inputs/ https://blog.openai.com/robust-adversarial-inputs/ . You're not allowed to discriminate on race in housing. But is an AI that determines your creditworthiness for mortgages allowed to discriminate on what you eat, where you go to church, what your favorite music is, etc.? Maybe it doesn't have that data, but it will have one level higher - what store credit cards you have and how much you use them and where you opened them. If you train an AI on a segregated city with a history of actively discriminatory citizens, where the few people of race X who moved into a not-race-X neighborhood got harassed out and sold before they paid off their mortgages, how easily will the AI conclude that people born in certain neighborhoods are more likely to pay off their mortgage if they avoid certain other neighborhoods? Is that illegal? (My guess is there's no way to prove to a court, to the court's usual standards, that the AI happened to learn the city's racial tensions.) Should it be illegal, if outright racial discrimination is illegal?
MichaelGG 8y ago
Homicide rates are unlikely to have this kind of bias though. And they're heavily against blacks. It's unlikely whites being murdered go unreported, or that anyone gets off with a warning. But inner city homicide has a low solving rate. If the reporting rate is also lower, then it could be even worse than the 4-7x overrepresentation of blacks in homicide. This seems to be the case in London, too, so it's a bigger issue than just the US. But for things like drug possession, yes, blacks probably get stopped much more and not let off with warnings much less.
- AstralStorm 8y ago
  Untangling cause from effect in this case is extremely hard. It is like with plagues. If you put many people with plague in one area do expect that others will catch it. If the rule to segregate is silly enough do expect a correlation of plague with certain characteristics of people put together and the locations. Or their sizes. Maybe it is "cities" or "presence of slums" not skin colour, combined with overrepresentation of people with certain skin colour in them. It takes some write genius analysis and experimentation to untangle such complex effects from causes. Call again when you have an AI that can deal with this. Essentially a researcher AI.
- gowld 8y ago
  "Blacks". Are homicide rates 4x-7x higher among "blacks" in Ben Carson's gated residential community? In Boca Raton and Beverly Hills? Or might their be other factors you are ignoring?
  - MichaelGG 8y ago
    In only responding to the claim that the stats are wrong because of police oversampling blacks. With homicide, no one can make that case. If lack of money causes homicides, we should expect to see all poor rural areas have high homicide rates, too. Maybe that's true. Another factor seems to be sex. Males commit so much more violence, so perhaps that's the only thing to focus on.

geofft 8y ago

One of the things I'm rather worried about here is AI's ability to learn on the most unlikely (to human cognition) features. Take the image recognizer that finds "sheep" by seeing pastures where it thinks sheep might graze http://aiweirdness.com/post/172894792687/when-algorithms-surprise-us http://aiweirdness.com/post/172894792687/when-algorithms-sur... , or take any of the adversarial perturbing examples like https://blog.openai.com/robust-adversarial-inputs/ https://blog.openai.com/robust-adversarial-inputs/ . You're not allowed to discriminate on race in housing. But is an AI that determines your creditworthiness for mortgages allowed to discriminate on what you eat, where you go to church, what your favorite music is, etc.? Maybe it doesn't have that data, but it will have one level higher - what store credit cards you have and how much you use them and where you opened them. If you train an AI on a segregated city with a history of actively discriminatory citizens, where the few people of race X who moved into a not-race-X neighborhood got harassed out and sold before they paid off their mortgages, how easily will the AI conclude that people born in certain neighborhoods are more likely to pay off their mortgage if they avoid certain other neighborhoods? Is that illegal? (My guess is there's no way to prove to a court, to the court's usual standards, that the AI happened to learn the city's racial tensions.) Should it be illegal, if outright racial discrimination is illegal?

MichaelGG 8y ago

Homicide rates are unlikely to have this kind of bias though. And they're heavily against blacks. It's unlikely whites being murdered go unreported, or that anyone gets off with a warning. But inner city homicide has a low solving rate. If the reporting rate is also lower, then it could be even worse than the 4-7x overrepresentation of blacks in homicide. This seems to be the case in London, too, so it's a bigger issue than just the US. But for things like drug possession, yes, blacks probably get stopped much more and not let off with warnings much less.

AstralStorm 8y ago
Untangling cause from effect in this case is extremely hard. It is like with plagues. If you put many people with plague in one area do expect that others will catch it. If the rule to segregate is silly enough do expect a correlation of plague with certain characteristics of people put together and the locations. Or their sizes. Maybe it is "cities" or "presence of slums" not skin colour, combined with overrepresentation of people with certain skin colour in them. It takes some write genius analysis and experimentation to untangle such complex effects from causes. Call again when you have an AI that can deal with this. Essentially a researcher AI.
gowld 8y ago
"Blacks". Are homicide rates 4x-7x higher among "blacks" in Ben Carson's gated residential community? In Boca Raton and Beverly Hills? Or might their be other factors you are ignoring?
- MichaelGG 8y ago
  In only responding to the claim that the stats are wrong because of police oversampling blacks. With homicide, no one can make that case. If lack of money causes homicides, we should expect to see all poor rural areas have high homicide rates, too. Maybe that's true. Another factor seems to be sex. Males commit so much more violence, so perhaps that's the only thing to focus on.