4 ms·

For a real world version of your "Bob" example, many people have negative associations with the names "Jamal" and "Mohammed". Do you think we should train AI t

by jfasi 8y ago

For a real world version of your "Bob" example, many people have negative associations with the names "Jamal" and "Mohammed". Do you think we should train AI to systematically ignore those sentiments?

benrbray 8y ago
Machine learning models are pretty good at reasoning with (or can be made to reason with) conditional probability, which I think can solve this problem. For example, people from X race might be more likely to commit crimes in the absence of any other information (marginal probability). However, person from X race is no more likely to commit crimes than a person from Y race, conditioned on something else like where they went to school, what they do for a living, etc..
- pgodzin 8y ago
  When talking about something like crime rates by race, your statement incorrectly assumes society enforces laws fairly/equally. For example, based on the numbers alone, black people are much more likely to commit drug-related crimes than white people. However, studies show that black people are stopped a lot more often and have unequal sentencing outcomes. It's important to remember that AI doesn't have context, and just because it's using "data" to make decisions doesn't mean the decisions are unbiased - the underlying data may be biased.
  - geofft 8y ago
    One of the things I'm rather worried about here is AI's ability to learn on the most unlikely (to human cognition) features. Take the image recognizer that finds "sheep" by seeing pastures where it thinks sheep might graze http://aiweirdness.com/post/172894792687/when-algorithms-surprise-us http://aiweirdness.com/post/172894792687/when-algorithms-sur... , or take any of the adversarial perturbing examples like https://blog.openai.com/robust-adversarial-inputs/ https://blog.openai.com/robust-adversarial-inputs/ . You're not allowed to discriminate on race in housing. But is an AI that determines your creditworthiness for mortgages allowed to discriminate on what you eat, where you go to church, what your favorite music is, etc.? Maybe it doesn't have that data, but it will have one level higher - what store credit cards you have and how much you use them and where you opened them. If you train an AI on a segregated city with a history of actively discriminatory citizens, where the few people of race X who moved into a not-race-X neighborhood got harassed out and sold before they paid off their mortgages, how easily will the AI conclude that people born in certain neighborhoods are more likely to pay off their mortgage if they avoid certain other neighborhoods? Is that illegal? (My guess is there's no way to prove to a court, to the court's usual standards, that the AI happened to learn the city's racial tensions.) Should it be illegal, if outright racial discrimination is illegal?
  - MichaelGG 8y ago
    Homicide rates are unlikely to have this kind of bias though. And they're heavily against blacks. It's unlikely whites being murdered go unreported, or that anyone gets off with a warning. But inner city homicide has a low solving rate. If the reporting rate is also lower, then it could be even worse than the 4-7x overrepresentation of blacks in homicide. This seems to be the case in London, too, so it's a bigger issue than just the US. But for things like drug possession, yes, blacks probably get stopped much more and not let off with warnings much less.
    - AstralStorm 8y ago
      Untangling cause from effect in this case is extremely hard. It is like with plagues. If you put many people with plague in one area do expect that others will catch it. If the rule to segregate is silly enough do expect a correlation of plague with certain characteristics of people put together and the locations. Or their sizes. Maybe it is "cities" or "presence of slums" not skin colour, combined with overrepresentation of people with certain skin colour in them. It takes some write genius analysis and experimentation to untangle such complex effects from causes. Call again when you have an AI that can deal with this. Essentially a researcher AI.
    - gowld 8y ago
      "Blacks". Are homicide rates 4x-7x higher among "blacks" in Ben Carson's gated residential community? In Boca Raton and Beverly Hills? Or might their be other factors you are ignoring?
      - MichaelGG 8y ago
        In only responding to the claim that the stats are wrong because of police oversampling blacks. With homicide, no one can make that case. If lack of money causes homicides, we should expect to see all poor rural areas have high homicide rates, too. Maybe that's true. Another factor seems to be sex. Males commit so much more violence, so perhaps that's the only thing to focus on.
- oldandtired 8y ago
  Let's get it straight, machine learning models cannot reason anything. They follow the rules and data supplied. They cannot recognise in any way that the rules by which they operate or the data they are supplied with are or are not reasonable. Much as there is much hype about AI and machine learning technologies, all of such systems will be, for the foreseeable future, very simplistic models of what we think we know of intelligence. As humans, we have blindly forgotten that we know very little about the world around us, including ourselves. We think that we have a handle on reality, but we are just plain ignorant. All the models that we have for deep learning and AI or GAI are barely scratching the surface of what "intelligence" is and means. We may get some useful tools as a result of the research being undertaken today and what we have undertaken over the last lot of decades. But we have millenia to go before we even scratch the surface of what we understand of the universe around us, let alone understand what intelligence means.
  - ben_w 8y ago
    > Let's get it straight, machine learning models cannot reason anything. They follow the rules and data supplied. That is reasoning. I’ll agree with your later point that we don’t really know what intelligence is yet, but that’s because reasoning is clearly only one type of intelligence. Current systems are terrible at language, for example (a year ago I would have said “and spatial awareness”, but this is progressing fast and I am no longer sure). > They cannot recognise in any way that the rules by which they operate or the data they are supplied with are or are not reasonable. Quite a lot of humans fit that description. For example, consider how much angry disagreement the following questions get: “is climate change real?”, “Brexit, yes or no?”, and “does god exist?” Also consider how many people (angrily!) refuse to believe these questions result in any angry arguments.

by jfasi 8y ago

benrbray 8y ago

Machine learning models are pretty good at reasoning with (or can be made to reason with) conditional probability, which I think can solve this problem. For example, people from X race might be more likely to commit crimes in the absence of any other information (marginal probability). However, person from X race is no more likely to commit crimes than a person from Y race, conditioned on something else like where they went to school, what they do for a living, etc..

pgodzin 8y ago
When talking about something like crime rates by race, your statement incorrectly assumes society enforces laws fairly/equally. For example, based on the numbers alone, black people are much more likely to commit drug-related crimes than white people. However, studies show that black people are stopped a lot more often and have unequal sentencing outcomes. It's important to remember that AI doesn't have context, and just because it's using "data" to make decisions doesn't mean the decisions are unbiased - the underlying data may be biased.
- geofft 8y ago
  One of the things I'm rather worried about here is AI's ability to learn on the most unlikely (to human cognition) features. Take the image recognizer that finds "sheep" by seeing pastures where it thinks sheep might graze http://aiweirdness.com/post/172894792687/when-algorithms-surprise-us http://aiweirdness.com/post/172894792687/when-algorithms-sur... , or take any of the adversarial perturbing examples like https://blog.openai.com/robust-adversarial-inputs/ https://blog.openai.com/robust-adversarial-inputs/ . You're not allowed to discriminate on race in housing. But is an AI that determines your creditworthiness for mortgages allowed to discriminate on what you eat, where you go to church, what your favorite music is, etc.? Maybe it doesn't have that data, but it will have one level higher - what store credit cards you have and how much you use them and where you opened them. If you train an AI on a segregated city with a history of actively discriminatory citizens, where the few people of race X who moved into a not-race-X neighborhood got harassed out and sold before they paid off their mortgages, how easily will the AI conclude that people born in certain neighborhoods are more likely to pay off their mortgage if they avoid certain other neighborhoods? Is that illegal? (My guess is there's no way to prove to a court, to the court's usual standards, that the AI happened to learn the city's racial tensions.) Should it be illegal, if outright racial discrimination is illegal?
- MichaelGG 8y ago
  Homicide rates are unlikely to have this kind of bias though. And they're heavily against blacks. It's unlikely whites being murdered go unreported, or that anyone gets off with a warning. But inner city homicide has a low solving rate. If the reporting rate is also lower, then it could be even worse than the 4-7x overrepresentation of blacks in homicide. This seems to be the case in London, too, so it's a bigger issue than just the US. But for things like drug possession, yes, blacks probably get stopped much more and not let off with warnings much less.
  - AstralStorm 8y ago
    Untangling cause from effect in this case is extremely hard. It is like with plagues. If you put many people with plague in one area do expect that others will catch it. If the rule to segregate is silly enough do expect a correlation of plague with certain characteristics of people put together and the locations. Or their sizes. Maybe it is "cities" or "presence of slums" not skin colour, combined with overrepresentation of people with certain skin colour in them. It takes some write genius analysis and experimentation to untangle such complex effects from causes. Call again when you have an AI that can deal with this. Essentially a researcher AI.
  - gowld 8y ago
    "Blacks". Are homicide rates 4x-7x higher among "blacks" in Ben Carson's gated residential community? In Boca Raton and Beverly Hills? Or might their be other factors you are ignoring?
    - MichaelGG 8y ago
      In only responding to the claim that the stats are wrong because of police oversampling blacks. With homicide, no one can make that case. If lack of money causes homicides, we should expect to see all poor rural areas have high homicide rates, too. Maybe that's true. Another factor seems to be sex. Males commit so much more violence, so perhaps that's the only thing to focus on.
oldandtired 8y ago
Let's get it straight, machine learning models cannot reason anything. They follow the rules and data supplied. They cannot recognise in any way that the rules by which they operate or the data they are supplied with are or are not reasonable. Much as there is much hype about AI and machine learning technologies, all of such systems will be, for the foreseeable future, very simplistic models of what we think we know of intelligence. As humans, we have blindly forgotten that we know very little about the world around us, including ourselves. We think that we have a handle on reality, but we are just plain ignorant. All the models that we have for deep learning and AI or GAI are barely scratching the surface of what "intelligence" is and means. We may get some useful tools as a result of the research being undertaken today and what we have undertaken over the last lot of decades. But we have millenia to go before we even scratch the surface of what we understand of the universe around us, let alone understand what intelligence means.
- ben_w 8y ago
  > Let's get it straight, machine learning models cannot reason anything. They follow the rules and data supplied. That is reasoning. I’ll agree with your later point that we don’t really know what intelligence is yet, but that’s because reasoning is clearly only one type of intelligence. Current systems are terrible at language, for example (a year ago I would have said “and spatial awareness”, but this is progressing fast and I am no longer sure). > They cannot recognise in any way that the rules by which they operate or the data they are supplied with are or are not reasonable. Quite a lot of humans fit that description. For example, consider how much angry disagreement the following questions get: “is climate change real?”, “Brexit, yes or no?”, and “does god exist?” Also consider how many people (angrily!) refuse to believe these questions result in any angry arguments.