7 ms·

Exactly, profiling.

mc32 7y ago
So profiling in and of itself isn’t automatically bad. It’s bad if it uses illegal methods , characteristics to assign weight to a profile. You can’t say if [some race] then that. But you can use other things (are they loitering, barefoot, poor hygiene, has sunglasses, truckers cap, pulls over hoodie on entering, have a decoy, etc)
- gbrown 7y ago
  So "legal" is the threshold for "bad"? I disagree. If the proportion of white shoplifters and black shoplifters being caught is different, AI will just scale up that disparity. If reality is biased, we should work to fix that: not make it more efficient.
  - mc32 7y ago
    I think if they want to target shoplifters, so long as their accuracy is within reasonable range of effectiveness, it’s okay. 90% of the Thai, 92% of AngloAmericans, 89% of Nigerians, 88% of South Americans, 87% of locals indeed processed were shoplifting, regardless of absolute numbers, this is okay.
    - gbrown 7y ago
      I think your attitude is really, really dangerous, because it ignores the long term effects these kinds of things have on society. People don't even really understand how humans apply rules in biased ways - when you attache the words "AI", "Machine Learning", or "Statistics", even well educated people tend to just accept the magic truth telling black box at face value (for a small version of this effect, Google "precision bias"). To specifically address your point: "so long as their accuracy is within reasonable range of effectiveness" This is a far too narrow look at what accuracy means - that's why we usually look at sensitivity, specificity, false negative rate, and false positive rates, and why we break those rates up over important subgroups. To the example numbers you give, let's posit (without evidence) that your numbers are correct - they don't tell us what proportion of people visiting the store belong to each group (is the model just learning to identify ethnicity?), and they don't tell us what proportion of each group is really shoplifting. In many areas (drug crimes come to mind), there's good evidence that different ethnic groups commit crimes at similar rates, but arrest rates are wildly, grotesquely disproportionate. Imagine that AI is applied to such a system - it learns, correctly, that black people are more likely to be arrested for particular crimes. It then targets black people, and the accuracy numbers look good on paper. In fact, because of the trust we have in technology, such a system might help normalize or justify this crazy preexisting disparity. If you were building a statistical model to describe predictors of shoplifting, this kind of bias wouldn't make it past a cursory level of peer review. When you're slapping "AI" on a product and implementing it at a huge scale, nobody bats an eye.
      - mc32 7y ago
        I get the positive feedback loop issue. I’m saying if I’m looking at the data (which is blind a -it doesn’t classify people by ethnicity) but manually when looking at and labeling the data we find the above proportions as the effective proportions, then I think we’re okay.
        gbrown 7y ago
        > I’m saying if I’m looking at the data My point is about the data generating mechanism - making inferences conditional on a bad selection procedure is a bad idea. It would be bad statistics, and it's bad ML. In this case, my argument is that it's also bad for society.
- Someone1234 7y ago
  You're conflating morals with legality. The two are completely different concepts. For example I can legally profile short people, but it isn't moral to do so. And I would rightfully be judged if I did so.
  - mc32 7y ago
    I don’t see why it would be immoral if empirically short people accounted for 75% of your shrinkage while being less than 50% of the shopping pop in your store.
    - Someone1234 7y ago
      Because you're treatment people differently based on something outside of their control. Just because you can justify why you'd want to act that way, in no way makes it any less wrong.
      - mc32 7y ago
        The problem is this a preposterous example, but let’s say we’re targeting kleptomaniacs. Is it immoral to target them, despite their propensity? I don’t see how.
        Someone1234 7y ago
        We're talking about profiling people on physical characteristics outside of their control. Kleptomania is a behaviour, not a physical characteristic. Asking if it is right to judge people based on their mental health is an entirely different topic than asking if we can judge people based on a physical characteristic they were born with and have no control over. Apples and oranges.
        mc32 7y ago
        Right but to go back to the beginning, we shouldn’t target short people per se, but the behavior of shoplifters, if the shoplifters in fact end up being predominantly short people while not looking specifically at short people I think it’s not immoral.
- doublekill 7y ago
  Nearly everything is correlated with race or protected social status. For instance, it is farcical to be able to target advertisements on Facebook Likes, but not on race. These systems will discriminate on racial proxies, and the question is if you are ok with that, not if those features happen to be legally allowed.
  - mc32 7y ago
    Ok but that’s like saying robberies are correlated with mobility. Sure, but that doesn’t mean we’re discriminating because they are mobile.
    - doublekill 7y ago
      I do not follow you, so I think you do not follow me. Robbers or mobility are not protected classes.
      - mc32 7y ago
        Seniors are a protected class (age discrimination) just because they are old doesn’t mean when we find out that they’re seniors are going to discount them (so long as we’re not targeting “age”).

Exactly, profiling.

mc32 7y ago

So profiling in and of itself isn’t automatically bad. It’s bad if it uses illegal methods , characteristics to assign weight to a profile. You can’t say if [some race] then that. But you can use other things (are they loitering, barefoot, poor hygiene, has sunglasses, truckers cap, pulls over hoodie on entering, have a decoy, etc)

gbrown 7y ago
So "legal" is the threshold for "bad"? I disagree. If the proportion of white shoplifters and black shoplifters being caught is different, AI will just scale up that disparity. If reality is biased, we should work to fix that: not make it more efficient.
- mc32 7y ago
  I think if they want to target shoplifters, so long as their accuracy is within reasonable range of effectiveness, it’s okay. 90% of the Thai, 92% of AngloAmericans, 89% of Nigerians, 88% of South Americans, 87% of locals indeed processed were shoplifting, regardless of absolute numbers, this is okay.
  - gbrown 7y ago
    I think your attitude is really, really dangerous, because it ignores the long term effects these kinds of things have on society. People don't even really understand how humans apply rules in biased ways - when you attache the words "AI", "Machine Learning", or "Statistics", even well educated people tend to just accept the magic truth telling black box at face value (for a small version of this effect, Google "precision bias"). To specifically address your point: "so long as their accuracy is within reasonable range of effectiveness" This is a far too narrow look at what accuracy means - that's why we usually look at sensitivity, specificity, false negative rate, and false positive rates, and why we break those rates up over important subgroups. To the example numbers you give, let's posit (without evidence) that your numbers are correct - they don't tell us what proportion of people visiting the store belong to each group (is the model just learning to identify ethnicity?), and they don't tell us what proportion of each group is really shoplifting. In many areas (drug crimes come to mind), there's good evidence that different ethnic groups commit crimes at similar rates, but arrest rates are wildly, grotesquely disproportionate. Imagine that AI is applied to such a system - it learns, correctly, that black people are more likely to be arrested for particular crimes. It then targets black people, and the accuracy numbers look good on paper. In fact, because of the trust we have in technology, such a system might help normalize or justify this crazy preexisting disparity. If you were building a statistical model to describe predictors of shoplifting, this kind of bias wouldn't make it past a cursory level of peer review. When you're slapping "AI" on a product and implementing it at a huge scale, nobody bats an eye.
    - mc32 7y ago
      I get the positive feedback loop issue. I’m saying if I’m looking at the data (which is blind a -it doesn’t classify people by ethnicity) but manually when looking at and labeling the data we find the above proportions as the effective proportions, then I think we’re okay.
      - gbrown 7y ago
        > I’m saying if I’m looking at the data My point is about the data generating mechanism - making inferences conditional on a bad selection procedure is a bad idea. It would be bad statistics, and it's bad ML. In this case, my argument is that it's also bad for society.
Someone1234 7y ago
You're conflating morals with legality. The two are completely different concepts. For example I can legally profile short people, but it isn't moral to do so. And I would rightfully be judged if I did so.
- mc32 7y ago
  I don’t see why it would be immoral if empirically short people accounted for 75% of your shrinkage while being less than 50% of the shopping pop in your store.
  - Someone1234 7y ago
    Because you're treatment people differently based on something outside of their control. Just because you can justify why you'd want to act that way, in no way makes it any less wrong.
    - mc32 7y ago
      The problem is this a preposterous example, but let’s say we’re targeting kleptomaniacs. Is it immoral to target them, despite their propensity? I don’t see how.
      - Someone1234 7y ago
        We're talking about profiling people on physical characteristics outside of their control. Kleptomania is a behaviour, not a physical characteristic. Asking if it is right to judge people based on their mental health is an entirely different topic than asking if we can judge people based on a physical characteristic they were born with and have no control over. Apples and oranges.
        mc32 7y ago
        Right but to go back to the beginning, we shouldn’t target short people per se, but the behavior of shoplifters, if the shoplifters in fact end up being predominantly short people while not looking specifically at short people I think it’s not immoral.
doublekill 7y ago
Nearly everything is correlated with race or protected social status. For instance, it is farcical to be able to target advertisements on Facebook Likes, but not on race. These systems will discriminate on racial proxies, and the question is if you are ok with that, not if those features happen to be legally allowed.
- mc32 7y ago
  Ok but that’s like saying robberies are correlated with mobility. Sure, but that doesn’t mean we’re discriminating because they are mobile.
  - doublekill 7y ago
    I do not follow you, so I think you do not follow me. Robbers or mobility are not protected classes.
    - mc32 7y ago
      Seniors are a protected class (age discrimination) just because they are old doesn’t mean when we find out that they’re seniors are going to discount them (so long as we’re not targeting “age”).