4 ms·

The problem here is, we don't even know what "fair" means. What do we mean by fair? Say we want to base fair on group outcomes, in which ways should we divide p

The problem here is, we don't even know what "fair" means. What do we mean by fair? Say we want to base fair on group outcomes, in which ways should we divide people into groups in order to perform some sort of balancing? Is hair color or texture a relevant way of subdividing people for instance? Are we being "fair" to red haired people? What if red haired people for some reason seem to be less likely to pay back loans? Will we need to adjust our credit rating model to bump the credit scores of red haired people slightly? Will we need to agonize over other features fed to our model to ensure that they can't be used as a proxy for red hair? Maybe???? But it seems very dishonest. I think, if anything, the models are probably too true for our tastes as social creatures who evolved to avoid social conflict (being 100% honest with your unruly neighbors can lead to a loss of harmony that is much more damaging than just putting up with some disturbances). Our idea of "unbiasing" the models is more likely actually making them biased in a way that pleases our monkey brains.

simion314 8y ago
Maybe don't split groups on genetics/biology/birth. Also make sure the data you use is not a proxy for the above criteria
- Iolaum 8y ago
  Easier said than done.
  - simion314 8y ago
    I know, this is why all this algorithms should not be hidden/proprietary. At least we have a chance to spot why/how the computer decided my fate.
    - hnaccount141 8y ago
      The problem is that in many cases (especially with deep learning), "the algorithm" is essentially a black box. There often isn't a straightforward way to determine exactly what the algorithm is actually learning.
      - simion314 8y ago
        I know that too, but if I(or the community) have access to it I can prove the algorithms is flawed. Say i change a bit the input and notice a huge output, like I change my non enlgish name to an english name and by magic my score triples then I prove the algorithm is discriminating me(same if I change my birth place or address). I don't think that opening the algorithms is a sufficient condition but I think is necessary.
- InitialLastName 8y ago
  But what if, due to historical happenstance, the data is already a proxy for groups split on genetics, biology, or birth? Using that data in a way that might create a feedback loop that propagates that happenstance is entirely likely. Part of the problem is that, when dealing with humans as the data set, the data is really messy and there's lots of noise correlated to historical circumstance.
paulmd 8y ago
> Is hair color or texture a relevant way of subdividing people for instance? Are we being "fair" to red haired people? What if red haired people for some reason seem to be less likely to pay back loans? Machine learning is going to be very good at identifying proxies for features we don't like, even if we don't directly provide those features. Weighting red hair color could be a proxy for the Irish, targeting hair texture could be a proxy for african americans, targeting zip codes has always been a proxy for race, etc. If we want machine learning to ignore these kind of things it's probably going to have to be a separately weighted training goal that racial groups and so on need to have equal outcomes in the final results. That's really what AOC is going for here. The black box will happily learn racist ways to weight things all on its own, we can't just throw up our hands and say oh well, if the algorithm is denying black people for home loans then we just have to accept it. Because, in the short term, being racist is an effective greedy strategy. It's not your business's problem that, say, black people have higher rates of default (or whatever, example). But it also leads to bad social outcomes in the aggregate, and we've agreed that shouldn't be a factor in home loans. And we have to find a way to make sure that The Algorithm (tm) doesn't just find a way to machine-learn redlining or some other proxy for race or other protected characteristics.
- deleted 8y ago
  [deleted]
- perl4ever 8y ago
  "And we have to find a way to make sure that The Algorithm (tm) doesn't just find a way to machine-learn redlining or some other proxy for race or other protected characteristics." This is not wrong, but it boils down to "don't make mistakes when optimizing functions". It's offensive because it's coming from someone who has nothing to contribute except "do your job better". It's like saying we have a problem with planes crashing, so engineers have a moral duty to build better planes. As though they weren't trying to build planes that stay in the air already. You can make serious sounding statements, or you can impose fines, or you can randomly pick people to publicly shame, but none of those things are positive contributions to hard problems.
  - 8y ago
jstarfish 8y ago
The problem is the self-reinforcing, external factors that these algorithms don't take into account. If you feed a ML algorithm crime stats, it will conclude crime is directly correlated to being black, or being a former criminal. But this correlation only exists because of how we've chosen to conduct the policing of black communities and treatment of ex-cons upon release that produced the the training data in the first place. Most people here are male, were once (or are) under 25 and likely drive. How'd you like paying more for insurance than your parents because some system decided men are bad drivers, and doubly so when they're young? Triply so when they happen to be driving a red car? Quadruply so if there are prior citations/accidents? Shit, with a spotless driving record, my own insurance went up when someone t-boned me. Other driver was at-fault and I subrogated their insurance; didn't claim with mine. But I'm being lumped into a higher-risk pool with my own insurer for factors completely beyond my control-- because to some algorithm, being in an accident means I'm more likely to be in accidents, and all drivers who are involved in collisions are higher risks than drivers who aren't. Note how the fact that I wasn't at fault isn't factored in. This makes sense to you? The social training data in much of Europe is skewed towards hating Gypsies and anyone perceived to be one. But it's the same as the ex-con problem-- deny them jobs, beat them, chase them away, and of course an ex-con or young Romani will commit future crimes. You're shaping behavior to validate your training data instead of making unbiased ("fair") predictions. It's the equivalent of betting on a horse, injuring its competition, then patting yourself on the back for being so goddamn good at this game. You can't look at the world in terms of sheer numbers. The numbers themselves are dishonest.
- deleted 8y ago
  [deleted]
- leetcrew 8y ago
  > How'd you like paying more for insurance than your parents because some system decided men are bad drivers, and doubly so when they're young? Triply so when they happen to be driving a red car? isn't this how it already works? young males cause a disproportionate amount of insurance claims so they get assessed higher rates. they'll even bump your premiums for driving the performance version of a given car. being a young male, I certainly don't like it, but I'm not sure I would call it unfair.
SideburnsOfDoom 8y ago
> The problem here is, we don't even know what "fair" means. Maybe, but we know blatantly unfair algorithms when we see them. And we see them. e.g. Remember the AI recruiting tool that automatically rejected women because that's what it learned from the input data? https://www.theverge.com/2018/10/10/17958784/ai-recruiting-tool-bias-amazon-report https://www.theverge.com/2018/10/10/17958784/ai-recruiting-t...

simion314 8y ago

Maybe don't split groups on genetics/biology/birth. Also make sure the data you use is not a proxy for the above criteria

Iolaum 8y ago
Easier said than done.
- simion314 8y ago
  I know, this is why all this algorithms should not be hidden/proprietary. At least we have a chance to spot why/how the computer decided my fate.
  - hnaccount141 8y ago
    The problem is that in many cases (especially with deep learning), "the algorithm" is essentially a black box. There often isn't a straightforward way to determine exactly what the algorithm is actually learning.
    - simion314 8y ago
      I know that too, but if I(or the community) have access to it I can prove the algorithms is flawed. Say i change a bit the input and notice a huge output, like I change my non enlgish name to an english name and by magic my score triples then I prove the algorithm is discriminating me(same if I change my birth place or address). I don't think that opening the algorithms is a sufficient condition but I think is necessary.
InitialLastName 8y ago
But what if, due to historical happenstance, the data is already a proxy for groups split on genetics, biology, or birth? Using that data in a way that might create a feedback loop that propagates that happenstance is entirely likely. Part of the problem is that, when dealing with humans as the data set, the data is really messy and there's lots of noise correlated to historical circumstance.

paulmd 8y ago

> Is hair color or texture a relevant way of subdividing people for instance? Are we being "fair" to red haired people? What if red haired people for some reason seem to be less likely to pay back loans? Machine learning is going to be very good at identifying proxies for features we don't like, even if we don't directly provide those features. Weighting red hair color could be a proxy for the Irish, targeting hair texture could be a proxy for african americans, targeting zip codes has always been a proxy for race, etc. If we want machine learning to ignore these kind of things it's probably going to have to be a separately weighted training goal that racial groups and so on need to have equal outcomes in the final results. That's really what AOC is going for here. The black box will happily learn racist ways to weight things all on its own, we can't just throw up our hands and say oh well, if the algorithm is denying black people for home loans then we just have to accept it. Because, in the short term, being racist is an effective greedy strategy. It's not your business's problem that, say, black people have higher rates of default (or whatever, example). But it also leads to bad social outcomes in the aggregate, and we've agreed that shouldn't be a factor in home loans. And we have to find a way to make sure that The Algorithm (tm) doesn't just find a way to machine-learn redlining or some other proxy for race or other protected characteristics.

deleted 8y ago
[deleted]
perl4ever 8y ago
"And we have to find a way to make sure that The Algorithm (tm) doesn't just find a way to machine-learn redlining or some other proxy for race or other protected characteristics." This is not wrong, but it boils down to "don't make mistakes when optimizing functions". It's offensive because it's coming from someone who has nothing to contribute except "do your job better". It's like saying we have a problem with planes crashing, so engineers have a moral duty to build better planes. As though they weren't trying to build planes that stay in the air already. You can make serious sounding statements, or you can impose fines, or you can randomly pick people to publicly shame, but none of those things are positive contributions to hard problems.
- 8y ago

jstarfish 8y ago

The problem is the self-reinforcing, external factors that these algorithms don't take into account. If you feed a ML algorithm crime stats, it will conclude crime is directly correlated to being black, or being a former criminal. But this correlation only exists because of how we've chosen to conduct the policing of black communities and treatment of ex-cons upon release that produced the the training data in the first place. Most people here are male, were once (or are) under 25 and likely drive. How'd you like paying more for insurance than your parents because some system decided men are bad drivers, and doubly so when they're young? Triply so when they happen to be driving a red car? Quadruply so if there are prior citations/accidents? Shit, with a spotless driving record, my own insurance went up when someone t-boned me. Other driver was at-fault and I subrogated their insurance; didn't claim with mine. But I'm being lumped into a higher-risk pool with my own insurer for factors completely beyond my control-- because to some algorithm, being in an accident means I'm more likely to be in accidents, and all drivers who are involved in collisions are higher risks than drivers who aren't. Note how the fact that I wasn't at fault isn't factored in. This makes sense to you? The social training data in much of Europe is skewed towards hating Gypsies and anyone perceived to be one. But it's the same as the ex-con problem-- deny them jobs, beat them, chase them away, and of course an ex-con or young Romani will commit future crimes. You're shaping behavior to validate your training data instead of making unbiased ("fair") predictions. It's the equivalent of betting on a horse, injuring its competition, then patting yourself on the back for being so goddamn good at this game. You can't look at the world in terms of sheer numbers. The numbers themselves are dishonest.

deleted 8y ago
[deleted]
leetcrew 8y ago
> How'd you like paying more for insurance than your parents because some system decided men are bad drivers, and doubly so when they're young? Triply so when they happen to be driving a red car? isn't this how it already works? young males cause a disproportionate amount of insurance claims so they get assessed higher rates. they'll even bump your premiums for driving the performance version of a given car. being a young male, I certainly don't like it, but I'm not sure I would call it unfair.

SideburnsOfDoom 8y ago

> The problem here is, we don't even know what "fair" means. Maybe, but we know blatantly unfair algorithms when we see them. And we see them. e.g. Remember the AI recruiting tool that automatically rejected women because that's what it learned from the input data? https://www.theverge.com/2018/10/10/17958784/ai-recruiting-tool-bias-amazon-report https://www.theverge.com/2018/10/10/17958784/ai-recruiting-t...