4 ms·

The problem is the self-reinforcing, external factors that these algorithms don't take into account. If you feed a ML algorithm crime stats, it will conclude c

The problem is the self-reinforcing, external factors that these algorithms don't take into account. If you feed a ML algorithm crime stats, it will conclude crime is directly correlated to being black, or being a former criminal. But this correlation only exists because of how we've chosen to conduct the policing of black communities and treatment of ex-cons upon release that produced the the training data in the first place. Most people here are male, were once (or are) under 25 and likely drive. How'd you like paying more for insurance than your parents because some system decided men are bad drivers, and doubly so when they're young? Triply so when they happen to be driving a red car? Quadruply so if there are prior citations/accidents? Shit, with a spotless driving record, my own insurance went up when someone t-boned me. Other driver was at-fault and I subrogated their insurance; didn't claim with mine. But I'm being lumped into a higher-risk pool with my own insurer for factors completely beyond my control-- because to some algorithm, being in an accident means I'm more likely to be in accidents, and all drivers who are involved in collisions are higher risks than drivers who aren't. Note how the fact that I wasn't at fault isn't factored in. This makes sense to you? The social training data in much of Europe is skewed towards hating Gypsies and anyone perceived to be one. But it's the same as the ex-con problem-- deny them jobs, beat them, chase them away, and of course an ex-con or young Romani will commit future crimes. You're shaping behavior to validate your training data instead of making unbiased ("fair") predictions. It's the equivalent of betting on a horse, injuring its competition, then patting yourself on the back for being so goddamn good at this game. You can't look at the world in terms of sheer numbers. The numbers themselves are dishonest.

deleted 8y ago
[deleted]
leetcrew 8y ago
> How'd you like paying more for insurance than your parents because some system decided men are bad drivers, and doubly so when they're young? Triply so when they happen to be driving a red car? isn't this how it already works? young males cause a disproportionate amount of insurance claims so they get assessed higher rates. they'll even bump your premiums for driving the performance version of a given car. being a young male, I certainly don't like it, but I'm not sure I would call it unfair.
- jstarfish 8y ago
  It does. That's the problem. Take an 18-year old with a red Mustang and a 58-year old with a midlife crisis and a 3-series. Both drive like assholes. Both will inevitably cause an accident. The latter is more likely to be wealthy enough to be able to pay cash for repairs and keep the entire incident off the radar. No insurance claim, no citations, no evidence. The 18-year old has no choice but to make an insurance claim, go to court and take points on their license. Can we truly conclude that 18-year olds with red Mustangs are worse drivers than their elder counterparts? (And yes, I realize that in the case of insurance, it's in their interest to minimize claims so in that context this is a valid conclusion. But this same skewed data gets shared outside of the industry as well.)
  - leetcrew 8y ago
    > Take an 18-year old with a red Mustang and a 58-year old with a midlife crisis and a 3-series. Both drive like assholes. Both will inevitably cause an accident. this is not a fair example; by assumption we have much more information about the 58yo than the 18yo. remove the bit about the midlife crisis and there's no reason to think the 58yo is anywhere near the same risk to insure. as a group, young people are less experienced drivers, have less facility for judgement, and the males probably drive more aggressively. unless you disagree that the goal of insurance is to spread risk among a bucket of people who have relatively similar risk factors, I'm not really sure where you're going with this. and by the way, I'm a young male who drives a fast car.
    - jstarfish 8y ago
      > unless you disagree that the goal of insurance is to spread risk among a bucket of people who have relatively similar risk factors, I'm not really sure where you're going with this. I don't disagree. In this context, if older men make fewer insurance claims, they're better customers. On paper, he's less of a risk (to the insurer). In reality, they're both risks to other drivers. But only the 18-year old will be held accountable. But assuming the under-25-and-male crowd are the worst drivers on the road based on number of insurance claims is not necessarily true, yet it's the basis for this oft-repeated assertion. That's my point. What isn't the insurance industry's data telling us when we use their data to make these conclusions? To get more accurate numbers, you need to be looking at the demographics of all body shop customers and the nature of their repairs. What demographics get the most front-end work done that isn't covered by an insurance check? This new age of computing seems to be to make machines understand how the world works by putting blind faith in statistics, without ever stopping to question where the stats came from or how accurate they really are. The numbers aren't handed to us on golden tablets carved by God himself-- we get them from imperfect systems designed by imperfect humans. And then we teach dynamic algorithms how reality works based on this data and sell them as solutions, which we trust absolutely. > and by the way, I'm a young male who drives a fast car. No offense intended. I once was, too.
burfog 8y ago
Your higher rate makes a bit of sense. The worst problem is the small sample size. If everybody crashed a minimum of 100 times, then the risk estimates would be much better. The fact that you were hit is influenced by your risk factors. For example, maybe that is a bad intersection, and you always drive through it. Maybe it really is just the other driver being bad... but he and you happen to leave for work every day at a reliable time, making it likely that you will encounter him again. Maybe your car is hard to see. Determining "not at fault" is a fuzzy thing; perhaps a better driver would have somehow avoided the crash. The number of miles you drive is an influence on the number of crashes you get, and perhaps you drive more than is typical.
yorwba 8y ago
> The problem is the self-reinforcing, external factors that these algorithms don't take into account. And changing the algorithms isn't the right way to fix those external factors. If you try, their self-reinforcing nature is going to cause exactly the opposite of what you want. To stay with the crime vs. race example, assume that the ratio of black to white criminals is greater than you'd expect from the ratio of blacks to whites in the overall population, and that is due to self-reinforcing external factors. Your algorithm for deciding whether to parole a prisoner picks up on that statistical regularity and rates blacks riskier than whites. Because you want to achieve representational fairness, you make the algorithm race-blind to equalize the rates. What happens? The black prison population shrinks. But your algorithm wasn't wrong when it predicted higher recidivism risk, so the black crime rate rises. Your algorithm now requires an even stronger correction to remain race-blind. Black criminals realize that they have an easier time getting paroled, so the black crime rate rises some more. Debiasing the algorithm has made the situation worse. In general, when you find out that your machine-learning model has some undesirable bias, the correct response is not to blind the algorithm. Instead, you should fix the data-generation process (the real world) until your model no longer picks up on the bias. Edit: For a mathematically precise description of how attempts to make an algorithm more fair cause worse results when you consider the feedback loop that it is embedded in, see "Delayed Impact of Fair Machine Learning" https://arxiv.org/abs/1803.04383 https://arxiv.org/abs/1803.04383
gdy 8y ago
"Note how the fact that I wasn't at fault isn't factored in. This makes sense to you?" Actually, yes. In Russia we have a principle called DDD which can be translated as Give Way to a Moron ("Дай дорогу дураку"). People who avoid dangerous drivers, even when it hurts their ego, are less likely to get involved in a collision.

deleted 8y ago

[deleted]

leetcrew 8y ago

> How'd you like paying more for insurance than your parents because some system decided men are bad drivers, and doubly so when they're young? Triply so when they happen to be driving a red car? isn't this how it already works? young males cause a disproportionate amount of insurance claims so they get assessed higher rates. they'll even bump your premiums for driving the performance version of a given car. being a young male, I certainly don't like it, but I'm not sure I would call it unfair.

jstarfish 8y ago
It does. That's the problem. Take an 18-year old with a red Mustang and a 58-year old with a midlife crisis and a 3-series. Both drive like assholes. Both will inevitably cause an accident. The latter is more likely to be wealthy enough to be able to pay cash for repairs and keep the entire incident off the radar. No insurance claim, no citations, no evidence. The 18-year old has no choice but to make an insurance claim, go to court and take points on their license. Can we truly conclude that 18-year olds with red Mustangs are worse drivers than their elder counterparts? (And yes, I realize that in the case of insurance, it's in their interest to minimize claims so in that context this is a valid conclusion. But this same skewed data gets shared outside of the industry as well.)
- leetcrew 8y ago
  > Take an 18-year old with a red Mustang and a 58-year old with a midlife crisis and a 3-series. Both drive like assholes. Both will inevitably cause an accident. this is not a fair example; by assumption we have much more information about the 58yo than the 18yo. remove the bit about the midlife crisis and there's no reason to think the 58yo is anywhere near the same risk to insure. as a group, young people are less experienced drivers, have less facility for judgement, and the males probably drive more aggressively. unless you disagree that the goal of insurance is to spread risk among a bucket of people who have relatively similar risk factors, I'm not really sure where you're going with this. and by the way, I'm a young male who drives a fast car.
  - jstarfish 8y ago
    > unless you disagree that the goal of insurance is to spread risk among a bucket of people who have relatively similar risk factors, I'm not really sure where you're going with this. I don't disagree. In this context, if older men make fewer insurance claims, they're better customers. On paper, he's less of a risk (to the insurer). In reality, they're both risks to other drivers. But only the 18-year old will be held accountable. But assuming the under-25-and-male crowd are the worst drivers on the road based on number of insurance claims is not necessarily true, yet it's the basis for this oft-repeated assertion. That's my point. What isn't the insurance industry's data telling us when we use their data to make these conclusions? To get more accurate numbers, you need to be looking at the demographics of all body shop customers and the nature of their repairs. What demographics get the most front-end work done that isn't covered by an insurance check? This new age of computing seems to be to make machines understand how the world works by putting blind faith in statistics, without ever stopping to question where the stats came from or how accurate they really are. The numbers aren't handed to us on golden tablets carved by God himself-- we get them from imperfect systems designed by imperfect humans. And then we teach dynamic algorithms how reality works based on this data and sell them as solutions, which we trust absolutely. > and by the way, I'm a young male who drives a fast car. No offense intended. I once was, too.

burfog 8y ago

Your higher rate makes a bit of sense. The worst problem is the small sample size. If everybody crashed a minimum of 100 times, then the risk estimates would be much better. The fact that you were hit is influenced by your risk factors. For example, maybe that is a bad intersection, and you always drive through it. Maybe it really is just the other driver being bad... but he and you happen to leave for work every day at a reliable time, making it likely that you will encounter him again. Maybe your car is hard to see. Determining "not at fault" is a fuzzy thing; perhaps a better driver would have somehow avoided the crash. The number of miles you drive is an influence on the number of crashes you get, and perhaps you drive more than is typical.

yorwba 8y ago

> The problem is the self-reinforcing, external factors that these algorithms don't take into account. And changing the algorithms isn't the right way to fix those external factors. If you try, their self-reinforcing nature is going to cause exactly the opposite of what you want. To stay with the crime vs. race example, assume that the ratio of black to white criminals is greater than you'd expect from the ratio of blacks to whites in the overall population, and that is due to self-reinforcing external factors. Your algorithm for deciding whether to parole a prisoner picks up on that statistical regularity and rates blacks riskier than whites. Because you want to achieve representational fairness, you make the algorithm race-blind to equalize the rates. What happens? The black prison population shrinks. But your algorithm wasn't wrong when it predicted higher recidivism risk, so the black crime rate rises. Your algorithm now requires an even stronger correction to remain race-blind. Black criminals realize that they have an easier time getting paroled, so the black crime rate rises some more. Debiasing the algorithm has made the situation worse. In general, when you find out that your machine-learning model has some undesirable bias, the correct response is not to blind the algorithm. Instead, you should fix the data-generation process (the real world) until your model no longer picks up on the bias. Edit: For a mathematically precise description of how attempts to make an algorithm more fair cause worse results when you consider the feedback loop that it is embedded in, see "Delayed Impact of Fair Machine Learning" https://arxiv.org/abs/1803.04383 https://arxiv.org/abs/1803.04383

gdy 8y ago

"Note how the fact that I wasn't at fault isn't factored in. This makes sense to you?" Actually, yes. In Russia we have a principle called DDD which can be translated as Give Way to a Moron ("Дай дорогу дураку"). People who avoid dangerous drivers, even when it hurts their ego, are less likely to get involved in a collision.