5 ms·

We've experienced such things in the insurance industry with actuaries and their statistics/models playing the role of oracle. Pricing discrimination takes plac

We've experienced such things in the insurance industry with actuaries and their statistics/models playing the role of oracle. Pricing discrimination takes place based on the customer's job, demographics, income, or neighborhood – and often it's the poorer, younger, and less privileged people who pay more due to their 'risk' merely based on the identity groups they're in. Eventually, people get upset enough so that laws are passed enforcing whatever the society thinks is fair, and then it's the job of the industry (or whoever is controlling the 'AI' in your example) to comply, such as in the EU post-2012 with banning gender discrimination in insurance whether or not it has any statistical merit. In your case, the solution seems, to me, to be as simple as making the system ignore whatever variables you feel shouldn't be taken into account, whether that's gender or something else.

deleted 6y ago
[deleted]
darepublic 6y ago
Don't insurance companies still take considerations like gender into consideration? Ie boys getting higher rates than girls? Which funnily enough matches up with the OP except in reverse
- bencollier49 6y ago
  That's illegal in the UK now.
- KMnO4 6y ago
  It’s worse than that, insurance companies can be both sexist and racist with no consequence. A friend of mine moved and was quoted a much higher rate for their homeowners insurance, to which the agent replied “rates are higher in predominantly-black neighbourhoods”. I don’t know how that’s legal.
  - sokoloff 6y ago
    IMO, rates should be based on projected actuarial losses plus overhead and a (statistical) profit for the insurance company. From my own driving past (male), I’d expect that I was a worse risk in the 16-25 age bracket than most women I knew in that age range. Why shouldn’t I pay more?
    - AlexTWithBeard 6y ago
      Should black male pay more than a white one? (and now we're stepping onto a really slippery slope)
      - sokoloff 6y ago
        Should they pay more because they’re black? Of course not! Because they’re in a group which the actuarial data say costs more in payouts? Why not? Being black should be irrelevant; it should not itself cost a premium nor protect from paying a premium.
        AlexTWithBeard 6y ago
        So if, say, black males 16-26 are statistically more likely to get into car crash than any other group, it's perfectly legal to make them pay the highest insurance premium, right?
        sokoloff 6y ago
        Assuming all other factors are equal (that the crashes are more frequent and equally costly per-crash to the insurance company, etc), then it’s perfectly appropriate to charge any cohort the most if they’re the most expensive cohort to insure. I can’t comment on the legality, as it might be illegal in some places.
        AlexTWithBeard 6y ago
        Well, following the same logic, should it be legal for a mall to ban entry to all black people, because they have the highest chances of being shoplifters?
  - AlexTWithBeard 6y ago
    The agent's reply was clearly illegal. And probably incorrect: I highly doubt these days any pricing model uses race as an input.
new_guy 6y ago
> In your case, the solution seems, to me, to be as simple as making the system ignore whatever variables you feel shouldn't be taken into account, whether that's gender or something else. But then what's the point of using 'AI' at all if people are just gonna ignore what it comes up with? People are seeing the world the way they want to see it, not the way it is. AI sees the world the way it is, not the way people would like it to be.
- petercooper 6y ago
  But then what's the point of using 'AI' at all if people are just gonna ignore what it comes up with? I admit it's a little naive but here's a metaphor that works for me. Imagine you have access to an "AI" that's the best route finder in the world. It finds the best possible route between any two places you wish to go. However, you have a fear of going through a certain neighborhood (maybe you grew up there and have bad memories) or maybe a family member died in a crash on the freeway once and now you only stick to regular streets. The AI is so good that you can communicate these psychological and messy human preferences to the AI and it re-routes as appropriate. Is this a better or worse outcome and does providing these provisos make the AI pointless?

deleted 6y ago

[deleted]

darepublic 6y ago

Don't insurance companies still take considerations like gender into consideration? Ie boys getting higher rates than girls? Which funnily enough matches up with the OP except in reverse

bencollier49 6y ago
That's illegal in the UK now.
KMnO4 6y ago
It’s worse than that, insurance companies can be both sexist and racist with no consequence. A friend of mine moved and was quoted a much higher rate for their homeowners insurance, to which the agent replied “rates are higher in predominantly-black neighbourhoods”. I don’t know how that’s legal.
- sokoloff 6y ago
  IMO, rates should be based on projected actuarial losses plus overhead and a (statistical) profit for the insurance company. From my own driving past (male), I’d expect that I was a worse risk in the 16-25 age bracket than most women I knew in that age range. Why shouldn’t I pay more?
  - AlexTWithBeard 6y ago
    Should black male pay more than a white one? (and now we're stepping onto a really slippery slope)
    - sokoloff 6y ago
      Should they pay more because they’re black? Of course not! Because they’re in a group which the actuarial data say costs more in payouts? Why not? Being black should be irrelevant; it should not itself cost a premium nor protect from paying a premium.
      - AlexTWithBeard 6y ago
        So if, say, black males 16-26 are statistically more likely to get into car crash than any other group, it's perfectly legal to make them pay the highest insurance premium, right?
        sokoloff 6y ago
        Assuming all other factors are equal (that the crashes are more frequent and equally costly per-crash to the insurance company, etc), then it’s perfectly appropriate to charge any cohort the most if they’re the most expensive cohort to insure. I can’t comment on the legality, as it might be illegal in some places.
        AlexTWithBeard 6y ago
        Well, following the same logic, should it be legal for a mall to ban entry to all black people, because they have the highest chances of being shoplifters?
- AlexTWithBeard 6y ago
  The agent's reply was clearly illegal. And probably incorrect: I highly doubt these days any pricing model uses race as an input.

new_guy 6y ago

> In your case, the solution seems, to me, to be as simple as making the system ignore whatever variables you feel shouldn't be taken into account, whether that's gender or something else. But then what's the point of using 'AI' at all if people are just gonna ignore what it comes up with? People are seeing the world the way they want to see it, not the way it is. AI sees the world the way it is, not the way people would like it to be.

petercooper 6y ago
But then what's the point of using 'AI' at all if people are just gonna ignore what it comes up with? I admit it's a little naive but here's a metaphor that works for me. Imagine you have access to an "AI" that's the best route finder in the world. It finds the best possible route between any two places you wish to go. However, you have a fear of going through a certain neighborhood (maybe you grew up there and have bad memories) or maybe a family member died in a crash on the freeway once and now you only stick to regular streets. The AI is so good that you can communicate these psychological and messy human preferences to the AI and it re-routes as appropriate. Is this a better or worse outcome and does providing these provisos make the AI pointless?