4 ms·
"Machine learning systems trained to minimize prediction error may often exhibit discriminatory behavior based on sensitive characteristics such as race and gen
by joe_the_user 9y ago
"Machine learning systems trained to minimize prediction error may often exhibit discriminatory behavior based on sensitive characteristics such as race and gender. One reason could be due to historical bias in the data."
The thing about this discussion is that it aims to balance social welfare goals and profit maximizing goals but without any criteria for attaining basic fairness on a general level.
To wit, suppose a machine learning algorithm is looking at twenty or fifty piece of data about a given individual, all of which are entirely irrelevant to the individual's chance of repaying a loan. But by random chance, one of those pieces of data, say handedness, happens to be correlated with a group's repayment history. So, some individuals with a good handedness are given loans more frequently and some individuals, with a bad handedness, are given loans less frequently. This situation doesn't matter to the company since the data is irrelevant and they only give out so many loans anyway and shitting on, say, left-handed people, gives them no grief. Moreover, if this trend is noted by all the companies soon it will be made "true".
Which to say, companies making life-defining decisions like mortgages or parole-granting, just should be prohibited from the using a lasagna of random data to make their decisions and instead should be required to use specific rules with specific reasons behind them.
And cry me a river about missed chances for optimization. This about the structure of society and optimization doesn't benefit society here imo.
- lambdaphagy 9y agoThis isn't about overfitting. Race is almost exactly the opposite problem: the signal in the data is so strong that you can't help but pick it up, even after you censor everything you can think of. If you try to predict any kind of consumer behavior from a racially heterogeneous dataset you will end up finding something that correlates with race, because practically everything correlates with race.
- mistermann 9y agoIs this actually true, that everything correlates with race? I know many things do, but does it even exceed 50%?
- deleted 9y ago[deleted]
- mistermann 9y agoHey mods....if you happen to monitor those of us with warnings, I wonder if you can run a query to determine my top 3 downvoters over time and see if a pattern emerges. This is starting to get rather annoying.
- paulsutter 9y ago> practically everything correlates with race. Hold on, excuse me?
- fossuser 9y agoIt's easy for ML to pick up on proxies that are related to race even if you exclude what the race is explicitly - zip code and names are two common ones, but this can appear in a lot of different ways. It's particularly insidious too since a model that accidentally is making credit decisions based on race helps to perpetuate the inequity in the future data.
- deleted 9y ago[deleted]
- mlevental 9y agolol. what this person is basically saying is that racism affects almost every part of a person's life. I know it seems unbelievable but "practically everything correlates with race" is the most nerd-intelligible way I've seen of explaining/expressing that racism is pervasive.
- lambdaphagy 9y agoThat's not what I'm saying.
- mlevental 9y agofine you're not actually saying it (though I think you should be) but the direct implication of what you're saying is that.
- echotango 9y agoHe is saying that race is correlated with almost everything, not racism. You seem to be claiming all statistical differences between racial groups are a result of racism. But black people for instance aren't more likey to listen to rap music because of racism.
- red75prime 9y agoIf you throw out the "random data", some groups could be overloaned, which leads to a decrease of average credit score of that group (caused by increase of defaults in that group), and that can decrease amount of loans the group will get in the future, even when compared to the unfair profit maximizing policy. That's what the article is about.