4 ms·

There seems to be a bit of explaining away all bias as the result of human creators here. If an algorithm trained on real world data displayed some traditional

There seems to be a bit of explaining away all bias as the result of human creators here. If an algorithm trained on real world data displayed some traditional bias, would we assume such a cause and dismiss it? One could just as easily make this argument to dismiss all correlation based evidence. Yes the code / methodology matters, but we'd best get ready for deep learning algorithms to confront us with some uncomfortable truths.

empath75 10y ago
Let's assume you have red and green people who are otherwise identical, and that there was systemic discrimination against green people in the past that led them to have lower incomes and lower education levels and live in areas with lower property values and higher crime rates. A naive computer model trained on objective data might find correlations between green people and high default rates on loans, for example, which might be accurate, but which aren't inherent to green people but were forced on them by external circumstances. A reasonable bank might then conclude that they should discriminate against green people without any personal bias against green people at all. The question you have to ask yourself is: is that fair? And what should the law say about it?
- iopq 10y ago
  They aren't inherent in green people, but you shouldn't give them loans anyway. Loans should be based on reality, not an ideal world that we don't live in. And certainly fairness has no basis in reality. It's a human concept.
- steveeq1 10y ago
  (quote by some guy named "oehead2000" on some other hacker news posting. It fit the context perfectly, so I am copying and pasting from his reply) "Unfortunately whatever algorithm computers come up with to assess creditworthiness is going to be 'racist' by the broad disparate impact definition of racism that the government uses. Ironically, the algorithms will have to be made actually racist to correct for this."
- andrewclunn 10y ago
  Most significant variable. If green is a proxy for poor, which is in turn a proxy for risky mortgage loan, then the poor variable is the better point of discrimination.
- h4nkoslo 10y ago
  Well, the proper response probably isn't to shoot yourself in the head and prevent yourself from knowing about the difference in default rates. It is entirely possible to figure out what is going on, to what extent, and then make an informed decision as to whether to continue. Avoiding discovering things, which is what all these asinine articles end up suggesting, is almost never a good idea for competent adults, and legally preventing people from discovering things is antithetical to a free society.
- atrendyguy 10y ago
  Forcing non-optimal decisions based on an arbitrary notion of "fairness" will only weaken our institutions and breed resentment: "group A gets loan approval with a lower credit score than members of my group, what gives!?" It also increases the fragility of the group you're trying to help by acclimating them to lower standards. The success of Asian and Jewish people suggests that things can correct themselves over time. The law should stay out of it. http://www.pewresearch.org/daily-number/asian-americans-lead-all-others-in-household-income/ http://www.pewresearch.org/daily-number/asian-americans-lead... http://www.zakkeith.com/articles%2Cblogs%2Cforums/anti-Chinese-persecution-in-the-USA-history-timeline.htm http://www.zakkeith.com/articles%2Cblogs%2Cforums/anti-Chine...

empath75 10y ago

Let's assume you have red and green people who are otherwise identical, and that there was systemic discrimination against green people in the past that led them to have lower incomes and lower education levels and live in areas with lower property values and higher crime rates. A naive computer model trained on objective data might find correlations between green people and high default rates on loans, for example, which might be accurate, but which aren't inherent to green people but were forced on them by external circumstances. A reasonable bank might then conclude that they should discriminate against green people without any personal bias against green people at all. The question you have to ask yourself is: is that fair? And what should the law say about it?

iopq 10y ago
They aren't inherent in green people, but you shouldn't give them loans anyway. Loans should be based on reality, not an ideal world that we don't live in. And certainly fairness has no basis in reality. It's a human concept.
steveeq1 10y ago
(quote by some guy named "oehead2000" on some other hacker news posting. It fit the context perfectly, so I am copying and pasting from his reply) "Unfortunately whatever algorithm computers come up with to assess creditworthiness is going to be 'racist' by the broad disparate impact definition of racism that the government uses. Ironically, the algorithms will have to be made actually racist to correct for this."
andrewclunn 10y ago
Most significant variable. If green is a proxy for poor, which is in turn a proxy for risky mortgage loan, then the poor variable is the better point of discrimination.
h4nkoslo 10y ago
Well, the proper response probably isn't to shoot yourself in the head and prevent yourself from knowing about the difference in default rates. It is entirely possible to figure out what is going on, to what extent, and then make an informed decision as to whether to continue. Avoiding discovering things, which is what all these asinine articles end up suggesting, is almost never a good idea for competent adults, and legally preventing people from discovering things is antithetical to a free society.
atrendyguy 10y ago
Forcing non-optimal decisions based on an arbitrary notion of "fairness" will only weaken our institutions and breed resentment: "group A gets loan approval with a lower credit score than members of my group, what gives!?" It also increases the fragility of the group you're trying to help by acclimating them to lower standards. The success of Asian and Jewish people suggests that things can correct themselves over time. The law should stay out of it. http://www.pewresearch.org/daily-number/asian-americans-lead-all-others-in-household-income/ http://www.pewresearch.org/daily-number/asian-americans-lead... http://www.zakkeith.com/articles%2Cblogs%2Cforums/anti-Chinese-persecution-in-the-USA-history-timeline.htm http://www.zakkeith.com/articles%2Cblogs%2Cforums/anti-Chine...