4 ms·

It sounds like a way to set up a positive feedback loop with terrible consequences to me. I don’t know enough about ML to be sure, but I know GIGO. If you feed

It sounds like a way to set up a positive feedback loop with terrible consequences to me. I don’t know enough about ML to be sure, but I know GIGO. If you feed bias into a model then let the model guide decision making, that seems obviously flawed. You bias the model not to give loans or credit to black people, reinforcing their lack of credit, which leads to the model confirming its bias. Maybe you can claim it’s a socioeconomic bias and not racist, and in some countries it would be true, but not in America. How long ago was it that black people had to fight past mobs to integrate schools? How long since redlining? If socioeconomic are tied to a history of oppression based on race, then ignoring half of that equation isn’t honest. Correlation is not causation, but does “AI” know that?

whatshisface 9y ago
If someone is poor but you want them to be given a chance anyways, you don't go to the banker and say, "please close your eyes to what by all means seems to be true." You say, "I know he's more likely to default, but society wants him lifted out of poverty and you're going to give him that loan."
- DmenshunlAnlsis 9y ago
  If those banks had to turn to the government to keep them afloat, and those banks played a large role in creating the problem in the first place, maybe you do? Maybe not, but what you definitely don’t do is set up black boxes to take the heat for centuries of prejudice and mistreatment.
  - whatshisface 9y ago
    > don’t [...] set up black boxes to take the heat I think I see now how we agree. This whole "algorithms" thing is a smokescreen over what is really just the AA debate: should institutions optimize purely for their own goals (in which case they would do nothing to help with old wounds, whenever ignoring them would be cheaper), or should they be expected to mix in social responsibility into their cost functions? Whether it's AI or a cunning but amoral banker, it's the same question.
    - iguy 9y ago
      Right. And note that those banks who got bailed out weren't free-floating rational agents before the crash either. Because it was deemed politically desirable to increase minority home-ownership, they were given pressure / incentives to make AA loans.

whatshisface 9y ago

If someone is poor but you want them to be given a chance anyways, you don't go to the banker and say, "please close your eyes to what by all means seems to be true." You say, "I know he's more likely to default, but society wants him lifted out of poverty and you're going to give him that loan."

DmenshunlAnlsis 9y ago
If those banks had to turn to the government to keep them afloat, and those banks played a large role in creating the problem in the first place, maybe you do? Maybe not, but what you definitely don’t do is set up black boxes to take the heat for centuries of prejudice and mistreatment.
- whatshisface 9y ago
  > don’t [...] set up black boxes to take the heat I think I see now how we agree. This whole "algorithms" thing is a smokescreen over what is really just the AA debate: should institutions optimize purely for their own goals (in which case they would do nothing to help with old wounds, whenever ignoring them would be cheaper), or should they be expected to mix in social responsibility into their cost functions? Whether it's AI or a cunning but amoral banker, it's the same question.
  - iguy 9y ago
    Right. And note that those banks who got bailed out weren't free-floating rational agents before the crash either. Because it was deemed politically desirable to increase minority home-ownership, they were given pressure / incentives to make AA loans.