5 ms·
> If others are denying loans for no good reason, that is a business opportunity. It doesn't require benevolence to offer people loans that they are qualified
by paganel 9y ago
> If others are denying loans for no good reason, that is a business opportunity. It doesn't require benevolence to offer people loans that they are qualified for.
Afaik the black families who were segregated out of their neighborhoods back in the 1940s, 1950s and going into the 1960s were never provided with another comparable "business opportunity" that could have "righted" their situation, they had to settle with living in what turned out to become "ghettos" (for lack of a better word).
> Would you object to a self-driving car that routes itself around more dangerous neighborhoods? What about a model that predicts that unknown men on the street at night are more dangerous than unknown women?
Following the same line of thought, would you be ok with an AI system giving a person named Deion or Jayla or Latisha a higher interest rate on their mortgage (and so, potentially, driving them out of certain markets) compared to the interest rate offered to persons named Chad or Emma or Sophia?
- PeterisP 9y agoIf we know nothing else about them but their names (which is a quite big assumption), a person named Deion or Jayla or Latisha is more likely to come from a disadvantaged socioeconomic background than a person named Chad or Emma or Sophia, have a worse job, have a somewhat higher chance of getting arrested in future, and due to these and other factors they causally, objectively have a somewhat higher risk of defaulting on that loan. It's not just bias, prejudice or superstition; it's not a mistake in data or process; it's an indication of some true, factual observation about reality. The correlation isn't huge, but it is there, and it's not a coincidence. If it's your money that you're choosing to lend (or not lend), would you ignore that information or take it into account when making a decision if you should offer that loan and how much the rate should be adjusted to cover the risk of default? What if it's not your money to do with it as you please, but money entrusted to you to invest for maximum results? Sure, if you knew what this particular person is actually like, then you'd base the decision on better data than just their name, and it could well be that this Jayla would get a much better deal than this Sophia; but even if you had a perfectly fair system that correctly estimates the individual risk of default without prejudice causing worthy individuals being dragged down by association with some group, then there would still be a correlation with the name simply because in the current reality the proportion of "risky Jaylas" is larger than the proportion of "risky Sophias".
- haberman 9y agoThe shameful history of "redlining" (housing segregation) was driven by policy, not market forces. Black people weren't being judged non-credit-worthy, they were being explicitly prohibited from borrowing and buying by FHA policy and housing covenants. So this history, while awful and something that we should not repeat, does not speak to my point that making rational decisions about credit-worthiness is good business sense. > Following the same line of thought, would you be ok with an AI system... You didn't answer my question. I asked it because I want to know if the people most vocally arguing that de-biasing is a moral imperative will admit even one case where there might be a compelling reason to see the world as it is, even if that association is considered "problematic". If people will admit this, then we can argue over where the line should be. But many don't appear to admit that a line even exists. I will admit that the line exists, and that an incident like this is a clear example where removing hurtful associations is proper. This was a case where a ML model reinforced a loaded and racist stereotype, and the harm of removing that from the model is almost zero: https://www.theverge.com/2015/7/1/8880363/google-apologizes-photos-app-tags-two-black-people-gorillas https://www.theverge.com/2015/7/1/8880363/google-apologizes-...
- paganel 9y ago> You didn't answer my question. I asked it because I want to know if the people most vocally arguing that de-biasing is a moral imperative will admit even one case where there might be a compelling reason to see the world as it is, even if that association is considered "problematic". I didn't find those questions to be that smart, to be honest, they're more on the scare-mongering side, and I find that a smart question should have half of the answer included in it and a scare-mongering question doesn't look like it contains anything smart (at least to me). But if you really want to know my answer is "yes" to all of your questions. To get into more details: I grew up as a kid into a middle-class-ish family and back then I had no issues going to the "ghetto"/dangerous area of the town I grew up in (I grew up in Eastern Europe, so that the "dangerous" area was populated by the local gipsy community instead of the AfroAmerican/Latino communities now associated with the "dangerous" areas of US cities). I turned up fine. > If people will admit this, then we can argue over where the line should be. But many don't appear to admit that a line even exists. I know of that line, I was just trying to say that further reinforcing it using ML techniques will only aggravate things at a societal level (so that "dangerous" areas will become even more "dangerous"). At least when the discrimination happens out of our (us, humans) own volition there are ways to fix it, but once we "outsource" our racist-tendencies to ML-like tools the voice inside of us telling us that this is all wrong will become even weaker. After all, the algorithms/machines are more "right" then us, humans, that's what we like to think.