5 ms·
The United States does not have anything as sweeping as this, but in the limited area of credit the Equal Credit Opportunity Act (ECOA) requires that lenders th
by tzs 10y ago
The United States does not have anything as sweeping as this, but in the limited area of credit the Equal Credit Opportunity Act (ECOA) requires that lenders that turn down your credit application give you an explanation. From the Federal Trade Commission's site:
The creditor must tell you the specific
reason for the rejection or that you are
entitled to learn the reason if you ask
within 60 days. An acceptable reason might
be: “your income was too low” or “you haven’t
been employed long enough.” An unacceptable
reason might be “you didn’t meet our minimum
standards.” That information isn’t specific
enough.
https://www.consumer.ftc.gov/articles/0347-your-equal-credit-opportunity-rights https://www.consumer.ftc.gov/articles/0347-your-equal-credit...
(The ECOA is at 15 U.S. Code § 1691, for those who want to go find the gory details)
In one of the lectures from Caltech's "Learning from Data" MOOC the professor mentioned that problem came up at a major lender that he had consulted for. He did not say how they solved the problem.
I wondered if you could generate a satisfactory rejection reason, at least as far as the law is concerned, by taking rejected applications and running them through the system again but with some of the inputs tweaked until they get an acceptance. Then report that the rejection was due to the un-tweaked parameter value.
For instance, if you picked income to tweak, you'd raise the income by, say, $5000 and try again. If that is rejected, raise it another $5000. If it then passes, you can tell the applicant they were rejected because their income was too low, and that an additional $10000 income would be enough to qualify them for a loan.
You'd have to put a bit of sophistication into this to make the explanations reasonable. For instance, if a rejected application would be approved with a very large tweak to income or with a small tweak to employment length it would probably be better to give employment length as the reason for rejection.
- PeterisP 10y agoA potential problem with this is that if you "tell the applicant they were rejected because their income was too low" then if this is disputed then they will likely be able to point to many other applicants with even lower income that were accepted, because their total combination of factors was better. A simple explanation "factor X is too low" implies that there exists a particular cutoff that is required and sufficient; but an explanation that accurately describes why you were rejected would likely be too complex to be understandable. But yes, probably the intended result could/should be simply identifying the top 1-2 factors that dragged your score down compared to the average accepted candidate (which can be rather easily done automatically even for the most black-box ML methods) and naming them appropriately.
- Houshalter 10y agoYou can look at the gradient of the inputs. It's not just that your income is too low, but also that you don't have enough credit score, past history, etc. This basically gives you a linear model that is locally accurate at predicting what the more complicated model will do. Linear models are pretty interpretable I think.
- basseq 10y ago> an explanation that accurately describes why you were rejected would likely be too complex to be understandable. This is the core of the problem. Have you ever tried to explain a statistical segmentation to someone? It requires them to let go of how they think about if-x-then-y criteria and attributes. Very intelligent people who understand statistics, weighting, clustering, etc. struggle with it--think about an "average" person. You can back into a reason: "Here's the attribute(s) on which you deviate the most from the median of the segment of which you want to be a part." But it's not a linear, single-variable "fix".
- dozzie 10y agoThere was this idea (I think I stumbled upon it on Bruce Schneier's blog) where the algorithm used to judge your credit score was actually public, and there was a way to dispute its outcome if you felt it was unfair in your case.
- basseq 10y agoInteresting from an academic perspective, but absolutely inaccessible to 99% of the population. If I don't get the outcome I want, I will dispute it (and people already do) because I think it's unfair. And maybe it is in your case. Are all algorithms designed to be "fair"? Should they be? Can they be, if everyone has an exception or a loophole? Or at some point do "disputes" go back to a human with the same biases that algorithms were intended to solve?
- hx87 10y ago> Then report that the rejection was due to the un-tweaked parameter value. Shouldn't it be the tweaked parameter value(s), since those are what made the difference between acceptance and rejection?
- klipt 10y ago> The United States does not have anything as sweeping as this The EU takes discrimination much more seriously. For example auto insurance in the EU can't charge men and women different amounts, meanwhile auto insurers in the US shamelessly advertise that they do so.
- Houshalter 10y agoBut men are much riskier drivers. That's has nothing to do with sexism or discrimination against men, it's just an empirical fact. Why should women have to subsidize the cost of men?
- klipt 10y agoWhy is it acceptable to you to have sex explicitly in the model, but not race? If black men are empirically riskier drivers than Asian men (something insurance companies are currently legally forbidden to discriminate on), would you be fine with charging black men more so Asian men don't "subsidize" them?
- Houshalter 10y agoOnly because race has a history of irrational prejudice and discrimination. And because race shouldn't really be predictive of anything, or at least that's the politically accepted belief. So it's been made illegal to take that into account. Which is fine, if it was just limited to that. But it's led to hundreds of other categories also being "protected", and now they are outlawing categorizing people at all. It's ridiculous. I'm perfectly fine with Asian drivers getting cheaper insurance, if they really are somehow safer drivers. Why should they pay more, if they aren't more risky? What about wooden buildings getting higher insurance rates? Wooden buildings are more likely to be damaged in natural disasters, burn down, suffer water damage, etc. But that's not the fault of the owner. Any specific owner could take really good care of their building, have sprinklers and alarms, maintain everything to perfect condition, etc. Yet they still pay the higher rate, so it's not a fair system. But as a group, wooden buildings are just less safe. An older person is probably going to pay much more in life insurance than a young person. Or an overweight person, or a smoker. It may not be fair on an individual level. After all not every smoker gets cancer, not every overweight person gets diabetes, etc. But as a group, some groups have more risk than others. The point of insurance is to predict risk as accurately as possible. Using whatever information is available. If we had time machines, we could go into the future and see what people would get into car accidents and take away their licenses. Or tear down buildings that are going to fall down and hurt people. But because we don't have that information, insurance acts as a way to hedge risk. There is zero benefit to society by making their predictions less accurate, and perhaps serious economic costs.