5 ms·
It is possible to inspect black box models. See for instance http://www.blackboxworkshop.org/pdf/Turner2015_MES.pdf http://www.blackboxworkshop.org/pdf/Turner20
by BickNowstrom 10y ago
It is possible to inspect black box models. See for instance http://www.blackboxworkshop.org/pdf/Turner2015_MES.pdf http://www.blackboxworkshop.org/pdf/Turner2015_MES.pdf and https://homes.cs.washington.edu/~marcotcr/blog/lime/ https://homes.cs.washington.edu/~marcotcr/blog/lime/ . It is also common to use highly accurate white box models for cases where it is important to not overfit, leak, or discriminate, like MARS and GA^2M (somewhat out of scope for an actuary using R).
If a computer program spots an irrelevant unhelpful correlation its generalization error will noticeable go up. If it is an irrelevant "helpful" correlation it means there is a problem with the data (such as leakage), not with the algorithm. If there is a problem with the data, all bets are of, both for black and white box models.
A blackbox model will probably not find that being an Armenian alone will lead to more crashes. Being non-linear in nature it will find interactions (young male Armenians are more likely to crash than young males in general). If the importance of such a feature is not significant enough to distinguish it from noise, then regularization may automatically remove it.
Even if we observe that 99 out 100 Armenians crash their cars, and you decline someone a loan, because he/she is Armenian, you may just have discriminated against the 1 Armenian who is a safe driver. Young male drivers who drive safely have a worse time getting loans, because their group (the set of young male drivers) spoiled it for them. So their only hope of getting a loan is you adding more features (like nationality), to be able to distinguish them as safe drivers, not removing them and lumping them into the status quo.
- tptacek 10y agoIsn't that the kind of work we're commenting on with this story?
- yummyfajitas 10y agoNo, it's not at all the same. Those articles are about eliminating overfitting and improving accuracy. This article is about how much fairness constraints will reduce your accuracy/objective function.
- tptacek 10y agoDid you read the paper, or just the web page with the simulator? The loan simulator is a motivating example, not the entirety of the work. The work is a comparison between the basic approach of demographic parity and the authors contributions, which seeks to minimize the expense you seem to be talking about.
- yummyfajitas 10y agoI read the paper a month ago, and I'm quite familiar with this field for my own reasons [1]. They attempt to minimize the expense I'm talking about. They don't reduce overfitting, improve accuracy, or even make any changes to the underlying predictive algorithm. The simulator is a great illustration of exactly what they did; the entirety of the work is generalizing that to arbitrary predictors (subject to a few conditions) and bounding the accuracy penalty. Feel free to cite the theorem improving accuracy if you disagree. [1] One idea I'm kicking around is the following. Banks/others are legally required to issue bad loans for fairness. I suspect there is a lot of money to be made hacking this, I just haven't figured out how yet.
- dharma1 10y agoI saw a talk by a fintech company CTO who lend to people in European countries without reliable credit scoring. They used lots of data points, effectively a black box system. But sometimes they loan to people they wouldn't normally loan to, just because that loan adds the kind of data they would otherwise not get, and they take a risk on default in exchange for more data. I'm assuming smaller loan amounts. So I guess that is one way for lending institutions to say they are issuing "bad loans for fairness" while still benefiting from it in terms of getting more training data and making the model more robust.
- deleted 10y ago[deleted]