5 ms·
One data point really isn't enough to draw a conclusion and I actually doubt it's true. GS is extremely scrupulous when it comes to avoiding liability and they
by tootie 7y ago
One data point really isn't enough to draw a conclusion and I actually doubt it's true. GS is extremely scrupulous when it comes to avoiding liability and they probably put a ton of diligence into their risk model. It's possible there's some emergent gender bias but we'd need a better show of proof.
- xenihn 7y agoWhat we really need is the ability to force organizations to expose their models.
- toomuchtodo 7y agoTo regulators, sure. Entirely reasonable. To your average Joe? No. This is already done to prevent redlining when originating mortgages. Disclaimer: Work in financial services in risk management, interface with regulators. Opinions are my own.
- xenihn 7y agoWhat's your argument for opposing full exposure, other than the potential impact on your personal livelihood?
- toomuchtodo 7y agoThe model is not your property. Do you plan on releasing all of your company’s code on Github gratis? Unlikely. As long as the model doesn’t violate the law, you have no right to it, unless you’re a regulator. Regulators and oversight bodies get full (audited and logged) access. And while it’s none of your business, I don’t need my job, or a job at all. I work for projects I enjoy.
- xenihn 7y agoIf you're really financially independent, that's even more incentive for you to support exposure and transparency. People are being harmed by these models. You would lose nothing, and others would benefit. >As long as the model doesn’t violate the law, you have no right to it, unless you’re a regulator. If you have no realistic alternative to being judged by a model, you absolutely should have a right to it. Maybe not the complete implementation, but a verified summary that could be understood by the average person.
- deleted 7y ago[deleted]
- eropple 7y agoIn the modern economy it’s impossible to not be constantly and pervasively impacted by these data models. I think it’s reasonable to assert that it is in society’s best interest to make sure that the effects and impacts of these models are communicated as perfectly as possible to those who are forced to live under their purview. What way is this done besides viewing the model? (And given the state of regulatory capture, I’m not sure you can rely on regulators as proxies, no?) It’s totally possible that those models could then be “gamed”, sure—but it’s more important that people are able to live than for lenders and the like to make yet another buck, isn’t it?
- toomuchtodo 7y agoLenders can still realize a reasonable profit while ensuring people are not improperly discriminated against by machine intelligence. Effective regulation is possible. I would even be willing to serve a tour of duty standing up such a regulatory body, perhaps as a division of the CFPB. Throwing up your hands that government is ineffective entirely is unreasonable, just as asking for all models to be public is.
- eropple 7y agoEffective regulation is possible, sure. Is effective regulation likely? And is it made more or less likely by transparency? Heck, as I think about this further--wouldn't transparent models help realize those reasonable profits? If everybody's cards are on the table, doesn't that encourage the market to move towards perfect competition?
- toomuchtodo 7y agoIf you’re open to a conversation about what legislation would look like, I am receptive to collaboration. The best people for public policy are the ones who don’t want the job, but do it because it’s necessary.
- resonantjacket5 7y agoProblem is once a model is exposed it can be easily gamed
- xenihn 7y agoYou don't need to expose the model to figure out how to game it. It's going to be gamed anyways.
- maxerickson 7y agoYou can't really fake some of the inputs. Income, number of loans, stuff like that. You can manipulate them of course, by taking out a cheap loan that you don't necessarily need and things like that. Plenty of people use a credit card for a few expenses (say fuel) just to establish a history.
- Terretta 7y ago> One data point really isn't enough to draw a conclusion My wife's credit history is longer and historically her score higher because I used no revolving credit until recently, and not using credit cards counts against you. Also married filing jointly... For the Apple Card, I was given her limit several times over. I did notice that the expected limit is shown before the hard pull on credit. This means they've got a pretty good idea before getting the latest credit report. > GS ... probably put a ton of diligence into their risk model. To your point about the risk model, I generally think the credit score the bureaus give us is wrong, while I think the decision GS made on the card limits is probably plausible ... if there's some chance or probability we might split up. For instance, we work in different states, so data patterns might look like we are already separated? I also don't know if she put her income or household income. My income has been several times hers since long before we got together. Rather than gender bias, I would imagine that given the probability of divorces at a certain age, executive level, and income bracket, that would put a thumb on the scale for ... what if you were not married filing jointly? Who makes more? What cash payment can they carry without going broke? Weighted that way, we should have different limits, and it's not a gender thing. This is the kind of correlation humans might not come up with, but ML probably would. If this is purely actuarial, the decision might be correct, while not feeling moral.
- judge2020 7y ago> To your point about the risk model, I generally think the credit score the bureaus give us is wrong, while I think the decision GS made on the card limits is probably plausible I find this likely too, the first time I applied I got denied and the credit score in the email was considerably lower (-100 points) that what is shown on Credit Karma.
- sgerenser 7y agoThis is a common misconception. There is no such thing as a credit score. Theres FICO2008. FICO2015. FICO for car loans. FICO for revolving credit. Vantagescore. And so on and so forth. And then each of these can be based on data in any of the three different Angencies, which may be significantly different. So have if two different credit scores that are off by 100 points isn’t even remotely unusual. Many times even the scale itself is different (max of 850 vs 950 on another model).