11 ms·
I hope that regulatory agencies start cracking down on these "alternative sources" for credit-worthiness data. Researchers have shown how it's very easy to attr
by hn_throwaway_99 6y ago
I hope that regulatory agencies start cracking down on these "alternative sources" for credit-worthiness data. Researchers have shown how it's very easy to attribute things like political party, race, sexuality, etc., things that are often illegal to take into consideration when determining credit, to other things like where you live, TV shows you like, apps you have on your phone, etc.
And strangely (or not) this blog post from the IMF barely touches on the huge privacy concerns with this stuff.
- thegrimmest 6y agoIf these sources of data are actually more accurate predictors of creditworthiness, then shouln't we be applauding their use? If it so happens that tall people are less likely to repay loans, why is it wrong to charge tall people higher interest? Otherwise aren't we just socializing the cost imposed by artificially fuzzy criteria?
- layoutIfNeeded 6y agoLet me illuminate the problem: s/tall/black/ (inb4 downvotes)
- thegrimmest 6y agoI actually wasn't thinking that at all. I chose tall because there is actually a major disparity in income between tall and short people. This disparty seems to be the ultimate counter-argument to any misguided attempt at "levelling the playing field".
- Retric 6y agoCredit worthiness is directly used on job applications etc, so it feeds back into people’s ability to pay back loans. As such accuracy isn’t an independent variable it’s part of a larger system. We don’t want to artificially make people with for example long fingers unable to pay back loans. As such you want to avoid feedback loops even at the cost of some accuracy.
- thereisnospork 6y agoI agree with the premise, but the problem is that it gets ethically, uh 'interesting', if you replace 'tall' with say, 'black', in your argument.
- insert_coin 6y agoI think it gets to the point. If blacks are for some reason (and I am not saying they are I do not know) less likely to pay back loans, then that is the problem that needs addressing. It might be a little painful to admit the huge racial divide but it is real and that is a problem we need to solve, taking the easy way out lying to ourselves only harms society in the long run and destroys the lives of people who cannot pay back the loans and end up in a huge debt cycle very difficult to get out of.
- hammock 6y agoSo should we use the algorithms to charge different races different interest rates? Or should we not use them. And address the problems you mention? Because in the previous comment it sounded like they wanted to use them with applause.
- insert_coin 6y agoUsing the algorithm is addressing the problems by not hiding them.
- hammock 6y agoShould we legalize drunk driving, and address the problem by trying to solve alcoholism?
- thegrimmest 6y agoI think this would be more aking to banning drinking and drunkenness altogether. I don't think that went so well the last time we tried it.
- deleted 6y ago[deleted]
- pmichaud 6y agoWhen I have this conversation with myself, here's what I come up with: Shouldn't we be allowed to everything into account when determining credit worthiness? Even if it's socially uncomfortable? We get locally more accurate pricing / risk assessment that way, but we also create feedback loops that we as a society have decided we want to not contribute to. The cost for this is the pricing inaccuracy which we collectively eat. So, I don't think you're wrong, but maybe consider if what we're socializing the cost of is worth the price?
- slg 6y agoThat is a good summary. An analogy I might add that could further clarify things would be preexisting condition protections for health insurance. Preventing insurance companies from turning down people who they know will require expensive medical care is going to lead to worse pricing for the rest of us. However much of society has decided that is a cost worth enduring in the name of fairness.
- WanderPanda 6y agoProblem with this is, this cuts open the feedback loop of being responsible for ones own health. Credit assignment problem all the way
- NeuNeurosis 6y agoAre you American? Because high cost of health insurance does not deter risky behaviors towards health or a glaring lack of responsibility.
- kortilla 6y agoHealth insurance is weird because it’s become the way you pay for something expected. In every other insurance you will absolutely get denied or charged a huge sum if you’re knowingly going to be a loss. It’s like applying for home fire insurance and saying you burned down your previous 3 homes and will likely burn down this one.
- 6y ago
- austhrow743 6y agoYes, we are just socialising the cost of different races, sexes, ages, etc being different. Many people like that.
- thegrimmest 6y agoPeople like many things, not all of them are ethical. Why do you think it's okay to impose this on people who don't consent?
- deleted 6y ago[deleted]
- throwaway189262 6y agoMore accurate isn't always good because creditworthiness is all about stereotyping people. Consider this example: lesbians are statistically more likely to divorce. The ML models decides that lesbians are a higher mortgage risk and raises their interest rates. It does this indirectly by raising interest rates on married couples with different last names and owners of Subaru vehicles. In another, the model sees that people who follow NBA players on twitter are more likely to default. It then assigns then higher interest rates. Well, it turns out that poor urban black people are far more likely to be interested in the NBA than the average person. Probably because it's one of the only sports you can play in a parking lot. The model decides to raise interest rates and cut lending to most black people. "Perfect" credit worthiness modelling is only going to be beneficial to rich white and asian people. Essentially favoring the stereotypically richest ethnic groups. This is extremely problematic for any sort of equality initiatives. Treating everyone the same regardless of means is definitely socialist. It's the same as removing the pre-existing conditions exclusions for health insurance. It favors the unfortunate at the expense of the fortunate. I'm not going to go into whether that is good or bad for society but I ask you to consider the extreme, historic level wealth inequality in the US where 1% of the population has over half the money.
- thegrimmest 6y agoA lot to unpack here, let me try. Are machine learning algorythms capable of stereotyping people? How accurate does a stereotype have to be before it's a useful predictor? If these models lead to good data, then what's wrong with them? If people who follow the NBA do in fact default on their loans more, shouldn't they pay higher interest? I agree that the healthy should not be compelled to subsidize the costs of healthcare for the unhealthy. Can you tell me why it's ethical to force my compliance? It seems you primarily see things along racial divides, grouping people into black, white, and asian. There are, more nuanced ways to group people, which is exactly what these agorythms are doing. Also, what's wrong with wealth inequality? I'd be much more concerned about absolute quality of life, which has increased dramatically for everyone in the last half century.
- dredmorbius 6y ago
- aneemzic 6y agoNo. Credit worthiness should be because of an individuals past actions and not some attribute they may have been born with. Why should a tall individual who's never missed a payment have some invisible penalization applied because some other tall people are worse at re-paying loans. Don't you see the issue here? You're penalizing individuals not based on their own measurable behavioral signals, but simply due to some physical trait they share with others. Not to mention, we wouldn't even understand why there's a correlation in the first place. Maybe tall people are 30% likely to have some unknown gene, why should the other 70% of tall people who don't have the gene be treated the same?
- hntrader 6y agoWe must disambiguate between political objectives and the practice of credit risk modelling. Credit risk, f(X), is an unknown population function that needs to be estimated using observed data X. If including tallness into X improves our estimate of f(X), then we've gotten a better model. You've asserted that X should only contain an individual's past actions instead of their inherent traits such as tallness. This may satisfy certain political objectives, and that's fine if we're being upfront about the underlying motivation, but from an ML perspective your prescription doesn't make much sense unless you have some prior knowledge about the function f(X) that tells you that tallness both isn't relevant and isn't acting as an instrumental variable for some other missing feature. Unless you have such domain knowledge, you've little business asserting what are appropriate features to use in order to improve model quality.
- aneemzic 6y agoI don't agree that this has anything to do with political objectives. It's a question of ethics. My domain knowledge is irrelevant, the ML Modelling aspect is irrelevant. The discussion was specifically around whether it's ok to include inherent traits when determining the credit worthiness of an individual. If you think it is, that's fine. We might as well just taking the same approach to crime, and start locking individuals up or not extending job offers, NOT because they've done a single thing wrong, but simply because they're statistically more likely to.
- ThrustVectoring 6y agoNo. We don't actually want the most economically efficient system here. There are other extremely important characteristics - fairness, due process, legibility, and incentives come to mind here. Like, presumably you could improve your creditworthiness prediction by cross-referencing against bankruptcy law related searches on Google and in library checkouts. We absolutely should not allow this, though - we care a lot more about having a society where people are free to read and speak about what they want than optimal consumer credit allocation.
- thegrimmest 6y agoThe requirement for "fairness" and "due process" indicate that you think credit is something a person is entitled to. Is this true? If so, can I ask why you think so?
- swiley 6y agoThis is a great idea if you'd like to continue down our current path of creating a large group of disenfranchised people who think violent revolution is the only way to get out of their current situation.
- drtillberg 6y agoThis. Slippery slope to denying credit based on a consumer's decision to avoid using Facebook or to search using DuckDuckGo.
- hntrader 6y agoDDG is more popular in the tech-literate crowd, most of whom are earning large salaries and therefore we'd expect that DDG usage should be positively (not negatively) correlated with credit worthiness
- weird-eye-issue 6y agoYou are completely missing the point and just making up assumptions to argue, stop
- hntrader 6y agoI wasn't arguing with or misunderstanding the point being made. I was just disputing that DDG usage would be positively related to credit unworthiness. I admit that it was pedantic.
- NeutronStar 6y agoWhere did you see the statistics on DDG users being more tech literate and having high incomes?
- wernercd 6y agoI think the denial to credit for DDG is less about credit worthyness and more about "deplatforming" based on personal details. Or thats what it'll become.
- 8note 6y agoThat's highly unlikely. As the GGP said, using ddg is associated with being wealthy, so it will have the opposite effect over time. If you're a rich white guy, this oppression is unlikely to apply to your choices. Instead, your choices will be used as filters to identify you at the exclusion of others
- beambot 6y agoI was once peripherally involved in a conversation about ML algorithms being used to determine credit worthiness. One of the bankers explained that race as a factor is illegal, but the algo partially reduced down to a handful of heuristics -- eg that "last names ending in 'z' were less credit worthy" -- and that while these all roughly correlated to race, they were perfectly permissible since the algo was a black box & no one ever ran the correlations (conveniently). TLDR: sickening.
- visarga 6y agoHere's a case where the Google AI ethics researchers should have unleashed their hounds, not on Yann LeCun and large language models (stochastic parrots? lol). Design evaluation datasets and checklists, do research into "bias washing", call them out on biased models in deployment, etc.
- dohnuts1919 6y agoAgreed. I can't remember the details but I was listening to a podcast lately (Sam Harris maybe?) where the guest was talking about an example of an ML-driven system that was being used to determine whether or not to approve mortgage applications, and upon closer inspection it turned out to be latching onto some data that gave clues as to the applicant's race and using that in its decision. You can guess the outcome: the software re-invented redlining.