3 ms·
There are legitimate reasons to reduce externalizations of societies innate biases. A mortgage AI that calculates premiums for the public shouldn't bias agains
by Jerrrry 4y ago
There are legitimate reasons to reduce externalizations of societies innate biases.
A mortgage AI that calculates premiums for the public shouldn't bias against people with historically black names, for example.
This problem is harder to tackle because it is difficult to expose and resign the "latent space" that results in these biases; it's difficult to massage the ML algo's to identify and remove the pathways that result in this bias.
It's simply much easier to allow the robot to be bias/racist/reflective of "reality" (its training data), and add a filter / band-aid on top; which is what they've attempted.
when this is appropriate is the more cultured question; I don't think we should attempt to band-aid these models, but for more socially-critical things, it is definitely appropriate.
It's naive on either extreme: do we reject reality, and substitute or own? Or do we call our substitute reality, and hope the zeitgeist follows?
- mh- 4y ago> A mortgage AI that calculates premiums for the public shouldn't bias against people with historically black names, for example. That's a great example, thanks. Also, I hope the teams working on that come up with a different solution...
- ceeplusplus 4y agoThat's great, but by doing so you are also inadvertently favoring, in your example, the people with black names. For example, Chinese people save on average, 50 times more than Americans according to the Fed [1]. That would mean they would generally be overrepresented in loan approvals because they have a better balance sheet. Does that necessarily mean that Americans are discriminated against in the approval process? No. My question to you is: is an algorithm that takes no racial inputs (name, race, address, etc) yet still produces disproportionate results biased or racist? I say no. [1] https://files.stlouisfed.org/files/htdocs/publications/es/08/ES0819.pdf https://files.stlouisfed.org/files/htdocs/publications/es/08...
- Jerrrry 4y agoI would agree that it is not. The government, and many people, have moved the definition and goal posts; so that anything that has the end result of a non-proportional uniformity can be labeled and treated as bias. Ultimately it is a nuanced game; is discriminating against certain clothing or hair-styles racist? Of course. Yet, neither of those are explicitly tied to one's skin color or ethnicity, but are an indirect associative trait because of culture. In America, we have intentionally muddled the waters of demarcation between culture and race, and are starting to see the cost of that.
- deleted 4y ago[deleted]
- whywhywhywhy 4y agoWouldn't the whole point of a "Mortgage AI" be to discriminate so the lenders hands could be clean. Not that I agree with that but I don't see why you would build one otherwise, if you wanted discrimination free mortgages wouldn't the whole process by anonymized and minimal personal information rather than the current system of having to hand over every detail of your life.