4 ms·

An obvious issue is AI thrown at a bank loan department reproducing redlining. Current AI tech allows for laundering this kind of shit that you couldn’t get aw

by rtpg 3y ago

An obvious issue is AI thrown at a bank loan department reproducing redlining. Current AI tech allows for laundering this kind of shit that you couldn’t get away with nearly as easily otherwise (obviously still completely possible in existing regulatory alignments, despite what conservative media likes to say. But there’s at least a paper trail!) This is a real issue possible with existing tech that could potentially be applied tomorrow. It’s not SF, it’s a legitimate concern. But it’s hard and nebulous and the status quo kinda sucks as well. So it’s tough to get into

bootsmann 3y ago
This kind of redlining is ironically what the EU is trying to prevent with the much-criticised AI Act. It has direct provisions about explainability for exactly this reason.
ndriscoll 3y ago
My experience working for a technology provider for banks is that banks aren't going to be using uncensored models. Auditing, record keeping, explainability, carefully selected/thoroughly reviewed wording etc. are par for the course, and in fact that's where there's money to be made. Individuals don't care about these sorts of features, so the FOSS model ecosystem is unlikely to put much if any effort into them. B2B use-cases are going to want that, so it's something you can build without worrying as much about it being completely commoditized.
concordDance 3y ago
The concern about redlining has always slightly puzzled me. Why do we only care that some people are being unjust denied loans when those being denied loans make up a recognizable ethnicity?
- pixl97 3y ago
  Because the law says if you fuck around with "race, religion, age, sex, disability" and a few other things you will get sued in federal court and lose your ass so bad that it will financially hurt for a while. Outside of protected classes unjust loan denial isn't really illegal. Now that can be your own series of complaints that need addressed, but they aren't ones covered by current laws.

by rtpg 3y ago

bootsmann 3y ago

This kind of redlining is ironically what the EU is trying to prevent with the much-criticised AI Act. It has direct provisions about explainability for exactly this reason.

ndriscoll 3y ago

My experience working for a technology provider for banks is that banks aren't going to be using uncensored models. Auditing, record keeping, explainability, carefully selected/thoroughly reviewed wording etc. are par for the course, and in fact that's where there's money to be made. Individuals don't care about these sorts of features, so the FOSS model ecosystem is unlikely to put much if any effort into them. B2B use-cases are going to want that, so it's something you can build without worrying as much about it being completely commoditized.

concordDance 3y ago

The concern about redlining has always slightly puzzled me. Why do we only care that some people are being unjust denied loans when those being denied loans make up a recognizable ethnicity?

pixl97 3y ago
Because the law says if you fuck around with "race, religion, age, sex, disability" and a few other things you will get sued in federal court and lose your ass so bad that it will financially hurt for a while. Outside of protected classes unjust loan denial isn't really illegal. Now that can be your own series of complaints that need addressed, but they aren't ones covered by current laws.