3 ms·

You could check a random sample of them with expert humans to ensure there isn't a systematic issue causing you to issue large loans that you shouldn't be issui

You could check a random sample of them with expert humans to ensure there isn't a systematic issue causing you to issue large loans that you shouldn't be issuing. I doubt regulators would be happy with this though, especially since regulations are often a jobs program for former employees of regulators.

rafram 4y ago
I don’t think regulator nepotism is the main reason that the authorities would be uncomfortable with loan decisions being made by a system that definitionally reinforces existing biases and is incapable of thought. It’s just a bad idea!
- LawTalkingGuy 4y ago
  Well, by definition in these cases it's making a correct decision but that decision isn't popular. If red-lining wasn't profitable the banks would have gone broke and we wouldn't be talking about it. The models aren't wrong, or even wrongly biased. We're systematically failing to ask the right questions because it's not politically correct to even call out the issues we're trying to combat. If you're trying to achieve "equity" then you need to define it and represent it in your equations, not ban various metrics until justice happens.
credit_guy 4y ago
You don't need to check a random sample. You can have a policy where every single loan application is checked by a human, and you can add whatever affirmation is needed. It will still increase the productivity of those loan officers by a factor of 5. (Put it differently, banks would be able to lay off 80% of their loan officers).
corbulo 4y ago
This is the exact kind of dystopic thinking that is feared with the use of AI. "We regularly take randomized samples and have not found error, your appeal has been denied." I mean come on, ethics anybody??
- sebzim4500 4y ago
  So long as it is better than the thing it replaces, I don't get the big deal.
  - corbulo 4y ago
    Anonymizing risk is not a better system. Think a few consequences down the line...no one knows who theyre lending money to or from, its all figured out by AI/LLMs and as long as the incident rate is below a certain percent in a random sampling it doesn't matter what the errors are. WCGW? Deleting discretion from the current system doesn't fix or improve it. Discretion is a really important part of the current system that prevents it from becoming more insane than it already is.
  - ekianjo 4y ago
    "better" is not a very clear criteria
  - shjake 4y ago
    It’s not though? Deterministic non black box models are superior unless the only thing you’re after is a coherent(ish) reply in English.
  - hughesjj 4y ago
    We can audit and fix the code of our existing laws as we slowly come to consensus, but how do you meaningly iterate on the output of an oracle in a way that converges to the societal consensus of societal structure?
  - concordDance 4y ago
    This is true, but is it actually better?
NBJack 4y ago
The fun part is when a LLM hits that small probabiliy where they decide to go 'offscript'. It can result in a beautifully terrifying cascade of grammatically acceptable nonsense, and joe fun it would be in a legal document. We go from a loan for a home to a generous offer that includes a unicorn, a purple dishwasher, a unicorn, and a few dangling participles at the going market rate, all for the low low rate of 555.555.1212. [END TOKEN]--- WASHINGTON, D.C. President Trump today met with lawmakers to
- cs702 4y ago
  I laughed really hard -- after trying to make sense of your comment! Thank you for posting this :-)
- sebzim4500 4y ago
  I think the hope is that as LLMs get larger these issues will go away. Certainly there are issues with GPT-2 that completely went away when moving to larger models. Honestly, I haven't even seen GPT-3.5-turbo exhibit this behavior myself, although I am willing to believe it could happen. Llama 7B, however, goes off-script constantly.
psychphysic 4y ago
You mean you could ask a LLM to look at a sample of the loans and decide if there was a bias.

rafram 4y ago

I don’t think regulator nepotism is the main reason that the authorities would be uncomfortable with loan decisions being made by a system that definitionally reinforces existing biases and is incapable of thought. It’s just a bad idea!

LawTalkingGuy 4y ago
Well, by definition in these cases it's making a correct decision but that decision isn't popular. If red-lining wasn't profitable the banks would have gone broke and we wouldn't be talking about it. The models aren't wrong, or even wrongly biased. We're systematically failing to ask the right questions because it's not politically correct to even call out the issues we're trying to combat. If you're trying to achieve "equity" then you need to define it and represent it in your equations, not ban various metrics until justice happens.

credit_guy 4y ago

You don't need to check a random sample. You can have a policy where every single loan application is checked by a human, and you can add whatever affirmation is needed. It will still increase the productivity of those loan officers by a factor of 5. (Put it differently, banks would be able to lay off 80% of their loan officers).

corbulo 4y ago

This is the exact kind of dystopic thinking that is feared with the use of AI. "We regularly take randomized samples and have not found error, your appeal has been denied." I mean come on, ethics anybody??

sebzim4500 4y ago
So long as it is better than the thing it replaces, I don't get the big deal.
- corbulo 4y ago
  Anonymizing risk is not a better system. Think a few consequences down the line...no one knows who theyre lending money to or from, its all figured out by AI/LLMs and as long as the incident rate is below a certain percent in a random sampling it doesn't matter what the errors are. WCGW? Deleting discretion from the current system doesn't fix or improve it. Discretion is a really important part of the current system that prevents it from becoming more insane than it already is.
- ekianjo 4y ago
  "better" is not a very clear criteria
- shjake 4y ago
  It’s not though? Deterministic non black box models are superior unless the only thing you’re after is a coherent(ish) reply in English.
- hughesjj 4y ago
  We can audit and fix the code of our existing laws as we slowly come to consensus, but how do you meaningly iterate on the output of an oracle in a way that converges to the societal consensus of societal structure?
- concordDance 4y ago
  This is true, but is it actually better?

NBJack 4y ago

The fun part is when a LLM hits that small probabiliy where they decide to go 'offscript'. It can result in a beautifully terrifying cascade of grammatically acceptable nonsense, and joe fun it would be in a legal document. We go from a loan for a home to a generous offer that includes a unicorn, a purple dishwasher, a unicorn, and a few dangling participles at the going market rate, all for the low low rate of 555.555.1212. [END TOKEN]--- WASHINGTON, D.C. President Trump today met with lawmakers to

cs702 4y ago
I laughed really hard -- after trying to make sense of your comment! Thank you for posting this :-)
sebzim4500 4y ago
I think the hope is that as LLMs get larger these issues will go away. Certainly there are issues with GPT-2 that completely went away when moving to larger models. Honestly, I haven't even seen GPT-3.5-turbo exhibit this behavior myself, although I am willing to believe it could happen. Llama 7B, however, goes off-script constantly.

psychphysic 4y ago

You mean you could ask a LLM to look at a sample of the loans and decide if there was a bias.