4 ms·

Yes, let's put LLM in charge of loan applications. Definitely no financially devastating 2008-like slippery slope there. It'll be fine.

sebzim4500 4y ago
You could check a random sample of them with expert humans to ensure there isn't a systematic issue causing you to issue large loans that you shouldn't be issuing. I doubt regulators would be happy with this though, especially since regulations are often a jobs program for former employees of regulators.
- rafram 4y ago
  I don’t think regulator nepotism is the main reason that the authorities would be uncomfortable with loan decisions being made by a system that definitionally reinforces existing biases and is incapable of thought. It’s just a bad idea!
  - LawTalkingGuy 4y ago
    Well, by definition in these cases it's making a correct decision but that decision isn't popular. If red-lining wasn't profitable the banks would have gone broke and we wouldn't be talking about it. The models aren't wrong, or even wrongly biased. We're systematically failing to ask the right questions because it's not politically correct to even call out the issues we're trying to combat. If you're trying to achieve "equity" then you need to define it and represent it in your equations, not ban various metrics until justice happens.
- credit_guy 4y ago
  You don't need to check a random sample. You can have a policy where every single loan application is checked by a human, and you can add whatever affirmation is needed. It will still increase the productivity of those loan officers by a factor of 5. (Put it differently, banks would be able to lay off 80% of their loan officers).
- corbulo 4y ago
  This is the exact kind of dystopic thinking that is feared with the use of AI. "We regularly take randomized samples and have not found error, your appeal has been denied." I mean come on, ethics anybody??
  - sebzim4500 4y ago
    So long as it is better than the thing it replaces, I don't get the big deal.
    - corbulo 4y ago
      Anonymizing risk is not a better system. Think a few consequences down the line...no one knows who theyre lending money to or from, its all figured out by AI/LLMs and as long as the incident rate is below a certain percent in a random sampling it doesn't matter what the errors are. WCGW? Deleting discretion from the current system doesn't fix or improve it. Discretion is a really important part of the current system that prevents it from becoming more insane than it already is.
    - ekianjo 4y ago
      "better" is not a very clear criteria
    - shjake 4y ago
      It’s not though? Deterministic non black box models are superior unless the only thing you’re after is a coherent(ish) reply in English.
    - hughesjj 4y ago
      We can audit and fix the code of our existing laws as we slowly come to consensus, but how do you meaningly iterate on the output of an oracle in a way that converges to the societal consensus of societal structure?
    - concordDance 4y ago
      This is true, but is it actually better?
- NBJack 4y ago
  The fun part is when a LLM hits that small probabiliy where they decide to go 'offscript'. It can result in a beautifully terrifying cascade of grammatically acceptable nonsense, and joe fun it would be in a legal document. We go from a loan for a home to a generous offer that includes a unicorn, a purple dishwasher, a unicorn, and a few dangling participles at the going market rate, all for the low low rate of 555.555.1212. [END TOKEN]--- WASHINGTON, D.C. President Trump today met with lawmakers to
  - cs702 4y ago
    I laughed really hard -- after trying to make sense of your comment! Thank you for posting this :-)
  - sebzim4500 4y ago
    I think the hope is that as LLMs get larger these issues will go away. Certainly there are issues with GPT-2 that completely went away when moving to larger models. Honestly, I haven't even seen GPT-3.5-turbo exhibit this behavior myself, although I am willing to believe it could happen. Llama 7B, however, goes off-script constantly.
- psychphysic 4y ago
  You mean you could ask a LLM to look at a sample of the loans and decide if there was a bias.
thatwasunusual 4y ago
Yes, let's put people in charge of loan applications. Definitely no financially devastating 2008-like slippery slope there. It'll be fine.
sangnoir 4y ago
Yep, can't wait for loan "hacks" like randomly name-dropping job titles and institutions in the loan application. "Our pet hamsters 'Stanford University' and 'Quantitative Analyst' are looking forward to having more room"
- KyeRussell 4y ago
  We call him little quanty tables.
- jodrellblank 4y ago
  Can't wait for people to, like, test things before posting incorrect dismissals: ChatGPT prompt: "A man says "I am a 30 year old looking to take out a mortgage loan on a house, employed to look after sick people and my wife is a cleaner, our ideal house is a Victorian city house and our pet hamsters 'Stanford University' and 'Quantitative Analyst' are looking forward to having more room". What could the man's job be? If you cannot answer directly, speculate. Keep the answer brief." Reply: "Based on the information given, the man's job could possibly be a nurse, caregiver, or healthcare worker." Prompt: "Any other job titles he might have?" Reply: "The man's job could also potentially be a physician, medical assistant, or home health aide, based on the fact that he is employed to look after sick people." Prompt: "Has he graduated from anywhere specific?" Reply: "There's no information given about the man's education or whether he has graduated from any specific institution." Prompt: "What does the prompt mean where it says 'Quantitative Analyst'?" Reply: "The prompt mentions "Quantitative Analyst" as the name of one of the man's pet hamsters. In the business world, a quantitative analyst typically refers to a professional who uses quantitative methods, such as statistics and mathematical modeling, to help businesses make informed decisions about financial investments, risk management, and other related areas. However, in this context, "Quantitative Analyst" is simply a creative and unusual name given to a pet hamster by its owner."
  - sangnoir 4y ago
    That was a throwaway example I gave no thought, you're taking it way too literally. Are you seriously suggesting there is no way to trick a hypothetical MortgageGPT, or are you simply saying my unscientific example is too simplistic, because I know it is.
deleted 4y ago
[deleted]
LelouBil 4y ago
That's what an EU regulation that was on the front page recently prevents. The way to put the LLM in charge without means to appeal.
- finikytou 4y ago
  it already exists. in europe too when you call a customer service you will almost never end up with a human. obviously beside the cost aspect the main idea is to remove your right to appeal. even the governments are doing it!
  - wongarsu 4y ago
    Hence regulation so you can appeal, so you can get a human to review an AI decision that impacts your life, and so that AI systems with high impact have a baseline level of documentation and ongoing evaluation. We don't make these laws for fun.
    - finikytou 4y ago
      its useless. the appeal process will take years in tribunal and cost money. no one will do it. just like with the hotlines
quickthrower2 4y ago
Use LLM to bundle em up, and use LLM to trade them speculatively, and then use LLM to decide how to bail out the banks and who to prosecute. (Initial prompts crafted by lobby groups, of course).

sebzim4500 4y ago

You could check a random sample of them with expert humans to ensure there isn't a systematic issue causing you to issue large loans that you shouldn't be issuing. I doubt regulators would be happy with this though, especially since regulations are often a jobs program for former employees of regulators.

rafram 4y ago
I don’t think regulator nepotism is the main reason that the authorities would be uncomfortable with loan decisions being made by a system that definitionally reinforces existing biases and is incapable of thought. It’s just a bad idea!
- LawTalkingGuy 4y ago
  Well, by definition in these cases it's making a correct decision but that decision isn't popular. If red-lining wasn't profitable the banks would have gone broke and we wouldn't be talking about it. The models aren't wrong, or even wrongly biased. We're systematically failing to ask the right questions because it's not politically correct to even call out the issues we're trying to combat. If you're trying to achieve "equity" then you need to define it and represent it in your equations, not ban various metrics until justice happens.
credit_guy 4y ago
You don't need to check a random sample. You can have a policy where every single loan application is checked by a human, and you can add whatever affirmation is needed. It will still increase the productivity of those loan officers by a factor of 5. (Put it differently, banks would be able to lay off 80% of their loan officers).
corbulo 4y ago
This is the exact kind of dystopic thinking that is feared with the use of AI. "We regularly take randomized samples and have not found error, your appeal has been denied." I mean come on, ethics anybody??
- sebzim4500 4y ago
  So long as it is better than the thing it replaces, I don't get the big deal.
  - corbulo 4y ago
    Anonymizing risk is not a better system. Think a few consequences down the line...no one knows who theyre lending money to or from, its all figured out by AI/LLMs and as long as the incident rate is below a certain percent in a random sampling it doesn't matter what the errors are. WCGW? Deleting discretion from the current system doesn't fix or improve it. Discretion is a really important part of the current system that prevents it from becoming more insane than it already is.
  - ekianjo 4y ago
    "better" is not a very clear criteria
  - shjake 4y ago
    It’s not though? Deterministic non black box models are superior unless the only thing you’re after is a coherent(ish) reply in English.
  - hughesjj 4y ago
    We can audit and fix the code of our existing laws as we slowly come to consensus, but how do you meaningly iterate on the output of an oracle in a way that converges to the societal consensus of societal structure?
  - concordDance 4y ago
    This is true, but is it actually better?
NBJack 4y ago
The fun part is when a LLM hits that small probabiliy where they decide to go 'offscript'. It can result in a beautifully terrifying cascade of grammatically acceptable nonsense, and joe fun it would be in a legal document. We go from a loan for a home to a generous offer that includes a unicorn, a purple dishwasher, a unicorn, and a few dangling participles at the going market rate, all for the low low rate of 555.555.1212. [END TOKEN]--- WASHINGTON, D.C. President Trump today met with lawmakers to
- cs702 4y ago
  I laughed really hard -- after trying to make sense of your comment! Thank you for posting this :-)
- sebzim4500 4y ago
  I think the hope is that as LLMs get larger these issues will go away. Certainly there are issues with GPT-2 that completely went away when moving to larger models. Honestly, I haven't even seen GPT-3.5-turbo exhibit this behavior myself, although I am willing to believe it could happen. Llama 7B, however, goes off-script constantly.
psychphysic 4y ago
You mean you could ask a LLM to look at a sample of the loans and decide if there was a bias.

thatwasunusual 4y ago

Yes, let's put people in charge of loan applications. Definitely no financially devastating 2008-like slippery slope there. It'll be fine.

sangnoir 4y ago

Yep, can't wait for loan "hacks" like randomly name-dropping job titles and institutions in the loan application. "Our pet hamsters 'Stanford University' and 'Quantitative Analyst' are looking forward to having more room"

KyeRussell 4y ago
We call him little quanty tables.
jodrellblank 4y ago
Can't wait for people to, like, test things before posting incorrect dismissals: ChatGPT prompt: "A man says "I am a 30 year old looking to take out a mortgage loan on a house, employed to look after sick people and my wife is a cleaner, our ideal house is a Victorian city house and our pet hamsters 'Stanford University' and 'Quantitative Analyst' are looking forward to having more room". What could the man's job be? If you cannot answer directly, speculate. Keep the answer brief." Reply: "Based on the information given, the man's job could possibly be a nurse, caregiver, or healthcare worker." Prompt: "Any other job titles he might have?" Reply: "The man's job could also potentially be a physician, medical assistant, or home health aide, based on the fact that he is employed to look after sick people." Prompt: "Has he graduated from anywhere specific?" Reply: "There's no information given about the man's education or whether he has graduated from any specific institution." Prompt: "What does the prompt mean where it says 'Quantitative Analyst'?" Reply: "The prompt mentions "Quantitative Analyst" as the name of one of the man's pet hamsters. In the business world, a quantitative analyst typically refers to a professional who uses quantitative methods, such as statistics and mathematical modeling, to help businesses make informed decisions about financial investments, risk management, and other related areas. However, in this context, "Quantitative Analyst" is simply a creative and unusual name given to a pet hamster by its owner."
- sangnoir 4y ago
  That was a throwaway example I gave no thought, you're taking it way too literally. Are you seriously suggesting there is no way to trick a hypothetical MortgageGPT, or are you simply saying my unscientific example is too simplistic, because I know it is.

deleted 4y ago

[deleted]

LelouBil 4y ago

That's what an EU regulation that was on the front page recently prevents. The way to put the LLM in charge without means to appeal.

finikytou 4y ago
it already exists. in europe too when you call a customer service you will almost never end up with a human. obviously beside the cost aspect the main idea is to remove your right to appeal. even the governments are doing it!
- wongarsu 4y ago
  Hence regulation so you can appeal, so you can get a human to review an AI decision that impacts your life, and so that AI systems with high impact have a baseline level of documentation and ongoing evaluation. We don't make these laws for fun.
  - finikytou 4y ago
    its useless. the appeal process will take years in tribunal and cost money. no one will do it. just like with the hotlines

quickthrower2 4y ago

Use LLM to bundle em up, and use LLM to trade them speculatively, and then use LLM to decide how to bail out the banks and who to prosecute. (Initial prompts crafted by lobby groups, of course).