4 ms·

I can see arguments for transparency obviously, I'd just also add that if there is a "black box" model that better estimates risk, it could benefit borrowers ju

I can see arguments for transparency obviously, I'd just also add that if there is a "black box" model that better estimates risk, it could benefit borrowers just as much as lenders, and so everybody can miss out because of artificial limitations. I'd also argue that fair or trustworthy models do not necessitate feature based explanations (which for complicated models are illusory anyway) and there are other ways, including simply stress testing, flagging out of distribution inputs, instance based methods, or simply human review, to be confident that the model works and that it isn't doing something you don't want. Having laypeople or special interests mandate how your computer program works or how you have to pretend it works is not the solution imo

toomuchtodo 4y ago
I can understand how the belief that improved risk pricing provided by a black box model could be arrived at to be the "better" methodology for pricing credit, but importantly, the observability and introspection of the model is a non functional requirement without a pricing/revenue component. Some profit will be given up to meet regulatory requirements; if you can't follow the rules, you don't get to operate, at which point your model value rapidly approaches zero except as a parlor trick (jurisdiction dependent of course). > Having laypeople or special interests mandate how your computer program works or how you have to pretend it works is not the solution imo Agree to disagree. Regulators exist to thwart predatory behavior, including black boxes making decisions no one understands, and companies will not regulate themselves (proven time and time again [1] [2] [3]). [1] https://www.consumerfinance.gov/enforcement/actions/ https://www.consumerfinance.gov/enforcement/actions/ [2] https://www.consumerfinance.gov/enforcement/enforcement-by-the-numbers/ https://www.consumerfinance.gov/enforcement/enforcement-by-t... [3] https://www.consumerfinance.gov/enforcement/payments-harmed-consumers/payments-by-case/ https://www.consumerfinance.gov/enforcement/payments-harmed-...
- rahimnathwani 4y ago
  What makes the use of black box decision models 'predatory behavior'? I can see that, if you're going to do something predatory, you should probably hide stuff. But is an unexplained model inherently predatory? (I'm not expressing an opinion either way about whether unexplainable models are good for lenders' profits.)

toomuchtodo 4y ago

I can understand how the belief that improved risk pricing provided by a black box model could be arrived at to be the "better" methodology for pricing credit, but importantly, the observability and introspection of the model is a non functional requirement without a pricing/revenue component. Some profit will be given up to meet regulatory requirements; if you can't follow the rules, you don't get to operate, at which point your model value rapidly approaches zero except as a parlor trick (jurisdiction dependent of course). > Having laypeople or special interests mandate how your computer program works or how you have to pretend it works is not the solution imo Agree to disagree. Regulators exist to thwart predatory behavior, including black boxes making decisions no one understands, and companies will not regulate themselves (proven time and time again [1] [2] [3]). [1] https://www.consumerfinance.gov/enforcement/actions/ https://www.consumerfinance.gov/enforcement/actions/ [2] https://www.consumerfinance.gov/enforcement/enforcement-by-the-numbers/ https://www.consumerfinance.gov/enforcement/enforcement-by-t... [3] https://www.consumerfinance.gov/enforcement/payments-harmed-consumers/payments-by-case/ https://www.consumerfinance.gov/enforcement/payments-harmed-...

rahimnathwani 4y ago
What makes the use of black box decision models 'predatory behavior'? I can see that, if you're going to do something predatory, you should probably hide stuff. But is an unexplained model inherently predatory? (I'm not expressing an opinion either way about whether unexplainable models are good for lenders' profits.)