> I’d be impressed if you could get anyone on the phone at any financial institution to explain the proprietary inputs to any calculation. Not only would they n

> I’d be impressed if you could get anyone on the phone at any financial institution to explain the proprietary inputs to any calculation. Not only would they not have access to that information, it’d be so off-script to reveal it that there’s no flow chart of comments you could make to get it. As blackbox machine learning algorithms are increasingly being used for these proprietary calculations, people are starting to realize that this sort of explanation probably does need to be made available. It's okay to use machine learning to do real-time image segmentation that can't be done robustly any other way. But it probably shouldn't be okay to use machine learning as "bias laundering" for business decisions that legally should be challengeable. You shouldn't be allowed to hide behind "it's not me, it's the algorithm!" if your algorithm is causing you to look like you're discriminating against a protected class.