3 ms·
Title is oddly framed. What is interesting or useful about merely not-quite predicting what a human will do? Do the AI's 5% discrepency picks perform better or
by Brian_K_White 5y ago
Title is oddly framed. What is interesting or useful about merely not-quite predicting what a human will do?
Do the AI's 5% discrepency picks perform better or worse than the human's picks?
The title is worded to suggest the AI can, or is very close, do the human's job, which could totally be so.
But it could also be that the AI loses 5% vs the human, and the bank only makes 5% on loans in the first place, and so losing 5% of them is like totally erasing the entire income of the bank (from loans) which makes the AI a complete Hindenberg, rather the opposite of the implication from the title.
- kqr 5y agoGiven the research by Meehl, if there's a decent numeric rule and a human decision that are closely predicting each other, my money would be on the numeric rule being slightly more accurate.
- Closi 5y agoAbsolutely! There is not enough data here to signify the actual performance of the algorithm. Without more data it's not clear if the algorithm is actually effective - e.g. if the approval rate of these loans is 80% for instance, a formula of "always return true" could be said to be '80% accurate compared to human decisions'. And if the AI makes the same decisions as a human 95% of the time, but then makes horrific errors the other 5% of the time, it wouldn't be an appropriate replacement for the human. This is the same issue with self driving - it doesn't matter if you make great decisions 99% of the time if 1% of the time you decide to drive into a wall (until you have got your error down to a place where it's safer on aggregate than a human).
- jsemrau 5y agoIt takes a long while to validate this statistically though as default cohorts move through the lending cycle.