Absolutely! There is not enough data here to signify the actual performance of the algorithm. Without more data it's not clear if the algorithm is actually eff

Absolutely! There is not enough data here to signify the actual performance of the algorithm. Without more data it's not clear if the algorithm is actually effective - e.g. if the approval rate of these loans is 80% for instance, a formula of "always return true" could be said to be '80% accurate compared to human decisions'. And if the AI makes the same decisions as a human 95% of the time, but then makes horrific errors the other 5% of the time, it wouldn't be an appropriate replacement for the human. This is the same issue with self driving - it doesn't matter if you make great decisions 99% of the time if 1% of the time you decide to drive into a wall (until you have got your error down to a place where it's safer on aggregate than a human).