The problem is that ML models are somewhat inscrutable and this inscrutability creates a problem for decision-based software. In particular you don't know why t

The problem is that ML models are somewhat inscrutable and this inscrutability creates a problem for decision-based software. In particular you don't know why the algorithm made the decision. As the author notes, this makes the outcomes hard to contest because it's not possible to know how the decision was made (only in a general sense). For example, you can't know if there is any bias that was taken into account. ML models can absolutely be biased. For example face recognition software has high false positive rates for groups of people that are less familiar. The author goes on to say in the paper that the outcomes from ML seem to not be much better than a human or a simpler statistical model doing the same role - but without the drawback of inscrutability.