I think what the parent is trying to convey is that usually when someone means when they say "this decision couldn't be done by a computer", what they mean is t

I think what the parent is trying to convey is that usually when someone means when they say "this decision couldn't be done by a computer", what they mean is that "I don't really know how I'm making this decision well enough to describe it precisely enough to encode it into a set of rules." To be able to decide if something is fair, one has to be able to understand the process by which a decision is made, no? And not just a woolly, after the fact justification that a person might well invent to explain their decision. If we can't reliably look at the inputs and determine what the outputs would be, how are we able to decide if the process is 'fair'? As someone who is looking to understand a system, the argument of "But it couldn't make a human decision" triggers alarm bells of "What you mean is: It couldn't make a non-rational, unfounded, biased decision influenced by whether the person is hungry or not". Is that how important decisions should be made?