>That kind of feedback is useful to the application. It tells them specific things they can do to improve their chances. In that sense it's very practical, but

>That kind of feedback is useful to the application. It tells them specific things they can do to improve their chances. In that sense it's very practical, but it kicks the can down the road. Maybe the thing has a hidden parameter that represents the risk of the applicant being fired, which increases the threshold by 5% if the salary is a round number. Or it is more likely to reject everyone with an income between 73 and 75k because it learned this is a proxy to a parameter you are explicitly forbidden to have. Let's just say it doesn't have a discontinuity, and actually produces the threshold which is deterministically compared with your income. How does it come up with this threshold? You may not be required to disclosed that to the applicant, but it would be a shame if people will figure out that the threshold is consistently higher for a certain population (for example people who's given name ends with a vowel). It's fairly reasonable to for a regulator to ask you to demonstrate it doesn't do any of this things.