I've also heard this line of reasoning referred to as "the soft bigotry of lowered expectations", due to its implication that not all groups of people can be he

I've also heard this line of reasoning referred to as "the soft bigotry of lowered expectations", due to its implication that not all groups of people can be held to the same standards of behavior as society as a whole.