Did no one ever pull back and just punch him in the face? If not, why? Even the worst possible repercussions don't stop some people. And if so, did it have any

Did no one ever pull back and just punch him in the face? If not, why? Even the worst possible repercussions don't stop some people. And if so, did it have any effect?