One thing to remember if you keep him on and he continues this behavior you might not find out about it for a while. People won't always speak out and tell you.

One thing to remember if you keep him on and he continues this behavior you might not find out about it for a while. People won't always speak out and tell you. What happens when he starts harassing your new hires? If he is new to the company and already this bold what happens in a year or two?