Don't you need a large group prepared to enact revenge on people for voting "wrongly" for non-anonymous voting, in general, to be a problem? That seems like it'

Don't you need a large group prepared to enact revenge on people for voting "wrongly" for non-anonymous voting, in general, to be a problem? That seems like it's not a problem until you've got a large group of fascists/totalitarians already?