One thing that has always bothered me is that almost any argument against polygamy works just as well as an argument against banning extra-marital sexual relati

One thing that has always bothered me is that almost any argument against polygamy works just as well as an argument against banning extra-marital sexual relationships. As you said, polygamy already exists, just without the legal framework calling it a marriage.