I hear about this argument and I can't help myself wondering: How many short men are getting excluded from gene pool as a result of this? My take: almost none

I hear about this argument and I can't help myself wondering: How many short men are getting excluded from gene pool as a result of this? My take: almost none. They might get "less desirable" women, but otherwise all men and all women are going to pair up.