You might have misunderstood GP (or I might have misunderstood you). What GP is saying is that people with the same language interact more with each other and

You might have misunderstood GP (or I might have misunderstood you). What GP is saying is that people with the same language interact more with each other and therefore reproduce, thus narrowing (averaging?) the gene pool to the speakers of that language. Not that you'd be unable speak a language because of your genes! The cause-effect GP proposed would be language -> genes, not genes -> language (which is what I guessed you meant to refute).