I'm a man and my neighbor is a man. I don't think we're the same. But thinking a born-woman and a trans-woman are different is wrong and oppressive? Transphobic

I'm a man and my neighbor is a man. I don't think we're the same. But thinking a born-woman and a trans-woman are different is wrong and oppressive? Transphobic is an easy word to flaunt to target people who have different opinions. I find it unbelieveable that someone could think impossible to make a difference while not having any problem with trans people. To label people transphobic for having this single opinion is itself bigotry.