I do not think it is offensive because you did not say anything negative about that group of people, but I think they find it offensive because when framed like

I do not think it is offensive because you did not say anything negative about that group of people, but I think they find it offensive because when framed like that it exposes some of the absurdity of that act, which could make them question their own self definition. Human actions often carry absurdity. I think we should recognise it and accept it, instead of trying to ignore it and hope it will disappear (hint, it won't)