I try to avoid it when I can because one female colleague commented on it (not sure if it was related to my uses or someone else's because prior to this I never

I try to avoid it when I can because one female colleague commented on it (not sure if it was related to my uses or someone else's because prior to this I never noticed it, but she brought it to HR). As a guy I feel like it's a "meh" issue, and most of the women I know agree and use the phrase themselves, but if it bugs someone it costs me zero to avoid it.