You're not just voting on issues, you're voting for a representative. If the party you agree with doesn't have the votes in Congress, whatever ends up happening

You're not just voting on issues, you're voting for a representative. If the party you agree with doesn't have the votes in Congress, whatever ends up happening in the next 2-6 years will be less likely to be resolved the way you like (or at all). And in fact things may get worse from your point of view. It's not a referendum. You're imparting political power on the party you vote for, relevant to anything that comes up in Congress in the next 2-6 years.