You have it backwards. The assumption is that there is no privileged reference frame. And that assumption (along with a few others) does imply an invariant sp

You have it backwards. The assumption is that there is no privileged reference frame. And that assumption (along with a few others) does imply an invariant speed. That's the whole point here.