Is there really a point in changing keys and passwords? It seems to me that if an attacker got the passwords, I should assume they already installed a root kit

Is there really a point in changing keys and passwords? It seems to me that if an attacker got the passwords, I should assume they already installed a root kit on my server? I'm honestly not sure how to react. I'm not really a sysadmin, but I have a server online. I suppose I could start a new server, but how can I be sure that the provider has already patched all their holes? If they've been hacked, maybe the images they use for preparing new servers have been compromised, too? Might be better to wait a little before restarting everything from scratch?