If you are making multiple copies of a private key in different places, why wouldn't you just keep a different private key in each of those places along with it

If you are making multiple copies of a private key in different places, why wouldn't you just keep a different private key in each of those places along with its corresponding public key though? That way, you can remove trust from just one of them if you (e.g.) have your computer stolen.