HSMs allows to have key stored with the option to disable key export. This means every cryptographic operation must be done by the HSM (commonly through pkcs 11

HSMs allows to have key stored with the option to disable key export. This means every cryptographic operation must be done by the HSM (commonly through pkcs 11 API). HSMs have backup features and the data cannot be restored without a secret split among multiple secret holders (like https://en.m.wikipedia.org/wiki/Shamir%27s_secret_sharing https://en.m.wikipedia.org/wiki/Shamir%27s_secret_sharing). So a backup can be done on physical media and put in a safe. It's a lot of things about key management, HSMs, pkcs interfaces to learn though.