The key thing here is _user controlled_ private keys: this means that those can leak, and if they leak, you're in trouble. A better solution is to use ephemera

The key thing here is _user controlled_ private keys: this means that those can leak, and if they leak, you're in trouble. A better solution is to use ephemeral SSH keys generated using an SSH CA. This kind of thing can be implemented with Hashicorp Vault, though I'm sure there are plenty of other solutions out there. It simplifies key checking on servers too, as they just need to the details of the signer and there's no need to juggle keys in LDAP.