Most systems would require you to anonymously prove ownership of a the credential, not have an unchanging bearer token. So yes, you could steal the private key

Most systems would require you to anonymously prove ownership of a the credential, not have an unchanging bearer token. So yes, you could steal the private key I guess. But that is no worse than if you have to prove your identity; someone could get your password to your account.