I really wish there were a PKI system for this stuff. I.e., my phone has a set of public keys, and it will only ring if someone with a valid private key calls

I really wish there were a PKI system for this stuff. I.e., my phone has a set of public keys, and it will only ring if someone with a valid private key calls me. If I want someone to be able to call me, I generate a new key pair and provide them with the private key. If anything sketchy starts happening, I forget that public key. Or, perhaps more practically, use something like asterisk to prompt them for an extension when calling me. My extensions are 10 digit numbers, and I whenever I give my phone number to someone, the number includes a unique, hard-to-guess extension.