You can not prevent the phone number form being found eventually but that's not the goal you just need to make it more expensive than a phone number could ever

You can not prevent the phone number form being found eventually but that's not the goal you just need to make it more expensive than a phone number could ever be worth to someone. If you use a secret you have the same problem as before the legit system need to have access to the secret but an attacker should never get it. So if an attacker gets hashes and the secret(s) he has everything.