I'm having a hard time seeing much danger in that. If your threat model assumes the attacker already has your public key having an easily guessable public key d

I'm having a hard time seeing much danger in that. If your threat model assumes the attacker already has your public key having an easily guessable public key doesnt give them any more information. My biggest worry would be some sort of spoofing. An attacker may be able to impersonate you (or rather a server you control) if they generate a similar vanity key and convince a 3rd party to accept it as yours.