You have that exactly backwards: if someone is hitting you with a password bruteforce from a single IP address (which is the only threat that fail2ban mitigates

You have that exactly backwards: if someone is hitting you with a password bruteforce from a single IP address (which is the only threat that fail2ban mitigates) then it is assuredly nothing personal at all. A personal insult, if you are ever unfortunate enough to receive one, will be much more stealthy and neither fail2ban nor any other magical rock will protect you against it.