You can potentially go down those rabbit holes, then an attacker gets fixated on you for being so unusual and they come back doing only 2 attempts per ip per da

You can potentially go down those rabbit holes, then an attacker gets fixated on you for being so unusual and they come back doing only 2 attempts per ip per day with thousands of IPs. I have been on both sides of this game many times. Better time investment in security is just do not have ssh or a shell in production in the first place, because ssh is probably not required for your production application to function.