Of course it lowers your risk. Unauthorized users will make fewer connections, meaning fewer login attempts, meaning less of a chance of being compromised. Fol

Of course it lowers your risk. Unauthorized users will make fewer connections, meaning fewer login attempts, meaning less of a chance of being compromised. Following from that, of course you should do it. In the vast majority of cases it's no inconvenience (add a host block to your ~/.ssh/config file if you don't want to type out/remember the port), so you're talking about a change that's all upside. It just doesn't get you off the hook for doing ANYTHING else in terms of security good practice. You don't get to use insecure passwords (in fact, preferably you're disallowing passwords entirely in your server config). You don't get to be lax about cleaning up unneeded accounts. You don't get to chmod everything 777 and ignore which binaries are SUID/SGID. You don't get to slack off patching your box.