I mentioned this in another response, but I'll refine my argument here. Security by obscurity's negative connotation comes from the ratio of effort on the part

I mentioned this in another response, but I'll refine my argument here. Security by obscurity's negative connotation comes from the ratio of effort on the part of the system implementer to the effort on the part of the attacker. While it may take time to break into an "obscured" system, it also takes a lot of effort to build the system in the first place. Worse, if the security is broken, you have to re-implement from scratch. Better approaches decouple the implementation/methods for enforcing security from the random components of the system that need to be kept secret. Whether or not this increases the effort for the attacker, it drastically reduces the effort for the defender. If an RSA key is cracked, a new one can be generated in less than a second, and once deployed the attacker is back to square one. Having SSH on a different port adds the cost of a full port-scan to any attempt to break in, vs the cost of changing one line in a config file. That all being said, there is a major difference between changing your port number and changing your RSA key: The attacker/defender effort ratio scales linearly (vs exponentially). Every time you change your port number (a fixed cost), the attacker has to run another port scan (another fixed cost) to get past this layer. By contrast, the amount of effort needed to crack an RSA key is exponentially higher than the cost required to create a new one. However, adding a fixed port-scanning cost to each attack does actually correspond to a very high overhead for a non-targeted attack (their costs go up by ~2^16).