11 ms·

I used to use lots of port knocking setups to hide my ssh port. That was, until I discovered Tailscale's SSH setup. Now my SSH is run over wireguard which is ve

by fooqux 2y ago

I used to use lots of port knocking setups to hide my ssh port. That was, until I discovered Tailscale's SSH setup. Now my SSH is run over wireguard which is very stealthy.

ray_v 2y ago
Same. It's amazing not having my server hammered by malicious actors and hardening it by not even offering the ssh service on the primary network interface
- Demiurge 2y ago
  Why is it so amazing? Sounds more complicated than fail2ban. I've been installing fail2ban for decades on countless servers, using decent passwords, and have never had SSH get brute-forced. It’s anecdotal, but if you’re getting blocked after three wrong attempts, the chances of a successful attack are pretty small. So, why bother with nonstandard ports or even other protocols?
  - tptacek 2y ago
    Getting your SSH brute-forced shouldn't be possible with or without screening, because absolutely nobody in 2024 should be using passwords with SSH to begin with. This is the most frustrating thing about fail2ban cargo culting; fail2ban hasn't made sense since the era of multiuser Unix shell accounts, which is 2 decades in the past.
    - fragmede 2y ago
      You've been mugged in a foreign country and have no phone, no wallet, or keys. How do you break back into your digital life? Everything's got two factor and you've not got your something-you-have factor. I'm all for almost all my servers not accepting passwords, but it's a scenario that I think about, so there's one server running ssh on an non-default port that takes a password so I can break back in using only what's in my head (hopefully I don't get hit so hard in this mugging so as to forget what I've memorized).
      - tptacek 2y ago
        This is not a good reason to put passwords on your server SSH accounts. Encrypt an SSH key you can recover.
        Demiurge 2y ago
        Really, what are the chances if you have a decent password and 3 attempts per 24 hours? Why would I take something %0.000000001 likely and make it %0.0000000000000000000001 likely if there is an added risk of %0.001 my house burns down and I will lose all my access?
        mkatx 2y ago
        Not to argue any of your other points, except.. you could get a lot more than 3 tries with a rotating proxy or a botnet.
        Demiurge 2y ago
        That is also true. That’s a good reason to block certain IP segments.
        kazinator 2y ago
        I've never seen SSH attackers probe a space of user IDs in such a way that they would eventually find a user ID like Z@an4ar. In fact, they just stick to standard user IDs like root, and probe only the password spaces. If your super user account is not named root, or not available by that name via SSH, it is safe from attacks which only try that name. sshd can be configured simply not to allow root logins. To use root, you log into some other account and then su. That other account can have a name that attackers will never try, and a decent password. Then, if you happen to be using a log based banning system, since you know that no legitimate user would be trying the name root, you can impose an instant ban on such an IP address, with a long duration. It's really just for reducing traffic more than anything. Regarding aliasing root, you can create an alias for the UID 0 user simply by editing your password and shadow files to create duplicate entry. If the root entry appears first, then that name is still used whenever a UID is resolved to a user name, like in your ls -l and whatnot. The shadow entry for root can have a star in the password field so that it cannot be used for logging in by any means; only the alternative name can be used via the other entry that has a password set up in its shadow entry.
  - sevg 2y ago
    > using decent passwords For anyone else reading this, generally speaking one shouldn't use passwords for SSH in 2024. Use public key auth instead. > Why is it so amazing? OpenSSH isn't invulnerable. It can have zero-day vulnerabilities. But if it isn't even listening on the public internet, that's one less attack vector.
    - Demiurge 2y ago
      Generally speaking, you’re right, but I have servers I want to be able to access from anywhere, because I support some app running on them. Until 1password agent setup, having keys only and password disabled was too difficult, and yet, also unnecessary. Zero day ssh bug? I’m not NSA, how often does this happen to random servers?? Again, never have been hacked in more than 20 years. Still support some servers with ~6 year uptime.
      - sevg 2y ago
        Ah yes, the "I've never been hacked so I must be secure" argument ;) Unfortunately, you're not convincing anyone. Amongst the security conscious, multi-year uptimes are the opposite of a brag. And it doesn't matter how you spin it, key-based auth is best practice, as is reducing your attack surface. It seems that some of these measures are too difficult for you, and that's fine. But trying to argue that the measures are pointless is just false.
        Demiurge 2y ago
        I’m not trying to convince anyone, I’m trying to understand what drives sone security focused people to make things more complicated and harder without practical justification. So, are you NSA? How many servers have you lost to the password attack vector?
        sevg 2y ago
        > I'm trying to understand Well on the one hand you make it seem like you're here for genuine adult conversation. On the other hand you call people that disagree with you the "NSA". And that is the point this conservation has outlived its usefulness :)
  - jjeaff 2y ago
    It can be difficult to pass some PCI compliance tests if your ssh port is available to the world. OpenSSH also leaks some information about your server unless you recompile it with those options removed.
    - Demiurge 2y ago
      Yeah, that makes sense, sometimes orgs or audits have requirements.
  - thyristan 2y ago
    fail2ban is dangerous imho. First, it will only block high-frequency maliciousness. If an attacker knows to stay below the default ban frequencies, or change endpoints often enough, they will have free reign. Second, fail2ban is a DoS risk, attackers can spoof connections from an IP they want to switch off. Third, fail2ban relies on parsing of textual logs. This is vulnerable to all kinds of injection attacks (there have been some CVEs to that end) where an attacker injects patterns that the fail2ban heuristics will latch on to, and wildly ban stuff. So you should not rely on fail2ban to keep you safe from anything, and you are introducing DoS risks. Very bad tradeoff imho, making it only good as a last resort.
    - Joel_Mckay 2y ago
      Like all these problems, the answer is it depends... In general, fail2ban is often setup to indirectly whisper to a firewall API. The firewall is smart enough to enforce white lists, and custom rate-limiting traffic rules. i.e. to survive a DoS, the server enforces the traffic profile that chokes off users violating normal rules (ratio of TCP packet types, UDP volume %, and TTL count variance per IP.) In general, for DDoS you just drop the traffic to a fixed cost CDN with a CAPTCHA to issue real users session tokens, or issue a 302 to 127.0.0.1 for everyone else hammering the site. Have a nice day, =3
deleted 2y ago
[deleted]
deleted 2y ago
[deleted]
jjeaff 2y ago
I also love tailscale's ssh option. I have been using it for a few months now. But I'm a little bit scared that if the tailscale daemon crashes, I'll lose access to my server.
- dvzk 2y ago
  You would also be locked out if you ran OpenSSH on Tailscale's autoconfigured WG interface. Setup WireGuard manually, or enable serial console login, or make sure your servers are dispensible. Tailscale (and Nebula) mostly alleviate the last case.
- fooqux 2y ago
  Same could be said for the sshd service crashing. And yeah, I suppose you could say it's got a longer track record, but I've yet to have tailscale crash on me in a few years.

by fooqux 2y ago

I used to use lots of port knocking setups to hide my ssh port. That was, until I discovered Tailscale's SSH setup. Now my SSH is run over wireguard which is very stealthy.

ray_v 2y ago

Same. It's amazing not having my server hammered by malicious actors and hardening it by not even offering the ssh service on the primary network interface

Demiurge 2y ago
Why is it so amazing? Sounds more complicated than fail2ban. I've been installing fail2ban for decades on countless servers, using decent passwords, and have never had SSH get brute-forced. It’s anecdotal, but if you’re getting blocked after three wrong attempts, the chances of a successful attack are pretty small. So, why bother with nonstandard ports or even other protocols?
- tptacek 2y ago
  Getting your SSH brute-forced shouldn't be possible with or without screening, because absolutely nobody in 2024 should be using passwords with SSH to begin with. This is the most frustrating thing about fail2ban cargo culting; fail2ban hasn't made sense since the era of multiuser Unix shell accounts, which is 2 decades in the past.
  - fragmede 2y ago
    You've been mugged in a foreign country and have no phone, no wallet, or keys. How do you break back into your digital life? Everything's got two factor and you've not got your something-you-have factor. I'm all for almost all my servers not accepting passwords, but it's a scenario that I think about, so there's one server running ssh on an non-default port that takes a password so I can break back in using only what's in my head (hopefully I don't get hit so hard in this mugging so as to forget what I've memorized).
    - tptacek 2y ago
      This is not a good reason to put passwords on your server SSH accounts. Encrypt an SSH key you can recover.
      - Demiurge 2y ago
        Really, what are the chances if you have a decent password and 3 attempts per 24 hours? Why would I take something %0.000000001 likely and make it %0.0000000000000000000001 likely if there is an added risk of %0.001 my house burns down and I will lose all my access?
        mkatx 2y ago
        Not to argue any of your other points, except.. you could get a lot more than 3 tries with a rotating proxy or a botnet.
        Demiurge 2y ago
        That is also true. That’s a good reason to block certain IP segments.
        kazinator 2y ago
        I've never seen SSH attackers probe a space of user IDs in such a way that they would eventually find a user ID like Z@an4ar. In fact, they just stick to standard user IDs like root, and probe only the password spaces. If your super user account is not named root, or not available by that name via SSH, it is safe from attacks which only try that name. sshd can be configured simply not to allow root logins. To use root, you log into some other account and then su. That other account can have a name that attackers will never try, and a decent password. Then, if you happen to be using a log based banning system, since you know that no legitimate user would be trying the name root, you can impose an instant ban on such an IP address, with a long duration. It's really just for reducing traffic more than anything. Regarding aliasing root, you can create an alias for the UID 0 user simply by editing your password and shadow files to create duplicate entry. If the root entry appears first, then that name is still used whenever a UID is resolved to a user name, like in your ls -l and whatnot. The shadow entry for root can have a star in the password field so that it cannot be used for logging in by any means; only the alternative name can be used via the other entry that has a password set up in its shadow entry.
- sevg 2y ago
  > using decent passwords For anyone else reading this, generally speaking one shouldn't use passwords for SSH in 2024. Use public key auth instead. > Why is it so amazing? OpenSSH isn't invulnerable. It can have zero-day vulnerabilities. But if it isn't even listening on the public internet, that's one less attack vector.
  - Demiurge 2y ago
    Generally speaking, you’re right, but I have servers I want to be able to access from anywhere, because I support some app running on them. Until 1password agent setup, having keys only and password disabled was too difficult, and yet, also unnecessary. Zero day ssh bug? I’m not NSA, how often does this happen to random servers?? Again, never have been hacked in more than 20 years. Still support some servers with ~6 year uptime.
    - sevg 2y ago
      Ah yes, the "I've never been hacked so I must be secure" argument ;) Unfortunately, you're not convincing anyone. Amongst the security conscious, multi-year uptimes are the opposite of a brag. And it doesn't matter how you spin it, key-based auth is best practice, as is reducing your attack surface. It seems that some of these measures are too difficult for you, and that's fine. But trying to argue that the measures are pointless is just false.
      - Demiurge 2y ago
        I’m not trying to convince anyone, I’m trying to understand what drives sone security focused people to make things more complicated and harder without practical justification. So, are you NSA? How many servers have you lost to the password attack vector?
        sevg 2y ago
        > I'm trying to understand Well on the one hand you make it seem like you're here for genuine adult conversation. On the other hand you call people that disagree with you the "NSA". And that is the point this conservation has outlived its usefulness :)
- jjeaff 2y ago
  It can be difficult to pass some PCI compliance tests if your ssh port is available to the world. OpenSSH also leaks some information about your server unless you recompile it with those options removed.
  - Demiurge 2y ago
    Yeah, that makes sense, sometimes orgs or audits have requirements.
- thyristan 2y ago
  fail2ban is dangerous imho. First, it will only block high-frequency maliciousness. If an attacker knows to stay below the default ban frequencies, or change endpoints often enough, they will have free reign. Second, fail2ban is a DoS risk, attackers can spoof connections from an IP they want to switch off. Third, fail2ban relies on parsing of textual logs. This is vulnerable to all kinds of injection attacks (there have been some CVEs to that end) where an attacker injects patterns that the fail2ban heuristics will latch on to, and wildly ban stuff. So you should not rely on fail2ban to keep you safe from anything, and you are introducing DoS risks. Very bad tradeoff imho, making it only good as a last resort.
  - Joel_Mckay 2y ago
    Like all these problems, the answer is it depends... In general, fail2ban is often setup to indirectly whisper to a firewall API. The firewall is smart enough to enforce white lists, and custom rate-limiting traffic rules. i.e. to survive a DoS, the server enforces the traffic profile that chokes off users violating normal rules (ratio of TCP packet types, UDP volume %, and TTL count variance per IP.) In general, for DDoS you just drop the traffic to a fixed cost CDN with a CAPTCHA to issue real users session tokens, or issue a 302 to 127.0.0.1 for everyone else hammering the site. Have a nice day, =3

deleted 2y ago

[deleted]

deleted 2y ago

[deleted]

jjeaff 2y ago

I also love tailscale's ssh option. I have been using it for a few months now. But I'm a little bit scared that if the tailscale daemon crashes, I'll lose access to my server.

dvzk 2y ago
You would also be locked out if you ran OpenSSH on Tailscale's autoconfigured WG interface. Setup WireGuard manually, or enable serial console login, or make sure your servers are dispensible. Tailscale (and Nebula) mostly alleviate the last case.
fooqux 2y ago
Same could be said for the sshd service crashing. And yeah, I suppose you could say it's got a longer track record, but I've yet to have tailscale crash on me in a few years.