19 ms·

We scanned the Internet for port 22

126 points by yammesicka 13y ago

zippergz 13y ago
Why do people think it's worthwhile to file abuse complaints about port 22 connections? If I wanted to file a complaint about every random connection to port 22 on one of my machines, it would be a full time job...
- WizzleKake 13y ago
  A lot of 'rouge' ssh traffic comes from hacked machines. If you contact the abuse department of a provider, they may get in touch with the owner of the machine to let them know that their box has been compromised.
  - zippergz 13y ago
    Yes I know that, but I'm not convinced that the cost/benefit really makes sense for complaining. Back in my more idealistic days I did report a lot of abuse. But it's so rare for the recipient to actually do anything about it, and the volume of scans so high, it's just not worth my time.
  - derleth 13y ago
    > A lot of 'rouge' ssh traffic comes from hacked machines. And they're owned by the Hell's Angles or Red China, I bet.
    - riffraff 13y ago
      Rouge Angles of Satin
- marshray 13y ago
  Well, if you had a "class A" address space, you would have seen 16777216 connection attempts in your firewall logs. Yeah, I imagine that would be a full time job.
- SpenserJ 13y ago
  Don't manually file them in that case. Install Fail2ban and automatically ban anyone with x failed attempts, and have it email fail2ban@blocklist.de. Their system will look up the correct abuse department, and forward it along on your behalf, along with thousands of others that report it. A large portion of them probably go ignored, but if you can get a fraction of them taken offline, then why not spend the five minutes to set it up? The bottom half of http://spenserj.com/blog/2013/07/15/securing-a-linux-server/ http://spenserj.com/blog/2013/07/15/securing-a-linux-server/ shows how few things you need to do to set up Fail2ban and Blocklist.de
  - robertgraham 13y ago
    My scanner doesn't try to brute-force the logins. It's just grabbing the banner.
andrewcooke 13y ago
if you're curious about your server keys: cd /etc/ssh for pub in `ls -1 *.pub`; do ssh-keygen -l -f $pub; done [edit: sorry; thought no-one had replied. earlier i asked what i should worry about in ssh config. edit2: actually, i am using fail2ban.]
- noir_lord 13y ago
  If you are using the default config the chances are you are using passwords to login? This is bad because unless you are using something like fail2ban you are leaving yourself open to brute force attacks. Rough best practice is, 1) Use SSH keys to login 2) Enforce SSH to login (no password) 3) Disable Root Logins (login then elevate permissions not logging in as root) 4) Move from port 22 to something else (if you do 1-3 this shouldn't matter I do it mostly to keep the crap out my logs) 5) install something like fail2ban to automatically blacklist IP addresses.
  - jebblue 13y ago
    I'd add setting these too: LoginGraceTime 30 MaxAuthTries 3 Also consider using AllowUsers to govern from where a valid connection can originate if you know that in advance.
    - rschmitty 13y ago
      I change my login grace time to 3 seconds, you either have the key or you don't :) No password entering
      - jebblue 13y ago
        Hmm, good point, with keys, 30 is a bit long these days.
      - wiml 13y ago
        Nice idea until you're logging in from a slow/overloaded connection, or your ssh host is being flooded with traffic, or has a runaway process that's causing it to thrash, or lots of other reasons you might want to use ssh. I can't think of a scenario where a really short login grace time would protect you from an attacker, either. What's the motivation?
noir_lord 13y ago
> However, please consider adding our scanner (71.6.151.167) to your "whitelist". We are well-known cyber-sec researchers, we aren't trying anything nefarious or evil, and we are being as transparent as possible about our scans. Sure and while we are at it I'll fetch the lube. I don't complain about port 22 connections because none of my machines run anything on port 22 (I move SSH to a random port mostly to deter whatever is coming out of China this week). Even if you do find the port you still have to get around the ssh key's (so unless you are the NSA (j/k)), you could try an exploit against ssh but as it doesn't report its version good look with that and do try to avoid triggering fail2ban. I'm not a systems administrator (I guess I'd be DevOps) but I know enough to know I don't know enough to disregard best practice.
- laumars 13y ago
  I kind of like running SSH on the default port as it gives me a list of IPs to firewall against on other web facing services across my infrastructure. If someone is trying to break into my server(s) via SSH, then I sure as hell don't want them to have access to any other services regardless of how secure they might be.
  - tlrobinson 13y ago
    Couldn't you just have your firewall log attempted connections to 22, regardless of whether or not you have SSH running on it?
    - noir_lord 13y ago
      I've also seen people run a fake SSH server on 22 and log attempts (if you know that you don't use 22 then any connection attempt on 22 is interesting).
    - laumars 13y ago
      "Just"? That requires more work to set up than my method and offers no additional security benefits as you're still running an open SSH port. You're falling into the common trap of security through obscurity. But in fact it's very easy to find which port SSH listening on even when you're using a non-standard on. However if you harden SSH properly then it doesnt matter what port it's listening on. So your method is more work for zero gain
chatman 13y ago
What good does it bring anyway to repeatedly scan for the port 22? Doing it once a month makes sense, but doing it daily doesn't.
- tlrobinson 13y ago
  They said their next scan will be in October... "We'll be scanning SSH again in October"
  - DanBC 13y ago
    They also said > We are going to be extending this to more ports, such as FTP and SMTP. Soon, we should have weekly scans going for about 10 ports. I'm moving slowly forward to resolve abuse complaints, like this one generated for port 22. We plan on publishing the results, such as the anonymous counts above, in a nice weekly report for the public. So, weekly, not daily.
rsync 13y ago
People. Port knocking. Seriously. There are very few things that have come and gone in the almost 20 years that I have been securing systems and dealing with attacks, but port knocking is a substantive, Truly Good Thing. Works for a lot of other ports too, but ssh is the obvious one.
- mwcampbell 13y ago
  Which port knocking implementation do you recommend?
  - yareally 13y ago
    I've been using knockd[1] for the last 5 or so years. Also has prebuilt binaries (and in many repositories[2]) for almost every distro, including cygwin, native windows and android. [1] http://www.zeroflux.org/projects/knock/ http://www.zeroflux.org/projects/knock/ [2] http://packages.debian.org/wheezy/knockd http://packages.debian.org/wheezy/knockd
    - narcissus 13y ago
      Not knowing much about this stuff, are there any clients that will 'automate' the knock for you? If so, can you make any suggestions? Or is this a matter of CLI SSH will do it, but I'm just not finding it?
      - yareally 13y ago
        https://help.ubuntu.com/community/PortKnocking https://help.ubuntu.com/community/PortKnocking (also the same config basically when using Debian) It's a client/server thing, so you have to have the server running on the PC you intend to connect to (like when using ssh). Never used it in any other way than with the CLI and not with every server I admin or use as there's other best practices also mentioned in the thread that are typically good enough without throwing in port knocking like: https://news.ycombinator.com/item?id=6384313 https://news.ycombinator.com/item?id=6384313 (and the child reply) https://news.ycombinator.com/item?id=6384418 https://news.ycombinator.com/item?id=6384418 https://news.ycombinator.com/item?id=6384457 https://news.ycombinator.com/item?id=6384457 Most things you can just use knocking and connect with ssh forwarding/proxy and tunnel everything through it like DB clients, IDE connections instead of having a bunch of ports open to internet. Assuming you don't kill the connection, then you can keep reusing it as a tunnel for anything else until rebooting or losing internet connection.
pdenya 13y ago
> Yesterday (Sept. 12) we scanned the entire Internet for port 22 > … result of 1,730,887 systems on the Internet … (Note: this is actually only 60% of the Internet So there are only 2,884,811-ish machines on the internet?
- bigiain 13y ago
  I'd guess a closer interpretation might be "there are only ~2,900,000-ish machines publicly answering on port 22 on the internet". (Most of my "important" servers have port 22 firewalled off an only open to a small set of external ip addresses. Some of them aren't running ssh at all.)
matt__rose 13y ago
OpenSSH 4.3 is likely the most popular because it is the version that comes with (redhat|centos|scientific|oracle) linux 5. It's still widely in use. 4.3 had a lot of bugs, but redhat has been backporting fixes to it since it came out.
- justincormack 13y ago
  I was wondering about correlating these to distros to get real world usage. Distinguishing redhat vs centos is not going to be easy though I guess.
virtualwhys 13y ago
I used to run SSH over some port != 22, does the trick to some degree. After I picked up a Cisco ASA, went back to standard port 22 but only allow access for connected VPN users. Of course if the ASA goes down, so does the entire network, yelp. SmartNET contract/warranty comes in handy, and the data center having backup ASAs on site for quick swap is pretty useful as well.
- Qantourisc 13y ago
  This doesn't make it secure, but this DOES save a lot on having a log full with login attempts from all kinds of IP addresses.
  - tsahyt 13y ago
    When I set up my first server I was thoroughly surprised at the amount of login attempts. By now I've got most of Asia on a blacklist for port 22 (which doesn't matter, since I'm not going to connect to my server from an Asian IP in the foreseeable future). That throttled it down to about 30 attempts/week, which is manageable in terms of log reading.
  - virtualwhys 13y ago
    Excuse me? How is VPN not secure? The ASA won't let any traffic through on port 22, period. You have to be connected over VPN in order to even attempt SSH access to servers behind the ASA.
- justincormack 13y ago
  Of course Cisco is at the top of the NSA insert backdoors list one assumes.
telephonetemp 13y ago
I run a VPS that only is only accessible over IPv6. I wonder if they'll ever scan it. Is there a way to narrow down the whole IPv6 search space to the most populated subranges?
- justincormack 13y ago
  Well you could get hold of the routing tables and filter out unallocated space. But thats not going to help much as there are still 64 low end bits to guess, some people use xxx::1 and ::2 but others use mac address based IPs (and random ones). So we are probably at the best point to portscan before the possibility goes away again...
rorrr2 13y ago
Maybe you should randomize the order of IPs you scan, so you don't hit the same network at 100K requests/sec.
- gpvos 13y ago
  Of course they do, they are not stupid. Did you read the article?
sebcat 13y ago
I'm on a phone with shitty wifi atm so looking at the code would be hard to say the least. However, I am curious, how do you deal with packet loss? The reason I'm asking is because, most people who claim to "scan the Internet" assume that the network is reliable. And they don't follow up on potential false negatives. If you scan the IPv4 address-space sequentially while only limiting bandwidth or time, rest assured that packets will be dropped.
swalsh 13y ago
Reading through the source code, there's actually a reasonably well used goto (https://github.com/robertdavidgraham/masscan/blob/master/src/main-throttle.c https://github.com/robertdavidgraham/masscan/blob/master/src...)
mabhatter 13y ago
If they are only hitting a port 22 a few times, who's wasting the ISP's time with abuse requests... I suppose its probably those guys with Class-A blocks and too much time on their hands.

126 points by yammesicka 13y ago

Why do people think it's worthwhile to file abuse complaints about port 22 connections? If I wanted to file a complaint about every random connection to port 22 on one of my machines, it would be a full time job...

WizzleKake 13y ago
A lot of 'rouge' ssh traffic comes from hacked machines. If you contact the abuse department of a provider, they may get in touch with the owner of the machine to let them know that their box has been compromised.
- zippergz 13y ago
  Yes I know that, but I'm not convinced that the cost/benefit really makes sense for complaining. Back in my more idealistic days I did report a lot of abuse. But it's so rare for the recipient to actually do anything about it, and the volume of scans so high, it's just not worth my time.
- derleth 13y ago
  > A lot of 'rouge' ssh traffic comes from hacked machines. And they're owned by the Hell's Angles or Red China, I bet.
  - riffraff 13y ago
    Rouge Angles of Satin
marshray 13y ago
Well, if you had a "class A" address space, you would have seen 16777216 connection attempts in your firewall logs. Yeah, I imagine that would be a full time job.
SpenserJ 13y ago
Don't manually file them in that case. Install Fail2ban and automatically ban anyone with x failed attempts, and have it email fail2ban@blocklist.de. Their system will look up the correct abuse department, and forward it along on your behalf, along with thousands of others that report it. A large portion of them probably go ignored, but if you can get a fraction of them taken offline, then why not spend the five minutes to set it up? The bottom half of http://spenserj.com/blog/2013/07/15/securing-a-linux-server/ http://spenserj.com/blog/2013/07/15/securing-a-linux-server/ shows how few things you need to do to set up Fail2ban and Blocklist.de
- robertgraham 13y ago
  My scanner doesn't try to brute-force the logins. It's just grabbing the banner.

andrewcooke 13y ago

if you're curious about your server keys: cd /etc/ssh for pub in `ls -1 *.pub`; do ssh-keygen -l -f $pub; done [edit: sorry; thought no-one had replied. earlier i asked what i should worry about in ssh config. edit2: actually, i am using fail2ban.]

noir_lord 13y ago
If you are using the default config the chances are you are using passwords to login? This is bad because unless you are using something like fail2ban you are leaving yourself open to brute force attacks. Rough best practice is, 1) Use SSH keys to login 2) Enforce SSH to login (no password) 3) Disable Root Logins (login then elevate permissions not logging in as root) 4) Move from port 22 to something else (if you do 1-3 this shouldn't matter I do it mostly to keep the crap out my logs) 5) install something like fail2ban to automatically blacklist IP addresses.
- jebblue 13y ago
  I'd add setting these too: LoginGraceTime 30 MaxAuthTries 3 Also consider using AllowUsers to govern from where a valid connection can originate if you know that in advance.
  - rschmitty 13y ago
    I change my login grace time to 3 seconds, you either have the key or you don't :) No password entering
    - jebblue 13y ago
      Hmm, good point, with keys, 30 is a bit long these days.
    - wiml 13y ago
      Nice idea until you're logging in from a slow/overloaded connection, or your ssh host is being flooded with traffic, or has a runaway process that's causing it to thrash, or lots of other reasons you might want to use ssh. I can't think of a scenario where a really short login grace time would protect you from an attacker, either. What's the motivation?

noir_lord 13y ago

> However, please consider adding our scanner (71.6.151.167) to your "whitelist". We are well-known cyber-sec researchers, we aren't trying anything nefarious or evil, and we are being as transparent as possible about our scans. Sure and while we are at it I'll fetch the lube. I don't complain about port 22 connections because none of my machines run anything on port 22 (I move SSH to a random port mostly to deter whatever is coming out of China this week). Even if you do find the port you still have to get around the ssh key's (so unless you are the NSA (j/k)), you could try an exploit against ssh but as it doesn't report its version good look with that and do try to avoid triggering fail2ban. I'm not a systems administrator (I guess I'd be DevOps) but I know enough to know I don't know enough to disregard best practice.

laumars 13y ago
I kind of like running SSH on the default port as it gives me a list of IPs to firewall against on other web facing services across my infrastructure. If someone is trying to break into my server(s) via SSH, then I sure as hell don't want them to have access to any other services regardless of how secure they might be.
- tlrobinson 13y ago
  Couldn't you just have your firewall log attempted connections to 22, regardless of whether or not you have SSH running on it?
  - noir_lord 13y ago
    I've also seen people run a fake SSH server on 22 and log attempts (if you know that you don't use 22 then any connection attempt on 22 is interesting).
  - laumars 13y ago
    "Just"? That requires more work to set up than my method and offers no additional security benefits as you're still running an open SSH port. You're falling into the common trap of security through obscurity. But in fact it's very easy to find which port SSH listening on even when you're using a non-standard on. However if you harden SSH properly then it doesnt matter what port it's listening on. So your method is more work for zero gain

chatman 13y ago

What good does it bring anyway to repeatedly scan for the port 22? Doing it once a month makes sense, but doing it daily doesn't.

tlrobinson 13y ago
They said their next scan will be in October... "We'll be scanning SSH again in October"
- DanBC 13y ago
  They also said > We are going to be extending this to more ports, such as FTP and SMTP. Soon, we should have weekly scans going for about 10 ports. I'm moving slowly forward to resolve abuse complaints, like this one generated for port 22. We plan on publishing the results, such as the anonymous counts above, in a nice weekly report for the public. So, weekly, not daily.

rsync 13y ago

People. Port knocking. Seriously. There are very few things that have come and gone in the almost 20 years that I have been securing systems and dealing with attacks, but port knocking is a substantive, Truly Good Thing. Works for a lot of other ports too, but ssh is the obvious one.

mwcampbell 13y ago
Which port knocking implementation do you recommend?
- yareally 13y ago
  I've been using knockd[1] for the last 5 or so years. Also has prebuilt binaries (and in many repositories[2]) for almost every distro, including cygwin, native windows and android. [1] http://www.zeroflux.org/projects/knock/ http://www.zeroflux.org/projects/knock/ [2] http://packages.debian.org/wheezy/knockd http://packages.debian.org/wheezy/knockd
  - narcissus 13y ago
    Not knowing much about this stuff, are there any clients that will 'automate' the knock for you? If so, can you make any suggestions? Or is this a matter of CLI SSH will do it, but I'm just not finding it?
    - yareally 13y ago
      https://help.ubuntu.com/community/PortKnocking https://help.ubuntu.com/community/PortKnocking (also the same config basically when using Debian) It's a client/server thing, so you have to have the server running on the PC you intend to connect to (like when using ssh). Never used it in any other way than with the CLI and not with every server I admin or use as there's other best practices also mentioned in the thread that are typically good enough without throwing in port knocking like: https://news.ycombinator.com/item?id=6384313 https://news.ycombinator.com/item?id=6384313 (and the child reply) https://news.ycombinator.com/item?id=6384418 https://news.ycombinator.com/item?id=6384418 https://news.ycombinator.com/item?id=6384457 https://news.ycombinator.com/item?id=6384457 Most things you can just use knocking and connect with ssh forwarding/proxy and tunnel everything through it like DB clients, IDE connections instead of having a bunch of ports open to internet. Assuming you don't kill the connection, then you can keep reusing it as a tunnel for anything else until rebooting or losing internet connection.

pdenya 13y ago

> Yesterday (Sept. 12) we scanned the entire Internet for port 22 > … result of 1,730,887 systems on the Internet … (Note: this is actually only 60% of the Internet So there are only 2,884,811-ish machines on the internet?

bigiain 13y ago
I'd guess a closer interpretation might be "there are only ~2,900,000-ish machines publicly answering on port 22 on the internet". (Most of my "important" servers have port 22 firewalled off an only open to a small set of external ip addresses. Some of them aren't running ssh at all.)

matt__rose 13y ago

OpenSSH 4.3 is likely the most popular because it is the version that comes with (redhat|centos|scientific|oracle) linux 5. It's still widely in use. 4.3 had a lot of bugs, but redhat has been backporting fixes to it since it came out.

justincormack 13y ago
I was wondering about correlating these to distros to get real world usage. Distinguishing redhat vs centos is not going to be easy though I guess.

virtualwhys 13y ago

I used to run SSH over some port != 22, does the trick to some degree. After I picked up a Cisco ASA, went back to standard port 22 but only allow access for connected VPN users. Of course if the ASA goes down, so does the entire network, yelp. SmartNET contract/warranty comes in handy, and the data center having backup ASAs on site for quick swap is pretty useful as well.

Qantourisc 13y ago
This doesn't make it secure, but this DOES save a lot on having a log full with login attempts from all kinds of IP addresses.
- tsahyt 13y ago
  When I set up my first server I was thoroughly surprised at the amount of login attempts. By now I've got most of Asia on a blacklist for port 22 (which doesn't matter, since I'm not going to connect to my server from an Asian IP in the foreseeable future). That throttled it down to about 30 attempts/week, which is manageable in terms of log reading.
- virtualwhys 13y ago
  Excuse me? How is VPN not secure? The ASA won't let any traffic through on port 22, period. You have to be connected over VPN in order to even attempt SSH access to servers behind the ASA.
justincormack 13y ago
Of course Cisco is at the top of the NSA insert backdoors list one assumes.

telephonetemp 13y ago

I run a VPS that only is only accessible over IPv6. I wonder if they'll ever scan it. Is there a way to narrow down the whole IPv6 search space to the most populated subranges?

justincormack 13y ago
Well you could get hold of the routing tables and filter out unallocated space. But thats not going to help much as there are still 64 low end bits to guess, some people use xxx::1 and ::2 but others use mac address based IPs (and random ones). So we are probably at the best point to portscan before the possibility goes away again...

rorrr2 13y ago

Maybe you should randomize the order of IPs you scan, so you don't hit the same network at 100K requests/sec.

gpvos 13y ago
Of course they do, they are not stupid. Did you read the article?

sebcat 13y ago

I'm on a phone with shitty wifi atm so looking at the code would be hard to say the least. However, I am curious, how do you deal with packet loss? The reason I'm asking is because, most people who claim to "scan the Internet" assume that the network is reliable. And they don't follow up on potential false negatives. If you scan the IPv4 address-space sequentially while only limiting bandwidth or time, rest assured that packets will be dropped.

swalsh 13y ago

Reading through the source code, there's actually a reasonably well used goto (https://github.com/robertdavidgraham/masscan/blob/master/src/main-throttle.c https://github.com/robertdavidgraham/masscan/blob/master/src...)

mabhatter 13y ago

If they are only hitting a port 22 a few times, who's wasting the ISP's time with abuse requests... I suppose its probably those guys with Class-A blocks and too much time on their hands.