6 ms·

You cannot hide anything on the internet anymore, the full IPv4 range is scanned regularly by multiple entities. If you open a port on a public IP it will get f

by gnyman 3y ago

You cannot hide anything on the internet anymore, the full IPv4 range is scanned regularly by multiple entities. If you open a port on a public IP it will get found. If it's a obscure non-standard port it might take longer, but if it's on any of the standard ports it will get probed very quickly and included tools like shodan.io The reason why I'm repeating this, is that not everyone knows this. People still (albeit less) put up elastic and mongodb instances with no authentication on public IP's. The second thing which isn't well known is the Certificate Transparency logs. This is the reason why you can't (without a wildcard cert) hide any HTTPS service. When you ask Let's Encrypt (or any CA actually) to generate veryobscure.domain.tld they will send that to the Certificate Transparency logs. You can find every certificate which was minted for a domain on a tool like https://crt.sh https://crt.sh There are many tools like subdomain.center, https://hackertarget.com/find-dns-host-records/ https://hackertarget.com/find-dns-host-records/ comes to mind. The most impressive one I've seen, which found more much more than expected, is Detectify (which is a paid service, no affiliation), they seem to combine the passive data collection (like subdomain.center) with active brute to find even more subdomains. But you can probably get 95% there by using CT and a brute-force tool like https://github.com/aboul3la/Sublist3r https://github.com/aboul3la/Sublist3r

tamimio 3y ago
> This is the reason why you can't (without a wildcard cert) Guess being security conscious pays off, as testing those on some domains I have, they only managed to show what I want to show, since wildcard will just mask them. That being said, I don’t think anyone should consider a subdomain as a hidden thing, it’s an address after all and should not be considered hidden, assume it’s accessible or put it behind a FW or VPN and have a proper authentication, security by obscurity never works.
fragmede 3y ago
not to underestimate the power of shodan, and oh god don't spin up a default mongo with no auth, but port knocking would seen to counteract this to enough of a degree, not to mention having a service only accessible via Tor. https://wiki.archlinux.org/title/Port_knocking#:~:text=Port%20knocking%20is%20a%20stealth,series%20of%20predefined%20closed%20ports https://wiki.archlinux.org/title/Port_knocking#:~:text=Port%....
- gnyman 3y ago
  Yes, you can hide with a little bit of effort. Port knocking or Tor will stop almost any thing (but don't rely on it as the sole protection, just as another layer). I like to prefix anything "I don't want scraped" with a random prefix, like domain.com/kwo4sx_grafana/ and nobody will find it (as long as you don't link to it anywhere). But I still have auth enabled, but at least I don't have to worry about any automated attacks exploiting it before I have time to patch. Something as simple as moving SSH on a non standard port reduces the amount of noise from most automated scanners 99% (made up number, but a lot).
  - Vorh 3y ago
    Have you had any problems with browsers leaking the prefixed sites, as seen here? https://news.ycombinator.com/item?id=35703789 https://news.ycombinator.com/item?id=35703789
implements 3y ago
Recently, I opened 80 and 443 so I could use LetsEncrypt’s acme-client to get a certificate (and then test it). Tightening up security a bit, I configured an http relay to filter people accessing 80 by ip address rather than domain name - some scanners are still trying domain and sub-domain names I was using weeks ago - which goes to show how organised hackers are about attacking targets.
- intothemild 3y ago
  Once you expose something long enough to get scanned. It's going to continue to get scanned pretty much forever. I self host a couple web services, but none are open, you need strong authentication to get in. It's not ideal, ideally I'd close the https web traffic and use some form of VPN to get in. But sadly that's just not feasible in my use case. So strong auth it is.
- BoberMod 3y ago
  You can use DNS-01 challenge [1] to get certificate. You just need to add temporary TXT record to your DNS. It also supports wildcart certificates. Most popular DNS providers (like Cloudflare) has API, so it can be easily automated. I'm using it in my local network: I have publicly available domain for it (intranet.domain.com) and I don't wont to expose my local services to the world to issue certificate trusted by root CA on all my devices. So, this method allows me to issue valid Let's encrypt wildcard cert (*.intranet.domain.com) for all my internal services without opening any ports to the world. [1]: https://letsencrypt.org/docs/challenge-types/#dns-01-challenge https://letsencrypt.org/docs/challenge-types/#dns-01-challen...
sgjohnson 3y ago
You don't even need "multiple entities". Absolutely anyone can do that. Scanning a single port on the entire IPv4 internet takes about 40 minutes.
fuzzy2 3y ago
The Certificate Transparency Log is very important. I recently spun up a service with HTTPS certs by Let's Encrypt. By coincidence I was watching the logs. Within just 80 seconds of the certificate being issued I could see the first automated "attacks". If you get a certificate, be ready for the consequences.
- tikkabhuna 3y ago
  This is really interesting. For my Homelab I've been playing around with using Lets Encrypt rather than spinning up my own CA. "What's the worst that could happen?" Guess I'll be looking to spin up my own CA now!
  - teekert 3y ago
    Can Tailscale magic DNS + tunnel obscure things? Or only when you keep a service within the tailnet? (Still a + for selfhosters)
  - KronisLV 3y ago
    > Guess I'll be looking to spin up my own CA now! I was looking for a lazy/easy way to do this manually and settled on KeyStore Explorer, which is a GUI tool that lets you work with various keystores and do everything from making your own CA, to signing and exporting certificates in various formats: https://github.com/kaikramer/keystore-explorer https://github.com/kaikramer/keystore-explorer (to me it feels easier than working with OpenSSL directly, provided I trust the tool) In addition, I also setup mTLS or even basicauth at the web server (reverse proxy) level for some of my sites, which seems to help that little bit more, given that some automated attacks might choose to ignore TLS errors, but won't be able to provide my client certs or the username/password. In addition, I also run fail2ban and mod_security, though that's more opinionated.
  - ahoka 3y ago
    Didn’t you read the original comment? It’s just a matter of time until someone starts to poke your IPs. Your own CA will be harder to get right.
  - 3y ago
deleted 3y ago
[deleted]
pid-1 3y ago
> full IPv4 range is scanned regularly by multiple entities. Yet another good reason to use IPv6
- gnyman 3y ago
  IPv6 won't get found by brute-force but there are a few projects which tries to gather IPv6 addresses using various means and scans them as they are found. Shodan did (maybe still does) provided NTP servers to some ntp-pools and scanned anyone who sent incoming requests. https://arstechnica.com/information-technology/2016/02/using-ipv6-with-linux-youve-likely-been-visited-by-shodan-and-other-scanners/ https://arstechnica.com/information-technology/2016/02/using... So as with everything, layer the defences, don't rely on you IPv6 address being secret as the only defence.
danielvaughn 3y ago
At my company we got bit by this several months ago. Luckily the database was either empty or only had testing data, but like you said the port was exposed and someone found it.
codethief 3y ago
> You cannot hide anything on the internet anymore, the full IPv4 range is scanned regularly by multiple entities. If you open a port on a public IP it will get found. Sure but you might still host multiple virtual hosts (e.g. subdomains) on the same web server. Unless an attacker knows their exact hostnames, they won't be able to access them.
- gustavus 3y ago
  There are several easy ways to the skirt that. First you can simply try bruteforcing subdomains, secondly if you are using https you can simply pull the cert and look at the aliases listed there. 2 ways off the top of my head.
  - codethief 3y ago
    Of course, but my point was that none of them involve IP scanning.
matheusmoreira 3y ago
> the full IPv4 range is scanned regularly by multiple entities Single packet authorization. Server just drops any and all packets unless you send a cryptographically signed packet to it first. To all these observers, it's like the server is not even there.

by gnyman 3y ago

tamimio 3y ago

> This is the reason why you can't (without a wildcard cert) Guess being security conscious pays off, as testing those on some domains I have, they only managed to show what I want to show, since wildcard will just mask them. That being said, I don’t think anyone should consider a subdomain as a hidden thing, it’s an address after all and should not be considered hidden, assume it’s accessible or put it behind a FW or VPN and have a proper authentication, security by obscurity never works.

fragmede 3y ago

not to underestimate the power of shodan, and oh god don't spin up a default mongo with no auth, but port knocking would seen to counteract this to enough of a degree, not to mention having a service only accessible via Tor. https://wiki.archlinux.org/title/Port_knocking#:~:text=Port%20knocking%20is%20a%20stealth,series%20of%20predefined%20closed%20ports https://wiki.archlinux.org/title/Port_knocking#:~:text=Port%....

gnyman 3y ago
Yes, you can hide with a little bit of effort. Port knocking or Tor will stop almost any thing (but don't rely on it as the sole protection, just as another layer). I like to prefix anything "I don't want scraped" with a random prefix, like domain.com/kwo4sx_grafana/ and nobody will find it (as long as you don't link to it anywhere). But I still have auth enabled, but at least I don't have to worry about any automated attacks exploiting it before I have time to patch. Something as simple as moving SSH on a non standard port reduces the amount of noise from most automated scanners 99% (made up number, but a lot).
- Vorh 3y ago
  Have you had any problems with browsers leaking the prefixed sites, as seen here? https://news.ycombinator.com/item?id=35703789 https://news.ycombinator.com/item?id=35703789

implements 3y ago

Recently, I opened 80 and 443 so I could use LetsEncrypt’s acme-client to get a certificate (and then test it). Tightening up security a bit, I configured an http relay to filter people accessing 80 by ip address rather than domain name - some scanners are still trying domain and sub-domain names I was using weeks ago - which goes to show how organised hackers are about attacking targets.

intothemild 3y ago
Once you expose something long enough to get scanned. It's going to continue to get scanned pretty much forever. I self host a couple web services, but none are open, you need strong authentication to get in. It's not ideal, ideally I'd close the https web traffic and use some form of VPN to get in. But sadly that's just not feasible in my use case. So strong auth it is.
BoberMod 3y ago
You can use DNS-01 challenge [1] to get certificate. You just need to add temporary TXT record to your DNS. It also supports wildcart certificates. Most popular DNS providers (like Cloudflare) has API, so it can be easily automated. I'm using it in my local network: I have publicly available domain for it (intranet.domain.com) and I don't wont to expose my local services to the world to issue certificate trusted by root CA on all my devices. So, this method allows me to issue valid Let's encrypt wildcard cert (*.intranet.domain.com) for all my internal services without opening any ports to the world. [1]: https://letsencrypt.org/docs/challenge-types/#dns-01-challenge https://letsencrypt.org/docs/challenge-types/#dns-01-challen...

sgjohnson 3y ago

You don't even need "multiple entities". Absolutely anyone can do that. Scanning a single port on the entire IPv4 internet takes about 40 minutes.

fuzzy2 3y ago

The Certificate Transparency Log is very important. I recently spun up a service with HTTPS certs by Let's Encrypt. By coincidence I was watching the logs. Within just 80 seconds of the certificate being issued I could see the first automated "attacks". If you get a certificate, be ready for the consequences.

tikkabhuna 3y ago
This is really interesting. For my Homelab I've been playing around with using Lets Encrypt rather than spinning up my own CA. "What's the worst that could happen?" Guess I'll be looking to spin up my own CA now!
- teekert 3y ago
  Can Tailscale magic DNS + tunnel obscure things? Or only when you keep a service within the tailnet? (Still a + for selfhosters)
- KronisLV 3y ago
  > Guess I'll be looking to spin up my own CA now! I was looking for a lazy/easy way to do this manually and settled on KeyStore Explorer, which is a GUI tool that lets you work with various keystores and do everything from making your own CA, to signing and exporting certificates in various formats: https://github.com/kaikramer/keystore-explorer https://github.com/kaikramer/keystore-explorer (to me it feels easier than working with OpenSSL directly, provided I trust the tool) In addition, I also setup mTLS or even basicauth at the web server (reverse proxy) level for some of my sites, which seems to help that little bit more, given that some automated attacks might choose to ignore TLS errors, but won't be able to provide my client certs or the username/password. In addition, I also run fail2ban and mod_security, though that's more opinionated.
- ahoka 3y ago
  Didn’t you read the original comment? It’s just a matter of time until someone starts to poke your IPs. Your own CA will be harder to get right.
- 3y ago

deleted 3y ago

[deleted]

pid-1 3y ago

> full IPv4 range is scanned regularly by multiple entities. Yet another good reason to use IPv6

gnyman 3y ago
IPv6 won't get found by brute-force but there are a few projects which tries to gather IPv6 addresses using various means and scans them as they are found. Shodan did (maybe still does) provided NTP servers to some ntp-pools and scanned anyone who sent incoming requests. https://arstechnica.com/information-technology/2016/02/using-ipv6-with-linux-youve-likely-been-visited-by-shodan-and-other-scanners/ https://arstechnica.com/information-technology/2016/02/using... So as with everything, layer the defences, don't rely on you IPv6 address being secret as the only defence.

danielvaughn 3y ago

At my company we got bit by this several months ago. Luckily the database was either empty or only had testing data, but like you said the port was exposed and someone found it.

codethief 3y ago

> You cannot hide anything on the internet anymore, the full IPv4 range is scanned regularly by multiple entities. If you open a port on a public IP it will get found. Sure but you might still host multiple virtual hosts (e.g. subdomains) on the same web server. Unless an attacker knows their exact hostnames, they won't be able to access them.

gustavus 3y ago
There are several easy ways to the skirt that. First you can simply try bruteforcing subdomains, secondly if you are using https you can simply pull the cert and look at the aliases listed there. 2 ways off the top of my head.
- codethief 3y ago
  Of course, but my point was that none of them involve IP scanning.

matheusmoreira 3y ago

> the full IPv4 range is scanned regularly by multiple entities Single packet authorization. Server just drops any and all packets unless you send a cryptographically signed packet to it first. To all these observers, it's like the server is not even there.