10 ms·
Certificate Transparency logs, or they don't actually know the domain name: just port-scanning[1] then making requests to open web ports. [1] Turns out you can
by parliament32 2y ago
Certificate Transparency logs, or they don't actually know the domain name: just port-scanning[1] then making requests to open web ports.
[1] Turns out you can port-scan the entire internet in under 5 minutes: https://github.com/robertdavidgraham/masscan https://github.com/robertdavidgraham/masscan
- andix 2y agoPort scanning usually can't discover subdomains. Most servers don't expose the of the domains they server content for. In case of HTTP they usually only serve the subdomain content if the Host: request-header includes it.
- hombre_fatal 2y agoMost servers just listen on :80 and respond to all requests. Almost nobody checks the host header intentionally, it's just a happy mistake if they use a reverse proxy. You can often decloak servers behind Cloudflare because of this. But OP's post already answered their question: someone scanned ipv4 space. And what they mean is that a server they point to via DNS is receiving requests, but DNS is a red herring.
- andix 2y agoThis really depends on the setup. Most web servers host multiple virtual hosts. IP addresses are expensive. If you're deploying a service behind a reverse proxy, it either must be only accessible from the reverse proxy via an internal network, or check the IP address of the reverse proxy. It absolutely must not trust X-Forwarded-For: headers from random IPs.
- hombre_fatal 2y agoI just don't see how any of this matters. OP's server is reachable via ipv4 and someone sent an http request to it. Their post even says that this is the case.
- andix 2y agoI'm guessing they meant it discovered a virtual host behind a subdomain.
- benfortuna 2y agoI could be wrong, but the Palo Alto scanner says it's using global ipv4 space, so not using DNS at all. So actually the subdomain has not been discovered at all.
- reactordev 2y agoThis is exactly what’s happening based on the log snippet posted. Has nothing to do with subdomains, has everything to do with it being on the internet.
- parliament32 2y agoHow deep in the domain hierarchy you are doesn't matter from a network layer: a bare tld (yes this exists), a normal domain, a subdomain, a sub-subdomain, etc can all be assigned different IPs and go different places. You can issue a GET against / for any IP you want (like we see in the logs OP posted). The only time this would actually matter is if a host at an address is serving content for multiple hostnames and depends on the Host header to figure out which one to serve -- but even those will almost always have a default.
- andix 2y agoYou can discover IP adresses, sure. Just enumerate them. But this doesn't give you the domain, as long as there is no reverse dns record. I'm quite sure OP meant a virtual host only reachable with the correct Host: header.
- cryptonector 2y agoAnd in the case of HTTPS they need to insist on SNI (and TLSv3 requires it).
- giancarlostoro 2y agoLast few times I tried to do this my ISP cut off my internet every time. Assholes. It comes back, but they're still assholes for it.
- icehawk 2y agoThis. I have a DNS client that feeds into my passive DNS database by reading CT logs and then trying to resolve them.
- toomuchtodo 2y agoWhat do you use it for?