19 ms·

Subdomain.center – discover all subdomains for a domain

385 points by adam_gyroscope 3y ago

Xorakios 3y ago
Just for giggles, does anyone else remember when "subdomains" were called "machine names" because physical devices were limited to one service? www. ftp. mail. ... weren't theoretical or merely mnemonic. Felt like an old coot when using "machine name" to a 40 year old IT professional and she was perplexed!
- semi 3y ago
  I'm a somewhat old coot and do remember those days, but I think the term still makes sense but only in a lan environment. machines still have hostnames, and home routers will often trust your dhcp clients machine name. So I can still look up steamdeck.lan and find the IP of my steam deck and in that context calling it a machine name is perfectly apt and I think would still be well understood.
judge2020 3y ago
https://dnsdumpster.com https://dnsdumpster.com
hankchinaski 3y ago
I would be keen to know what techniques are used. Usually subdomain discovery is done with dns axfr transfer request which leaks the entire dns zone (but this only works on ancient and unpatched nameservers) or with dictionary attacks. There are some other techniques you can check if you look at the source code of amass (open source Golang reconnaissance/security tool), or CT logs. Dns dumpster is one of the tools I used alongside pentest tools (commercial) and amass (oss)
- cobertos 3y ago
  I mean, doesn't it say right on the front page? * Apache Nutch - So they're crawling either some part of the root itself or some other websites to find subdomains. Honestly might help to query CommonCrawl too. * Calidog's Certstream - As you said, you can look at the CT logs * OpenAI Embeddings - So I guess it also uses LLM to try to generate ones to test too. * Proprietary Tools - your guess is as good as mine Probably a common list of subdomains to test against too. Seems like multiple techniques to try to squeeze out as much info as possible.
  - Zuiii 3y ago
    I'd also add insecure DNSSEC implementations that allow you to "walk" the entire record chain for the domain.
    - kevincox 3y ago
      Calling this "insecure" is a bit harsh. This is required for offline signing which provides better security but worse privacy.
      - Zuiii 3y ago
        Sorry, you're right. I mistakenly thought this issue was solved by a later standard.
        imrejonk 3y ago
        Could that later standard be NSEC3? It’s like the easily walkable NSEC, but with hashed names and special flags for opting out of delegation security features. The 3 appears to stand for the number of people that fully understand how it works…
  - piffey 3y ago
    Proprietary tools means passive DNS.
    - smarx007 3y ago
      How can one avoid their browsing ending up in the passive DNS logs? For example, is using 1.1.1.1, 8.8.8.8, or 9.9.9.9 (CF, Google, and Quad9, respectively) good or bad in this regard? For example, where does Spamhaus get their passive DNS data? They write [1] that it comes from "trusted third parties, including hosting companies, enterprises, and ISPs." But that's rather vague. Are CF, Google, and Quad9 some of those "hosting companies, enterprises, and ISPs"? [1]: https://www.spamhaus.com/resource-center/what-is-passive-dns-a-beginners-guide/ https://www.spamhaus.com/resource-center/what-is-passive-dns...
RockRobotRock 3y ago
This is certificate transparency doing most of the work, right?
- out-of-ideas 3y ago
  it may utilize a few techniques as there are subdomains I am aware of that've never been published other than in the zone config on my registrar that are returned from api query
  - pbhjpbhj 3y ago
    I use Siteground and it has a staging server that AFAIK hasn't been used for at least 6 years ... Nothing at the host has any details of that, archive.org doesn't have it in their site URLs, it's not in DNS records, not in .well-known, it was a transient test years ago ... really curious, must be historic data from somewhere?
  - RockRobotRock 3y ago
    I use Cloudflare for DNS and the only ones it found had LE certs. It's not doing a simple brute-force on common names, I don't think. Otherwise it probably would have found a lot more. Curious about how it works.
- zootboy 3y ago
  I would assume so. I tested on one of my private domains that generally isn't linked to anywhere, and it just returned the few domains that I generate Let's Encrypt certs for, plus my nameservers. Interestingly, I did not receive any DNS queries on my authoritative nameservers during the query, so they don't seem to be doing any active DNS probes.
derefr 3y ago
Interesting. Our domain has some subdomains with a numeric suffix; and the API response here has entries in that pattern for not only the particular subdomains that exist or ever existed, but also for subdomains of the same pattern that go beyond any suffix number we've ever actually used. You'd think they'd at least be filtering their response by checking which subdomains actually have an A/AAAA/CNAME record on them...
- cm2187 3y ago
  In fact my suspicion is that they would also cheat by looking into the dns cache of the current machine.
- ornornor 3y ago
  Maybe that’s the Open AI part of their secret sauce hallucinating subdomains?
johntiger1 3y ago
Took a while, but was impressed it detected all of ours: https://api.subdomain.center/?domain=radiantai.health https://api.subdomain.center/?domain=radiantai.health
- DistractionRect 3y ago
  Certificate transparency does a lot of the heavy lifting: https://crt.sh/?q=radiantai.health https://crt.sh/?q=radiantai.health
  - Semaphor 3y ago
    Only that actually works. I get hundreds of entries for my domain there, including entries before Lets Encrypt was a thing, while the subdomain checker returns an empty array.
Ocha 3y ago
Missed some for me
- Ocha 3y ago
  Maybe because I use wild card certs with let’s encrypt
  - ThePowerOfFuet 3y ago
    Instead of replying to yourself, try editing your first comment!
perryizgr8 3y ago
It detects only some of mine. To be precise, it does not detect subdomains being served by a service behind a CloudFlare tunnel.
internet2000 3y ago
For my personal domain: it got the ones I have on the SSL cert alternative subject names, made up three, returned one I deleted more than a year ago, and didn't find two. Very curious.
- DaiPlusPlus 3y ago
  Those SAN and CN names will appear in publicly visible certificate transparency lists ( https://en.wikipedia.org/wiki/Certificate_Transparency https://en.wikipedia.org/wiki/Certificate_Transparency ): so if you ever get a TLS certificate for a super-seeekret internal sub-sub-sub-domain-name from a major CA then it won't be secret for long. The only way to keep a publicly-resolvable DNS subdomain confidential is to either get a wildcard cert for the parent domain or find a dodgy (yet somehow widely-trusted) CA that doesn't particiate in CT - or use a self-signed cert. This subdomain.center database returned one of my "private" sub-sub-domains (which just points to my NAS) for which I did get a cert from LetsEncrypt, but it doesn't have any of my other sub-sub domains listed (despite resolving to the same A IPv4 address as the listed subdomain) because those subdomains have only ever been secured by a wildcard cert.
tobinfekkes 3y ago
This is crazy, I was just looking for this exact thing a couple days ago. Thank you for sharing. Brilliant work.
banana_giraffe 3y ago
Cute, it managed to find 121486 subdomains for amazonaws.com [1], and somehow I suspect that's a tiny fraction of what's in use. https://gist.githubusercontent.com/Q726kbXuN/bf8a9a22b81fe651e38f67548aeb7f3f/raw/e47a312c7a9dc7a9ce41ca1c522a687479a0cf4c/amazonaws.json https://gist.githubusercontent.com/Q726kbXuN/bf8a9a22b81fe65...
cm2187 3y ago
One thing I noticed looking at my logs is that there is almost no unsolicited traffic (i.e. failed authentication attempts, exploits of various worldpress bugs, etc) through ipv6. I think it's a function of 1) those coming from networks (compromised home devices, etc) that don't support v6, 2) the v6 address space being too large to scan (the size of an encryption key), so good security by obscurity. This would nullify 2).
zX41ZdbW 3y ago
How can I download the entire dataset from this service?
franky47 3y ago
Sublist3r [1] does a similar job, as long as you have the authorisation to use it on a particular domain, as it uses more aggressive discovery techniques. [1] https://github.com/aboul3la/Sublist3r https://github.com/aboul3la/Sublist3r
webprofusion 3y ago
This is a CT log search right?
keepamovin 3y ago
This is fantastic!!! What kind of security considerations are there to having multi-tenant user applications on subdomains and then having them exposed like this? I'm building a SaaS right now, and I guess one thing is that a given username can then be discovered as a valid login for the system...but obviously that's only part of the login credential. Maintaining a list of mappings to opaque subdomains seems to reduce targeting, and conceal login partial credentials, but doesn't seem to offer much besides. Analysis?
- thorum 3y ago
  It doesn’t seem to detect subdomains set up with Kubernetes ingresses, based on results for one of my domains, so that might be a place to start research.
  - davidkuennen 3y ago
    It also doesn't find any subdomains for my domain. In my case I use Google Cloud DNS. Maybe they have some sort of protection in place (I wouldn't be surprised).
pabs3 3y ago
More options here: https://wiki.archiveteam.org/index.php/Finding_subdomains https://wiki.archiveteam.org/index.php/Finding_subdomains
- sea-gold 3y ago
  Thanks. This is a really helpful list which includes many of the sites/tools listed here.
hbcondo714 3y ago
Seems similar yet still useful to Wolfram Alpha; just enter a domain and click on the "Subdomains" button: https://www.wolframalpha.com/input?i=ycombinator.com https://www.wolframalpha.com/input?i=ycombinator.com
Brananarchy 3y ago
As others have said, certificate transparency seems to be doing some heavy lifting here. It reports subdomains for me that have never had a public CNAME or A record, but have had let's encrypt certs issued for internal use. It's also missing some that have not had certs issued, but that are in public DNS
- TekMol 3y ago
  That's why HTTPS is still a pain in the butt. 30 years after it was invented. I don't want internally used subdomains to be public. Because of certificate transparency, the only way to achieve that is via wildcard certs. Let's encrypt only supports cumbersome validation methods for those. Like changing DNS records every time you need to renew the cert. Pretty annoying.
  - proto_lambda 3y ago
    If the subdomains aren't supposed to be public, the public also doesn't need to trust the TLS certs. Sign them with your own CA and trust it on the devices that should be able to access the domains.
    - paranoidrobot 3y ago
      Adding CAs to trust stores on devices and in apps is a major pain. If you have unmanaged devices this becomes even more painful. "Oh, hi, welcome to the company, please install this Root CA onto your machine to access <internal service>" Because you can't scope CAs to specific domains, this causes everyone with any idea about security to start being concerned.
      - eastbound 3y ago
        Every single company does it. The 3 of them: Asking employees to install a CA, using it for “.internal” resources, then ask employees to use a web proxy and MITM their connections. And optionally, leak the CA’s pk to get pawned. It’s the standard operating procedure of any well-run business.
weird-eye-issue 3y ago
I got back an empty list for my domain on Cloudflare with several subdomains (non wildcard) edit: I retried on my computer (was on my phone earlier) and now it returns all of our subdomains, even picking up our test R2 bucket. In guessing I was rate limited because I accidentally loaded the example file a few times
TechBro8615 3y ago
I get a rate limit error when I click the text input (I'm on a VPN).
- 867-5309 3y ago
  use an obscure country like North Macedonia
mmarquezs 3y ago
Nice, last time I used Wolframalpha for this.
ohuf 3y ago
The subdomain explorer may be fun, but their Exploit Observer is really useful: https://www.exploit.observer/ https://www.exploit.observer/
- g147 3y ago
  thanks!
_cenw 3y ago
https://github.com/projectdiscovery/subfinder https://github.com/projectdiscovery/subfinder does this, but it explains all the methods and lets you choose to only do a passive scan.
gnyman 3y ago
You cannot hide anything on the internet anymore, the full IPv4 range is scanned regularly by multiple entities. If you open a port on a public IP it will get found. If it's a obscure non-standard port it might take longer, but if it's on any of the standard ports it will get probed very quickly and included tools like shodan.io The reason why I'm repeating this, is that not everyone knows this. People still (albeit less) put up elastic and mongodb instances with no authentication on public IP's. The second thing which isn't well known is the Certificate Transparency logs. This is the reason why you can't (without a wildcard cert) hide any HTTPS service. When you ask Let's Encrypt (or any CA actually) to generate veryobscure.domain.tld they will send that to the Certificate Transparency logs. You can find every certificate which was minted for a domain on a tool like https://crt.sh https://crt.sh There are many tools like subdomain.center, https://hackertarget.com/find-dns-host-records/ https://hackertarget.com/find-dns-host-records/ comes to mind. The most impressive one I've seen, which found more much more than expected, is Detectify (which is a paid service, no affiliation), they seem to combine the passive data collection (like subdomain.center) with active brute to find even more subdomains. But you can probably get 95% there by using CT and a brute-force tool like https://github.com/aboul3la/Sublist3r https://github.com/aboul3la/Sublist3r
- tamimio 3y ago
  > This is the reason why you can't (without a wildcard cert) Guess being security conscious pays off, as testing those on some domains I have, they only managed to show what I want to show, since wildcard will just mask them. That being said, I don’t think anyone should consider a subdomain as a hidden thing, it’s an address after all and should not be considered hidden, assume it’s accessible or put it behind a FW or VPN and have a proper authentication, security by obscurity never works.
- fragmede 3y ago
  not to underestimate the power of shodan, and oh god don't spin up a default mongo with no auth, but port knocking would seen to counteract this to enough of a degree, not to mention having a service only accessible via Tor. https://wiki.archlinux.org/title/Port_knocking#:~:text=Port%20knocking%20is%20a%20stealth,series%20of%20predefined%20closed%20ports https://wiki.archlinux.org/title/Port_knocking#:~:text=Port%....
yadnst 3y ago
[dead]
p4bl0 3y ago
It gave me empty results for some of my domains that have multiple subdomains that have TLS certificate associated with them so that must appear in the certificate transparency log. I guess it should be "discover some subdomains for some domains".
- Semaphor 3y ago
  Empty for all my and my work’s domains. Then I tested random .com domains and got results. Seems pretty useless.
donatj 3y ago
Interesting. It only found less that a quarter of the subdomains of the site I work on, and everything it did find is public facing. I wonder if that’s maybe something to do with how we set up certificates for public vs internal subdomains? It even missed “staging.” which should be nearly identical in configuration to www
chillbill 3y ago
[dead]
blueflow 3y ago
I entered my own domains and i got so many garbage entries. It feels like an AI reading letsencrypt logs and then adding made up shit to it.
SushiHippie 3y ago
Note, if you looked up a domain and it had no results, you should check back again after some minutes. I looked my domain up and had zero results, which was weird as it should at least find some in the ct logs, but a few minutes later it showed some subdomains.
- LinuxBender 3y ago
  It took about 5 minutes for me. It found my apex domain and a sub-domain that must have belonged to the previous renter of my domain name. [1] So I was curious and it turns out the previous renters pages were in Wayback. [2] That page renders as mostly little boxes for me. Funny, I had never bothered to check that. I should check if any of my other domains have snapshots from before I rented them. [1] - https://api.subdomain.center/?domain=ohblog.net https://api.subdomain.center/?domain=ohblog.net [2] - https://web.archive.org/web/20090302094112/https://ohblog.net/ https://web.archive.org/web/20090302094112/https://ohblog.ne...
  - SushiHippie 3y ago
    Web archive can also somewhat act as a subdomain finder (not really in this case, only the www subdomain, but still interesting): https://web.archive.org/web/*/ohblog.net https://web.archive.org/web/*/ohblog.net*
TheHappyOddish 3y ago
Hardly "all subdomains". Unless it's doing an AXFR of my zone file (unlikely), this isn't possible. It's a scraper/guesser, using cert transparency, common names, etc. Cute toy, but false claims.
- panki27 3y ago
  You are correct, I've tested it with my own domains. It does not know the ones running with a wildcard certificate for example.
  - wlonkly 3y ago
    It knows many of the wildcard-served customer subdomains of one of my former employers. (They're probably just scraped from search or something, but a wildcard is not sufficient to prevent discovery.)
Andrew018 3y ago
[dead]
xg15 3y ago
I think as soon as cert transparency was introduced, it was pretty clear we would eventually get something like this.
deleted 3y ago
[deleted]
Arubis 3y ago
If this were able to determine which wildcard subdomains were active for a given domain, you could use it to figure out a lot of B2B companies’ client/customer list.
maul666 3y ago
dpd.co.uk

Just for giggles, does anyone else remember when "subdomains" were called "machine names" because physical devices were limited to one service? www. ftp. mail. ... weren't theoretical or merely mnemonic. Felt like an old coot when using "machine name" to a 40 year old IT professional and she was perplexed!

semi 3y ago
I'm a somewhat old coot and do remember those days, but I think the term still makes sense but only in a lan environment. machines still have hostnames, and home routers will often trust your dhcp clients machine name. So I can still look up steamdeck.lan and find the IP of my steam deck and in that context calling it a machine name is perfectly apt and I think would still be well understood.

judge2020 3y ago

https://dnsdumpster.com https://dnsdumpster.com

hankchinaski 3y ago

I would be keen to know what techniques are used. Usually subdomain discovery is done with dns axfr transfer request which leaks the entire dns zone (but this only works on ancient and unpatched nameservers) or with dictionary attacks. There are some other techniques you can check if you look at the source code of amass (open source Golang reconnaissance/security tool), or CT logs. Dns dumpster is one of the tools I used alongside pentest tools (commercial) and amass (oss)

cobertos 3y ago
I mean, doesn't it say right on the front page? * Apache Nutch - So they're crawling either some part of the root itself or some other websites to find subdomains. Honestly might help to query CommonCrawl too. * Calidog's Certstream - As you said, you can look at the CT logs * OpenAI Embeddings - So I guess it also uses LLM to try to generate ones to test too. * Proprietary Tools - your guess is as good as mine Probably a common list of subdomains to test against too. Seems like multiple techniques to try to squeeze out as much info as possible.
- Zuiii 3y ago
  I'd also add insecure DNSSEC implementations that allow you to "walk" the entire record chain for the domain.
  - kevincox 3y ago
    Calling this "insecure" is a bit harsh. This is required for offline signing which provides better security but worse privacy.
    - Zuiii 3y ago
      Sorry, you're right. I mistakenly thought this issue was solved by a later standard.
      - imrejonk 3y ago
        Could that later standard be NSEC3? It’s like the easily walkable NSEC, but with hashed names and special flags for opting out of delegation security features. The 3 appears to stand for the number of people that fully understand how it works…
- piffey 3y ago
  Proprietary tools means passive DNS.
  - smarx007 3y ago
    How can one avoid their browsing ending up in the passive DNS logs? For example, is using 1.1.1.1, 8.8.8.8, or 9.9.9.9 (CF, Google, and Quad9, respectively) good or bad in this regard? For example, where does Spamhaus get their passive DNS data? They write [1] that it comes from "trusted third parties, including hosting companies, enterprises, and ISPs." But that's rather vague. Are CF, Google, and Quad9 some of those "hosting companies, enterprises, and ISPs"? [1]: https://www.spamhaus.com/resource-center/what-is-passive-dns-a-beginners-guide/ https://www.spamhaus.com/resource-center/what-is-passive-dns...

RockRobotRock 3y ago

This is certificate transparency doing most of the work, right?

out-of-ideas 3y ago
it may utilize a few techniques as there are subdomains I am aware of that've never been published other than in the zone config on my registrar that are returned from api query
- pbhjpbhj 3y ago
  I use Siteground and it has a staging server that AFAIK hasn't been used for at least 6 years ... Nothing at the host has any details of that, archive.org doesn't have it in their site URLs, it's not in DNS records, not in .well-known, it was a transient test years ago ... really curious, must be historic data from somewhere?
- RockRobotRock 3y ago
  I use Cloudflare for DNS and the only ones it found had LE certs. It's not doing a simple brute-force on common names, I don't think. Otherwise it probably would have found a lot more. Curious about how it works.
zootboy 3y ago
I would assume so. I tested on one of my private domains that generally isn't linked to anywhere, and it just returned the few domains that I generate Let's Encrypt certs for, plus my nameservers. Interestingly, I did not receive any DNS queries on my authoritative nameservers during the query, so they don't seem to be doing any active DNS probes.

derefr 3y ago

Interesting. Our domain has some subdomains with a numeric suffix; and the API response here has entries in that pattern for not only the particular subdomains that exist or ever existed, but also for subdomains of the same pattern that go beyond any suffix number we've ever actually used. You'd think they'd at least be filtering their response by checking which subdomains actually have an A/AAAA/CNAME record on them...

cm2187 3y ago
In fact my suspicion is that they would also cheat by looking into the dns cache of the current machine.
ornornor 3y ago
Maybe that’s the Open AI part of their secret sauce hallucinating subdomains?

johntiger1 3y ago

Took a while, but was impressed it detected all of ours: https://api.subdomain.center/?domain=radiantai.health https://api.subdomain.center/?domain=radiantai.health

DistractionRect 3y ago
Certificate transparency does a lot of the heavy lifting: https://crt.sh/?q=radiantai.health https://crt.sh/?q=radiantai.health
- Semaphor 3y ago
  Only that actually works. I get hundreds of entries for my domain there, including entries before Lets Encrypt was a thing, while the subdomain checker returns an empty array.

Ocha 3y ago

Missed some for me

Ocha 3y ago
Maybe because I use wild card certs with let’s encrypt
- ThePowerOfFuet 3y ago
  Instead of replying to yourself, try editing your first comment!

perryizgr8 3y ago

It detects only some of mine. To be precise, it does not detect subdomains being served by a service behind a CloudFlare tunnel.

internet2000 3y ago

For my personal domain: it got the ones I have on the SSL cert alternative subject names, made up three, returned one I deleted more than a year ago, and didn't find two. Very curious.

DaiPlusPlus 3y ago
Those SAN and CN names will appear in publicly visible certificate transparency lists ( https://en.wikipedia.org/wiki/Certificate_Transparency https://en.wikipedia.org/wiki/Certificate_Transparency ): so if you ever get a TLS certificate for a super-seeekret internal sub-sub-sub-domain-name from a major CA then it won't be secret for long. The only way to keep a publicly-resolvable DNS subdomain confidential is to either get a wildcard cert for the parent domain or find a dodgy (yet somehow widely-trusted) CA that doesn't particiate in CT - or use a self-signed cert. This subdomain.center database returned one of my "private" sub-sub-domains (which just points to my NAS) for which I did get a cert from LetsEncrypt, but it doesn't have any of my other sub-sub domains listed (despite resolving to the same A IPv4 address as the listed subdomain) because those subdomains have only ever been secured by a wildcard cert.

tobinfekkes 3y ago

This is crazy, I was just looking for this exact thing a couple days ago. Thank you for sharing. Brilliant work.

banana_giraffe 3y ago

Cute, it managed to find 121486 subdomains for amazonaws.com [1], and somehow I suspect that's a tiny fraction of what's in use. https://gist.githubusercontent.com/Q726kbXuN/bf8a9a22b81fe651e38f67548aeb7f3f/raw/e47a312c7a9dc7a9ce41ca1c522a687479a0cf4c/amazonaws.json https://gist.githubusercontent.com/Q726kbXuN/bf8a9a22b81fe65...

cm2187 3y ago

One thing I noticed looking at my logs is that there is almost no unsolicited traffic (i.e. failed authentication attempts, exploits of various worldpress bugs, etc) through ipv6. I think it's a function of 1) those coming from networks (compromised home devices, etc) that don't support v6, 2) the v6 address space being too large to scan (the size of an encryption key), so good security by obscurity. This would nullify 2).

zX41ZdbW 3y ago

How can I download the entire dataset from this service?

franky47 3y ago

Sublist3r [1] does a similar job, as long as you have the authorisation to use it on a particular domain, as it uses more aggressive discovery techniques. [1] https://github.com/aboul3la/Sublist3r https://github.com/aboul3la/Sublist3r

webprofusion 3y ago

This is a CT log search right?

keepamovin 3y ago

This is fantastic!!! What kind of security considerations are there to having multi-tenant user applications on subdomains and then having them exposed like this? I'm building a SaaS right now, and I guess one thing is that a given username can then be discovered as a valid login for the system...but obviously that's only part of the login credential. Maintaining a list of mappings to opaque subdomains seems to reduce targeting, and conceal login partial credentials, but doesn't seem to offer much besides. Analysis?

thorum 3y ago
It doesn’t seem to detect subdomains set up with Kubernetes ingresses, based on results for one of my domains, so that might be a place to start research.
- davidkuennen 3y ago
  It also doesn't find any subdomains for my domain. In my case I use Google Cloud DNS. Maybe they have some sort of protection in place (I wouldn't be surprised).

pabs3 3y ago

More options here: https://wiki.archiveteam.org/index.php/Finding_subdomains https://wiki.archiveteam.org/index.php/Finding_subdomains

sea-gold 3y ago
Thanks. This is a really helpful list which includes many of the sites/tools listed here.

hbcondo714 3y ago

Seems similar yet still useful to Wolfram Alpha; just enter a domain and click on the "Subdomains" button: https://www.wolframalpha.com/input?i=ycombinator.com https://www.wolframalpha.com/input?i=ycombinator.com

Brananarchy 3y ago

As others have said, certificate transparency seems to be doing some heavy lifting here. It reports subdomains for me that have never had a public CNAME or A record, but have had let's encrypt certs issued for internal use. It's also missing some that have not had certs issued, but that are in public DNS

TekMol 3y ago
That's why HTTPS is still a pain in the butt. 30 years after it was invented. I don't want internally used subdomains to be public. Because of certificate transparency, the only way to achieve that is via wildcard certs. Let's encrypt only supports cumbersome validation methods for those. Like changing DNS records every time you need to renew the cert. Pretty annoying.
- proto_lambda 3y ago
  If the subdomains aren't supposed to be public, the public also doesn't need to trust the TLS certs. Sign them with your own CA and trust it on the devices that should be able to access the domains.
  - paranoidrobot 3y ago
    Adding CAs to trust stores on devices and in apps is a major pain. If you have unmanaged devices this becomes even more painful. "Oh, hi, welcome to the company, please install this Root CA onto your machine to access <internal service>" Because you can't scope CAs to specific domains, this causes everyone with any idea about security to start being concerned.
    - eastbound 3y ago
      Every single company does it. The 3 of them: Asking employees to install a CA, using it for “.internal” resources, then ask employees to use a web proxy and MITM their connections. And optionally, leak the CA’s pk to get pawned. It’s the standard operating procedure of any well-run business.

weird-eye-issue 3y ago

I got back an empty list for my domain on Cloudflare with several subdomains (non wildcard) edit: I retried on my computer (was on my phone earlier) and now it returns all of our subdomains, even picking up our test R2 bucket. In guessing I was rate limited because I accidentally loaded the example file a few times

TechBro8615 3y ago

I get a rate limit error when I click the text input (I'm on a VPN).

867-5309 3y ago
use an obscure country like North Macedonia

mmarquezs 3y ago

Nice, last time I used Wolframalpha for this.

ohuf 3y ago

The subdomain explorer may be fun, but their Exploit Observer is really useful: https://www.exploit.observer/ https://www.exploit.observer/

g147 3y ago
thanks!

_cenw 3y ago

https://github.com/projectdiscovery/subfinder https://github.com/projectdiscovery/subfinder does this, but it explains all the methods and lets you choose to only do a passive scan.

gnyman 3y ago

You cannot hide anything on the internet anymore, the full IPv4 range is scanned regularly by multiple entities. If you open a port on a public IP it will get found. If it's a obscure non-standard port it might take longer, but if it's on any of the standard ports it will get probed very quickly and included tools like shodan.io The reason why I'm repeating this, is that not everyone knows this. People still (albeit less) put up elastic and mongodb instances with no authentication on public IP's. The second thing which isn't well known is the Certificate Transparency logs. This is the reason why you can't (without a wildcard cert) hide any HTTPS service. When you ask Let's Encrypt (or any CA actually) to generate veryobscure.domain.tld they will send that to the Certificate Transparency logs. You can find every certificate which was minted for a domain on a tool like https://crt.sh https://crt.sh There are many tools like subdomain.center, https://hackertarget.com/find-dns-host-records/ https://hackertarget.com/find-dns-host-records/ comes to mind. The most impressive one I've seen, which found more much more than expected, is Detectify (which is a paid service, no affiliation), they seem to combine the passive data collection (like subdomain.center) with active brute to find even more subdomains. But you can probably get 95% there by using CT and a brute-force tool like https://github.com/aboul3la/Sublist3r https://github.com/aboul3la/Sublist3r

tamimio 3y ago
> This is the reason why you can't (without a wildcard cert) Guess being security conscious pays off, as testing those on some domains I have, they only managed to show what I want to show, since wildcard will just mask them. That being said, I don’t think anyone should consider a subdomain as a hidden thing, it’s an address after all and should not be considered hidden, assume it’s accessible or put it behind a FW or VPN and have a proper authentication, security by obscurity never works.
fragmede 3y ago
not to underestimate the power of shodan, and oh god don't spin up a default mongo with no auth, but port knocking would seen to counteract this to enough of a degree, not to mention having a service only accessible via Tor. https://wiki.archlinux.org/title/Port_knocking#:~:text=Port%20knocking%20is%20a%20stealth,series%20of%20predefined%20closed%20ports https://wiki.archlinux.org/title/Port_knocking#:~:text=Port%....

yadnst 3y ago

[dead]

p4bl0 3y ago

It gave me empty results for some of my domains that have multiple subdomains that have TLS certificate associated with them so that must appear in the certificate transparency log. I guess it should be "discover some subdomains for some domains".

Semaphor 3y ago
Empty for all my and my work’s domains. Then I tested random .com domains and got results. Seems pretty useless.

donatj 3y ago

Interesting. It only found less that a quarter of the subdomains of the site I work on, and everything it did find is public facing. I wonder if that’s maybe something to do with how we set up certificates for public vs internal subdomains? It even missed “staging.” which should be nearly identical in configuration to www

chillbill 3y ago

[dead]

blueflow 3y ago

I entered my own domains and i got so many garbage entries. It feels like an AI reading letsencrypt logs and then adding made up shit to it.

SushiHippie 3y ago

Note, if you looked up a domain and it had no results, you should check back again after some minutes. I looked my domain up and had zero results, which was weird as it should at least find some in the ct logs, but a few minutes later it showed some subdomains.

LinuxBender 3y ago
It took about 5 minutes for me. It found my apex domain and a sub-domain that must have belonged to the previous renter of my domain name. [1] So I was curious and it turns out the previous renters pages were in Wayback. [2] That page renders as mostly little boxes for me. Funny, I had never bothered to check that. I should check if any of my other domains have snapshots from before I rented them. [1] - https://api.subdomain.center/?domain=ohblog.net https://api.subdomain.center/?domain=ohblog.net [2] - https://web.archive.org/web/20090302094112/https://ohblog.net/ https://web.archive.org/web/20090302094112/https://ohblog.ne...
- SushiHippie 3y ago
  Web archive can also somewhat act as a subdomain finder (not really in this case, only the www subdomain, but still interesting): https://web.archive.org/web/*/ohblog.net https://web.archive.org/web/*/ohblog.net*

TheHappyOddish 3y ago

Hardly "all subdomains". Unless it's doing an AXFR of my zone file (unlikely), this isn't possible. It's a scraper/guesser, using cert transparency, common names, etc. Cute toy, but false claims.

panki27 3y ago
You are correct, I've tested it with my own domains. It does not know the ones running with a wildcard certificate for example.
- wlonkly 3y ago
  It knows many of the wildcard-served customer subdomains of one of my former employers. (They're probably just scraped from search or something, but a wildcard is not sufficient to prevent discovery.)

Andrew018 3y ago

[dead]

xg15 3y ago

I think as soon as cert transparency was introduced, it was pretty clear we would eventually get something like this.

deleted 3y ago

[deleted]

Arubis 3y ago

If this were able to determine which wildcard subdomains were active for a given domain, you could use it to figure out a lot of B2B companies’ client/customer list.

maul666 3y ago

dpd.co.uk