5 ms·

As others have said, certificate transparency seems to be doing some heavy lifting here. It reports subdomains for me that have never had a public CNAME or A re

As others have said, certificate transparency seems to be doing some heavy lifting here. It reports subdomains for me that have never had a public CNAME or A record, but have had let's encrypt certs issued for internal use. It's also missing some that have not had certs issued, but that are in public DNS

TekMol 3y ago
That's why HTTPS is still a pain in the butt. 30 years after it was invented. I don't want internally used subdomains to be public. Because of certificate transparency, the only way to achieve that is via wildcard certs. Let's encrypt only supports cumbersome validation methods for those. Like changing DNS records every time you need to renew the cert. Pretty annoying.
- proto_lambda 3y ago
  If the subdomains aren't supposed to be public, the public also doesn't need to trust the TLS certs. Sign them with your own CA and trust it on the devices that should be able to access the domains.
  - paranoidrobot 3y ago
    Adding CAs to trust stores on devices and in apps is a major pain. If you have unmanaged devices this becomes even more painful. "Oh, hi, welcome to the company, please install this Root CA onto your machine to access <internal service>" Because you can't scope CAs to specific domains, this causes everyone with any idea about security to start being concerned.
    - eastbound 3y ago
      Every single company does it. The 3 of them: Asking employees to install a CA, using it for “.internal” resources, then ask employees to use a web proxy and MITM their connections. And optionally, leak the CA’s pk to get pawned. It’s the standard operating procedure of any well-run business.
    - numpad0 3y ago
      Will HSTS on a random cert work? Click to accept once, pinned thereafter, or do browsers ignore that for untrusted certs?
      - Thorrez 3y ago
        That's not HSTS, that's TOFU. HSTS is about remembering to do an http:// http:// -> https:// https:// redirect. It's not about remembering a cert. The downside of TOFU in browsers, is that it trains users to always click through cert warnings. Train them to do it once, and they'll click through it again when there's a real attack. The warning is the same on the first time visiting the site and on a later time visiting it if the cert has changed. The TOFU UX in SSH is better, because it displays a different warning for when SSHing to a site for the first time vs SSHing to a site again and the cert has changed. https://en.wikipedia.org/wiki/Trust_on_first_use https://en.wikipedia.org/wiki/Trust_on_first_use
        dspillett 3y ago
        Many of our clients send automated updates for our systems for data managed in other services via SFTP. It surprises me that few seem to bother verifying the host fingerprints, just blindly accepting them on first connection, given how paranoid they are (quite rightly, the data contains staff and customer information) otherwise.
    - tgsovlerkhgsel 3y ago
      You can scope CAs with name constraints. However, I believe many implementations ignore constraints on root CAs. Not sure if there is some practical way with cross-signing around that (giving users the choice between trusting your CA and creating their own and cross-signing your CA with that).
      - jpace121 3y ago
        I looked before I started using Let’s Encrypt for some internal stuff and there really isn’t a way to use name constraints in a practical way with modern web browsers at this point. If you’re not using a browser, things get a lot easier, but for browsers you sort of got to suck up that you can’t really avoid the “big” internet.
        msm_ 3y ago
        There is a way, I've recently generated my own CA with domain name constraint, trusted it, and used it cross sign my company's self signed CA. It works like a charm.
    - bostik 3y ago
      As sibling poster already wrote, technically you can scope a CA to a set of subdomains only. Or try. The spec entry is "nameConstraints" but for a number of reasons it may not be well supported. Some of those reasons are absolutely hilarious. I needed to set up an internal CA back in 2015, and wanted to limit the blast radius in case the private key was leaked. (Usually a "when", not "if" scenario.) I learned about the nameConstraints field and tried to use it. OpenSSL would ignore the key in a CSR input file. Okay, fine, the spec has an OID for the field so I reached for the nearest ASN.1 library to construct a modified CSR with the field in place. OpenSSL broke trying to parse the file. Go's implementation blew up with a magnificent trace. I gave up and the internal CA was generated with a global validity scope. I later learned that apparently Microsoft's PKI libraries had support for scope limits, but the feature was not used in real life. Likely because if such a thing came into contact with anything else in the wild, the underlying libraries would just implode. If you had a self-signed client cert with a nameConstraints in the supplied CA chain, you could probably still crash a non-trivial fraction of web servers.
      - tikkabhuna 3y ago
        > OpenSSL would ignore the key in a CSR input file. OpenSSL by default ignores many (/all?) extensions for security. You can still manually add the nameConstraints when signing the CA cert. https://security.stackexchange.com/a/150175 https://security.stackexchange.com/a/150175
    - layer8 3y ago
      Where I work, having internal services be accessed by employees’ own, unmanaged devices would be a no-go anyway. It would be considered a huge security loophole.
  - withinboredom 3y ago
    Yeah. In that case, it's just easier to get a really cheap wild-card cert signed by a low-cost reseller for <50 bucks. They only reason to care about big-name certs is compatibility with all the devices out there, but if you don't need compatibility, then get the cheapest thing you can.
  - Figs 3y ago
    Non-public usage doesn't necessarily mean that only devices under your direct control need access. Slack needs access to some of my organization's systems, for example, to support the way we collaborate on our projects -- but the general public doesn't and would likely just be confused if they stumbled into one of our infrastructure subdomains instead of visiting our public website.
  - justsomehnguy 3y ago
    > and trust it on the devices that should be able to access the domains Sometimes it's not an option. I spent too many hours trying to figure out why some Android apps didn't want to talk with a service I self-hosted. They just ignored my Root CA cert installed on the phone.
- speedgoose 3y ago
  I think you are supposed to automate the renewal with the DNS record method.
  - tjoff 3y ago
    Which most registrars don't support.
    - Ayesh 3y ago
      Nobody should be using DNS service provided by their registrars. They tend to be insecure, slow, and like you said, missing a lot of features.
      - tjoff 3y ago
        We are getting back to the pain in the butt though...
Hamuko 3y ago
I have a single wildcard certificate for my internal domain name and ~10 CNAMEs for various service subdomains in the network (plex.server.com, grafana.server.com, etc). This tool found zero subdomains for my internal domain.
- rft 3y ago
  I have a similar setup (*.home.domain.com DNS auth with LE -> service1.home.domain.com etc.) for my personal, but externally reachable domain, and I get the same results. I went the wildcard route just due to a bit of paranoia, nice to see that it actually worked out in this case. As this (I expect) heavily uses cert transparency in the background, I want to point out another use case for that service. You can search the CT logs with wildcards to find your domain "neighbors" on other TLDs: https://crt.sh/?Identity=google.%25&match=ILIKE https://crt.sh/?Identity=google.%25&match=ILIKE This usually gives you somewhat more active websites compared to just checking whether you can register the domain and somewhat weeds out squatted domains. I found that for our company one TLD contained a NSFW games store that way.
- SushiHippie 3y ago
  You should check again after some time, the first time I looked up my domain there were no results, few minutes later it found some of my subdomains.
  - Hamuko 3y ago
    Still nothing.
Symbiote 3y ago
At work we have a wildcard certificate for most services we host on our own infrastructure. Most public websites have been detected, and some internal ones which have probably been referenced in public GitHub issues and so on. They've done simple reverse DNS lookups on our public IP range and indexed all those hostnames. Certificate transparency logs have found names used for externally hosted websites. There are some pretty old hostnames which haven't been used for 5 years or more, and were probably found with reverse DNS at the time.

TekMol 3y ago

That's why HTTPS is still a pain in the butt. 30 years after it was invented. I don't want internally used subdomains to be public. Because of certificate transparency, the only way to achieve that is via wildcard certs. Let's encrypt only supports cumbersome validation methods for those. Like changing DNS records every time you need to renew the cert. Pretty annoying.

proto_lambda 3y ago
If the subdomains aren't supposed to be public, the public also doesn't need to trust the TLS certs. Sign them with your own CA and trust it on the devices that should be able to access the domains.
- paranoidrobot 3y ago
  Adding CAs to trust stores on devices and in apps is a major pain. If you have unmanaged devices this becomes even more painful. "Oh, hi, welcome to the company, please install this Root CA onto your machine to access <internal service>" Because you can't scope CAs to specific domains, this causes everyone with any idea about security to start being concerned.
  - eastbound 3y ago
    Every single company does it. The 3 of them: Asking employees to install a CA, using it for “.internal” resources, then ask employees to use a web proxy and MITM their connections. And optionally, leak the CA’s pk to get pawned. It’s the standard operating procedure of any well-run business.
  - numpad0 3y ago
    Will HSTS on a random cert work? Click to accept once, pinned thereafter, or do browsers ignore that for untrusted certs?
    - Thorrez 3y ago
      That's not HSTS, that's TOFU. HSTS is about remembering to do an http:// http:// -> https:// https:// redirect. It's not about remembering a cert. The downside of TOFU in browsers, is that it trains users to always click through cert warnings. Train them to do it once, and they'll click through it again when there's a real attack. The warning is the same on the first time visiting the site and on a later time visiting it if the cert has changed. The TOFU UX in SSH is better, because it displays a different warning for when SSHing to a site for the first time vs SSHing to a site again and the cert has changed. https://en.wikipedia.org/wiki/Trust_on_first_use https://en.wikipedia.org/wiki/Trust_on_first_use
      - dspillett 3y ago
        Many of our clients send automated updates for our systems for data managed in other services via SFTP. It surprises me that few seem to bother verifying the host fingerprints, just blindly accepting them on first connection, given how paranoid they are (quite rightly, the data contains staff and customer information) otherwise.
  - tgsovlerkhgsel 3y ago
    You can scope CAs with name constraints. However, I believe many implementations ignore constraints on root CAs. Not sure if there is some practical way with cross-signing around that (giving users the choice between trusting your CA and creating their own and cross-signing your CA with that).
    - jpace121 3y ago
      I looked before I started using Let’s Encrypt for some internal stuff and there really isn’t a way to use name constraints in a practical way with modern web browsers at this point. If you’re not using a browser, things get a lot easier, but for browsers you sort of got to suck up that you can’t really avoid the “big” internet.
      - msm_ 3y ago
        There is a way, I've recently generated my own CA with domain name constraint, trusted it, and used it cross sign my company's self signed CA. It works like a charm.
  - bostik 3y ago
    As sibling poster already wrote, technically you can scope a CA to a set of subdomains only. Or try. The spec entry is "nameConstraints" but for a number of reasons it may not be well supported. Some of those reasons are absolutely hilarious. I needed to set up an internal CA back in 2015, and wanted to limit the blast radius in case the private key was leaked. (Usually a "when", not "if" scenario.) I learned about the nameConstraints field and tried to use it. OpenSSL would ignore the key in a CSR input file. Okay, fine, the spec has an OID for the field so I reached for the nearest ASN.1 library to construct a modified CSR with the field in place. OpenSSL broke trying to parse the file. Go's implementation blew up with a magnificent trace. I gave up and the internal CA was generated with a global validity scope. I later learned that apparently Microsoft's PKI libraries had support for scope limits, but the feature was not used in real life. Likely because if such a thing came into contact with anything else in the wild, the underlying libraries would just implode. If you had a self-signed client cert with a nameConstraints in the supplied CA chain, you could probably still crash a non-trivial fraction of web servers.
    - tikkabhuna 3y ago
      > OpenSSL would ignore the key in a CSR input file. OpenSSL by default ignores many (/all?) extensions for security. You can still manually add the nameConstraints when signing the CA cert. https://security.stackexchange.com/a/150175 https://security.stackexchange.com/a/150175
  - layer8 3y ago
    Where I work, having internal services be accessed by employees’ own, unmanaged devices would be a no-go anyway. It would be considered a huge security loophole.
- withinboredom 3y ago
  Yeah. In that case, it's just easier to get a really cheap wild-card cert signed by a low-cost reseller for <50 bucks. They only reason to care about big-name certs is compatibility with all the devices out there, but if you don't need compatibility, then get the cheapest thing you can.
- Figs 3y ago
  Non-public usage doesn't necessarily mean that only devices under your direct control need access. Slack needs access to some of my organization's systems, for example, to support the way we collaborate on our projects -- but the general public doesn't and would likely just be confused if they stumbled into one of our infrastructure subdomains instead of visiting our public website.
- justsomehnguy 3y ago
  > and trust it on the devices that should be able to access the domains Sometimes it's not an option. I spent too many hours trying to figure out why some Android apps didn't want to talk with a service I self-hosted. They just ignored my Root CA cert installed on the phone.
speedgoose 3y ago
I think you are supposed to automate the renewal with the DNS record method.
- tjoff 3y ago
  Which most registrars don't support.
  - Ayesh 3y ago
    Nobody should be using DNS service provided by their registrars. They tend to be insecure, slow, and like you said, missing a lot of features.
    - tjoff 3y ago
      We are getting back to the pain in the butt though...

Hamuko 3y ago

I have a single wildcard certificate for my internal domain name and ~10 CNAMEs for various service subdomains in the network (plex.server.com, grafana.server.com, etc). This tool found zero subdomains for my internal domain.

rft 3y ago
I have a similar setup (*.home.domain.com DNS auth with LE -> service1.home.domain.com etc.) for my personal, but externally reachable domain, and I get the same results. I went the wildcard route just due to a bit of paranoia, nice to see that it actually worked out in this case. As this (I expect) heavily uses cert transparency in the background, I want to point out another use case for that service. You can search the CT logs with wildcards to find your domain "neighbors" on other TLDs: https://crt.sh/?Identity=google.%25&match=ILIKE https://crt.sh/?Identity=google.%25&match=ILIKE This usually gives you somewhat more active websites compared to just checking whether you can register the domain and somewhat weeds out squatted domains. I found that for our company one TLD contained a NSFW games store that way.
SushiHippie 3y ago
You should check again after some time, the first time I looked up my domain there were no results, few minutes later it found some of my subdomains.
- Hamuko 3y ago
  Still nothing.

Symbiote 3y ago

At work we have a wildcard certificate for most services we host on our own infrastructure. Most public websites have been detected, and some internal ones which have probably been referenced in public GitHub issues and so on. They've done simple reverse DNS lookups on our public IP range and indexed all those hostnames. Certificate transparency logs have found names used for externally hosted websites. There are some pretty old hostnames which haven't been used for 5 years or more, and were probably found with reverse DNS at the time.