9 ms·

Keep in mind, adding local entries to your external DNS will expose internal details of your network, such as hostnames and IPs. Same goes for Let's Encrypt, du

Keep in mind, adding local entries to your external DNS will expose internal details of your network, such as hostnames and IPs. Same goes for Let's Encrypt, due to Certificate Transparency logging.

iso1210 6y ago
While you'll get the hostnames leaked - you could register them as fake addresses (say an A record for 192.168.0.1 for every address), and have a local DNS server overriding with the real addresses. Whether this is worthwhile or not is debatable. Is the fact your internal server 'gubbins.mydomain.com' exists, or even that it exists on 10.0.41.43 really much use? The other option for internal certificates is to get a wildcard of *.internal.mydomain.com, and spread that wildcard certificate around your network. The final solution is run your own certificate authority and trust it on every browser. For some reason when you import a root certificate you can't typically allow that CA to only be used to authenticate a given subdomain. There are x509 constraints you can use in setting up the CA, but that's rare too, and I'm not sure every tool uses it. In any case, if you go for an internal DNS provision, make sure you set use-application-dns.net to NXDOMAIN on your internal dns server to override DoH too
- cassianoleal 6y ago
  > Is the fact your internal server 'gubbins.mydomain.com' exists, or even that it exists on 10.0.41.43 really much use? Pretty much this. What does it matter if you know certain hostnames or internal IPs on my network? It's all firewalled anyway, and if it wasn't it would be trivial to find them out on your own...
  - Xylakant 6y ago
    It may be of interest for attackers that have no visibility into your network and make cross site attacks against you easier. For example, if I know that your router is available at router.network.internal, I might just try and see if your browser is logged in and send you a link to a page that starts making requests against that interface. Enumerating network resources can certainly be done via other ways, but DNS is a particularly easy one.
    - cassianoleal 6y ago
      Interesting but 192.168.0.1 or 192.168.1.1 would probably cover 80% of that use case. This is just security by obscurity. It doesn't add much if you already implement actual security in the form of firewalls, network segregation, etc.
      - Xylakant 6y ago
        I agree that it's certainly not a substitute for a proper security setup, but it definitely makes an attackers life easier - up to the point where it may get interesting to automate this. So as always, it's a tradeoff. Both options may be acceptable (and I've run a setup where I'd have internal names exposed on public DNS for years), but you need to be aware of the tradeoff to make that judgement.
  - iso1210 6y ago
    I guess it could be useful for human level attacks, something like "Hi it's Bob down in IT, any chance you could reboot 'funkyserver123', it's on 172.18.12.5, for some reason I can't get in"
    - cassianoleal 6y ago
      Why would I answer a request from Bob down in IT about my home network? Edit: I don't know why this is being downvoted. The OP is about a home k8s setup, so unless you live in some castle or palace I very much doubt there will be a Bob down in IT.
  - fomine3 6y ago
    Possibly useful for SSRF attack but the matter is SSRF.
- vetinari 6y ago
  > when you import a root certificate you can't typically allow that CA to only be used to authenticate a given subdomain. While, true, there are some workarounds, that are, ehm, workaroundy. 1. For _your_ domain, you can have CAA records in your DNS. (That solves it for your domains, not for others, if they do not use CAA) 2. Some CA implementations, like FreeIPA one, allow you to issue certificates only for your own domain. They will refuse to sign a certificate for any other domain. So while it is not 100% solution, it is 80% one. > In any case, if you go for an internal DNS provision, make sure you set use-application-dns.net to NXDOMAIN on your internal dns server to override DoH too Also make sure that the browser honors this. Firefox honors canary domain only when in 'auto DoH' mode, but not, when the user has explicitly configured DoH. From https://support.mozilla.org/en-US/kb/canary-domain-use-application-dnsnet https://support.mozilla.org/en-US/kb/canary-domain-use-appli...: > The canary domain only applies to users who have DoH enabled as the default option. It does not apply for users who have made the choice to turn on DoH by themselves.
  - iso1210 6y ago
    I'm thinking more that if you create your own CA and trust it as a root CA, you are responsible for the security of that CA - it can be used against you. This is especially problematic if I want someone else to trust my certs in their browser, I'm asking them to trust any certificate I generate, including google.com, mybank.com, etc.
    - tialaramex 6y ago
      I don't want to trust your certificates in my browser, even if you invent all sorts of sophisticated technical hoops to jump through I have no reason to believe you actually did all that hoop jumping. Get certificates for the names you want from a PKI we both trust. The only one that's likely to be applicable is the Web PKI, which is what was done in this article. For some other purposes, where the Web PKI isn't applicable, you might have a solid argument for why people should trust your PKI. But then they aren't trusting certificates in their browser but in some other likely very constrained system that's easier to reason about. For example I've operated (for past employers) a service that was available over HTTPS to a handful of insurance and financial outfits using mutual TLS. I'd issue client certificates for them, and then they'd use their private key and their certificate to access the service over HTTPS. In that case I ran the CA, and so they're trusting me. But only narrowly to identify customers of the service I run, so it's not tricky to reason about. Now, technically you could use certificates from Let's Encrypt as client certificates, they have the right EKU for that purpose but then your identity has to be an Internet FQDN and you need new certificates every ninety days, which sucks. So although I'd have accommodated a paying customer who firmly insisted on doing that, none of them did.
- aforwardslash 6y ago
  > Whether this is worthwhile or not is debatable. Is the fact your internal server 'gubbins.mydomain.com' exists, or even that it exists on 10.0.41.43 really much use? You're probably aware of tools such as dnsdumpster (dnsdumpster.com) that attempt to map internal networks using dns. It is common to use descriptive prefixes (eg. intranet, gw, fw, dns, jira, mysqldb, etc) that also give some insight on the internal tools and topology. Using the CT logs you can also collect over time changes in network, for servers/endpoints with non-wildcard certificates. Its not a huge risk. But why give a map of the place to a potential attacker, when - as you quite well proposed - you can use simple ways of avoiding it?
  - iso1210 6y ago
    Downsides of a single wildcard certificate has its own issues, the more you pass a single certificate round the more opportunity for it to leak and the wider the risk If you have a per-host DNS based with no valid A records, DOH could well mean worse performance in future as devices don't listen to canary domain and only query locally once getting an NXDOMAIN (and even then might not). You're still leaking the CN and SAN entries via CT. If you run your own CA, every device has to have it installed, and it's a major security risk as losing control of the CA means all your machines can be compromised. Between certificate transparency and DNS over HTTPS, there's no perfect answer.
- tssva 6y ago
  There is no need to have an A record in the external DNS. The DNS challenge uses the contents of a TXT record, _acme-challenge.<YOUR_DOMAIN>, for verification. The certbot DNS plugins will automatically create the TXT record and delete it following verification.
- deleted 6y ago
  [deleted]
mjevans 6y ago
You'd actually end up with a 'split horizon' setup wherein LetsEncrypt (and the public) see those DNS names with the external gateway's IP and ping that for the validation files. Your options at that point are a central 'well known' directory that different hosts can write to (I recommend sshfs), different directories on one host that are checked for any valid file in any of them (by default) or by hostname match in specific, etc. The details depend on your security model.
- pm90 6y ago
  In this case, I don’t think there is a gateway involved at all. I believe Let’s encrypt is verifying the authors domains by (essentially) querying their selected DNS provider to prove ownership over a domain, which resolved to a private IP. So iiuc there is no split horizon, it’s just that the sites would only work for the author.
  - creztoe 6y ago
    I think you are half correct. The gateway has nothing to do with verifying the file during a DNS challenge. However, the IP of the machine requesting the cert IS saved with that cert information and made public. Let's encrypt will even warn you during the verification process.
    - tialaramex 6y ago
      The IP of the machine requesting the certificate is recorded by Let's Encrypt, but it is not (ordinarily) made public and certainly isn't (as you can see by inspecting it for yourself) saved with the certificate information. ISRG is required to keep enough information about the issuances they make to allow them to usefully diagnose problems after the fact. Ideally when we discover a problem it will be possible for the issuer to go back and figure out which (if any) previously issued certificates were affected so that these certificates can be revoked if appropriate. But although they had at one point planned to publish more of this information, they do not in fact do this routinely.
haik90 6y ago
for the domain problem I use wildcard dns. for IP I dont think lets encrypt logging the ip address publicly (let me know if I wrong about it), since I use dns-01 I can generate SSL from anywhere.
- iso1210 6y ago
  > for IP I dont think lets encrypt logging the ip address publicly (let me know if I wrong about it), since I use dns-01 I can generate SSL from anywhere. Yes, if you override DNS (or just provide it) internally then the IP is hidden. The hostname is still leaked though - for example Geodynamics limited have server names of the format fvdcsev01 and fvdccit03 -- https://crt.sh/?id=2381703940 https://crt.sh/?id=2381703940
Cthulhu_ 6y ago
Is that actually valuable information though? I mean internal IP addresses are fairly generic (like the 192.168.x range mentioned in the article).
- aforwardslash 6y ago
  Not always, but sometimes maybe. Check eg. dnsdumpster.com
Hamuko 6y ago
I've solved the IP exposure issue with CNAME records and a local DNS server. So if I want to run a Plex server at example.com, I'll set plex.example.com to be a CNAME for plex.internal and just have the local DNS server resolve what IP address plex.internal is at. Easy way to get HTTPS addresses for internal IP addresses without having the IP address listed publicly.
duskwuff 6y ago
You don't need to add those entries to your external DNS. It's possible to create and validate Let's Encrypt certificates for names which don't exist in public DNS, and use split-horizon DNS to define them on your local network.
w7 6y ago
With regards to DNS-01, only the temporary TXT record needs to be public. If you want everything to be internal you can run your own DNS server on your network and have all the actual A/AAAA/etc records there.
fomine3 6y ago
Anyway cert issued domain name is published for Certificate Transparency. Check crt.sh

Keep in mind, adding local entries to your external DNS will expose internal details of your network, such as hostnames and IPs. Same goes for Let's Encrypt, due to Certificate Transparency logging.

iso1210 6y ago

While you'll get the hostnames leaked - you could register them as fake addresses (say an A record for 192.168.0.1 for every address), and have a local DNS server overriding with the real addresses. Whether this is worthwhile or not is debatable. Is the fact your internal server 'gubbins.mydomain.com' exists, or even that it exists on 10.0.41.43 really much use? The other option for internal certificates is to get a wildcard of *.internal.mydomain.com, and spread that wildcard certificate around your network. The final solution is run your own certificate authority and trust it on every browser. For some reason when you import a root certificate you can't typically allow that CA to only be used to authenticate a given subdomain. There are x509 constraints you can use in setting up the CA, but that's rare too, and I'm not sure every tool uses it. In any case, if you go for an internal DNS provision, make sure you set use-application-dns.net to NXDOMAIN on your internal dns server to override DoH too

cassianoleal 6y ago
> Is the fact your internal server 'gubbins.mydomain.com' exists, or even that it exists on 10.0.41.43 really much use? Pretty much this. What does it matter if you know certain hostnames or internal IPs on my network? It's all firewalled anyway, and if it wasn't it would be trivial to find them out on your own...
- Xylakant 6y ago
  It may be of interest for attackers that have no visibility into your network and make cross site attacks against you easier. For example, if I know that your router is available at router.network.internal, I might just try and see if your browser is logged in and send you a link to a page that starts making requests against that interface. Enumerating network resources can certainly be done via other ways, but DNS is a particularly easy one.
  - cassianoleal 6y ago
    Interesting but 192.168.0.1 or 192.168.1.1 would probably cover 80% of that use case. This is just security by obscurity. It doesn't add much if you already implement actual security in the form of firewalls, network segregation, etc.
    - Xylakant 6y ago
      I agree that it's certainly not a substitute for a proper security setup, but it definitely makes an attackers life easier - up to the point where it may get interesting to automate this. So as always, it's a tradeoff. Both options may be acceptable (and I've run a setup where I'd have internal names exposed on public DNS for years), but you need to be aware of the tradeoff to make that judgement.
- iso1210 6y ago
  I guess it could be useful for human level attacks, something like "Hi it's Bob down in IT, any chance you could reboot 'funkyserver123', it's on 172.18.12.5, for some reason I can't get in"
  - cassianoleal 6y ago
    Why would I answer a request from Bob down in IT about my home network? Edit: I don't know why this is being downvoted. The OP is about a home k8s setup, so unless you live in some castle or palace I very much doubt there will be a Bob down in IT.
- fomine3 6y ago
  Possibly useful for SSRF attack but the matter is SSRF.
vetinari 6y ago
> when you import a root certificate you can't typically allow that CA to only be used to authenticate a given subdomain. While, true, there are some workarounds, that are, ehm, workaroundy. 1. For _your_ domain, you can have CAA records in your DNS. (That solves it for your domains, not for others, if they do not use CAA) 2. Some CA implementations, like FreeIPA one, allow you to issue certificates only for your own domain. They will refuse to sign a certificate for any other domain. So while it is not 100% solution, it is 80% one. > In any case, if you go for an internal DNS provision, make sure you set use-application-dns.net to NXDOMAIN on your internal dns server to override DoH too Also make sure that the browser honors this. Firefox honors canary domain only when in 'auto DoH' mode, but not, when the user has explicitly configured DoH. From https://support.mozilla.org/en-US/kb/canary-domain-use-application-dnsnet https://support.mozilla.org/en-US/kb/canary-domain-use-appli...: > The canary domain only applies to users who have DoH enabled as the default option. It does not apply for users who have made the choice to turn on DoH by themselves.
- iso1210 6y ago
  I'm thinking more that if you create your own CA and trust it as a root CA, you are responsible for the security of that CA - it can be used against you. This is especially problematic if I want someone else to trust my certs in their browser, I'm asking them to trust any certificate I generate, including google.com, mybank.com, etc.
  - tialaramex 6y ago
    I don't want to trust your certificates in my browser, even if you invent all sorts of sophisticated technical hoops to jump through I have no reason to believe you actually did all that hoop jumping. Get certificates for the names you want from a PKI we both trust. The only one that's likely to be applicable is the Web PKI, which is what was done in this article. For some other purposes, where the Web PKI isn't applicable, you might have a solid argument for why people should trust your PKI. But then they aren't trusting certificates in their browser but in some other likely very constrained system that's easier to reason about. For example I've operated (for past employers) a service that was available over HTTPS to a handful of insurance and financial outfits using mutual TLS. I'd issue client certificates for them, and then they'd use their private key and their certificate to access the service over HTTPS. In that case I ran the CA, and so they're trusting me. But only narrowly to identify customers of the service I run, so it's not tricky to reason about. Now, technically you could use certificates from Let's Encrypt as client certificates, they have the right EKU for that purpose but then your identity has to be an Internet FQDN and you need new certificates every ninety days, which sucks. So although I'd have accommodated a paying customer who firmly insisted on doing that, none of them did.
aforwardslash 6y ago
> Whether this is worthwhile or not is debatable. Is the fact your internal server 'gubbins.mydomain.com' exists, or even that it exists on 10.0.41.43 really much use? You're probably aware of tools such as dnsdumpster (dnsdumpster.com) that attempt to map internal networks using dns. It is common to use descriptive prefixes (eg. intranet, gw, fw, dns, jira, mysqldb, etc) that also give some insight on the internal tools and topology. Using the CT logs you can also collect over time changes in network, for servers/endpoints with non-wildcard certificates. Its not a huge risk. But why give a map of the place to a potential attacker, when - as you quite well proposed - you can use simple ways of avoiding it?
- iso1210 6y ago
  Downsides of a single wildcard certificate has its own issues, the more you pass a single certificate round the more opportunity for it to leak and the wider the risk If you have a per-host DNS based with no valid A records, DOH could well mean worse performance in future as devices don't listen to canary domain and only query locally once getting an NXDOMAIN (and even then might not). You're still leaking the CN and SAN entries via CT. If you run your own CA, every device has to have it installed, and it's a major security risk as losing control of the CA means all your machines can be compromised. Between certificate transparency and DNS over HTTPS, there's no perfect answer.
tssva 6y ago
There is no need to have an A record in the external DNS. The DNS challenge uses the contents of a TXT record, _acme-challenge.<YOUR_DOMAIN>, for verification. The certbot DNS plugins will automatically create the TXT record and delete it following verification.
deleted 6y ago
[deleted]

mjevans 6y ago

You'd actually end up with a 'split horizon' setup wherein LetsEncrypt (and the public) see those DNS names with the external gateway's IP and ping that for the validation files. Your options at that point are a central 'well known' directory that different hosts can write to (I recommend sshfs), different directories on one host that are checked for any valid file in any of them (by default) or by hostname match in specific, etc. The details depend on your security model.

pm90 6y ago
In this case, I don’t think there is a gateway involved at all. I believe Let’s encrypt is verifying the authors domains by (essentially) querying their selected DNS provider to prove ownership over a domain, which resolved to a private IP. So iiuc there is no split horizon, it’s just that the sites would only work for the author.
- creztoe 6y ago
  I think you are half correct. The gateway has nothing to do with verifying the file during a DNS challenge. However, the IP of the machine requesting the cert IS saved with that cert information and made public. Let's encrypt will even warn you during the verification process.
  - tialaramex 6y ago
    The IP of the machine requesting the certificate is recorded by Let's Encrypt, but it is not (ordinarily) made public and certainly isn't (as you can see by inspecting it for yourself) saved with the certificate information. ISRG is required to keep enough information about the issuances they make to allow them to usefully diagnose problems after the fact. Ideally when we discover a problem it will be possible for the issuer to go back and figure out which (if any) previously issued certificates were affected so that these certificates can be revoked if appropriate. But although they had at one point planned to publish more of this information, they do not in fact do this routinely.

haik90 6y ago

for the domain problem I use wildcard dns. for IP I dont think lets encrypt logging the ip address publicly (let me know if I wrong about it), since I use dns-01 I can generate SSL from anywhere.

iso1210 6y ago
> for IP I dont think lets encrypt logging the ip address publicly (let me know if I wrong about it), since I use dns-01 I can generate SSL from anywhere. Yes, if you override DNS (or just provide it) internally then the IP is hidden. The hostname is still leaked though - for example Geodynamics limited have server names of the format fvdcsev01 and fvdccit03 -- https://crt.sh/?id=2381703940 https://crt.sh/?id=2381703940

Cthulhu_ 6y ago

Is that actually valuable information though? I mean internal IP addresses are fairly generic (like the 192.168.x range mentioned in the article).

aforwardslash 6y ago
Not always, but sometimes maybe. Check eg. dnsdumpster.com

Hamuko 6y ago

I've solved the IP exposure issue with CNAME records and a local DNS server. So if I want to run a Plex server at example.com, I'll set plex.example.com to be a CNAME for plex.internal and just have the local DNS server resolve what IP address plex.internal is at. Easy way to get HTTPS addresses for internal IP addresses without having the IP address listed publicly.

duskwuff 6y ago

You don't need to add those entries to your external DNS. It's possible to create and validate Let's Encrypt certificates for names which don't exist in public DNS, and use split-horizon DNS to define them on your local network.

w7 6y ago

With regards to DNS-01, only the temporary TXT record needs to be public. If you want everything to be internal you can run your own DNS server on your network and have all the actual A/AAAA/etc records there.

fomine3 6y ago

Anyway cert issued domain name is published for Certificate Transparency. Check crt.sh