9 ms·

I prefer to assign an external name to an internal device and grab a free SSL cert from LetsEncrypt, but using DNS challenge instead as internal IP addresses ar

I prefer to assign an external name to an internal device and grab a free SSL cert from LetsEncrypt, but using DNS challenge instead as internal IP addresses aren't reachable by their servers.

DandyDev 2y ago
I do this as well, but be aware that these external names you're using for internal devices become a matter of public record this way. If that's okay for you (it is for me), then this is a good solution. The advantage is also that you run no risk of name clashes because you actually own the domain
- xfer 2y ago
  Or use a wildcard cert for all internal certs.
  - ndsipa_pomu 2y ago
    That could be a good idea, though it means that the certificate/key has to be well guarded.
  - project2501a 2y ago
    Please don't. Technical debt accumulates by force of practice.
    - qwertox 2y ago
      It's working good for me. My technical debt is to always make sure that I'm able to renew a certificate and that the distribution occurs successfully. I don't see how other solutions are less problematic.
  - pridkett 2y ago
    This is exactly what I do. After seeing how much of my internal network was exposed in certificate transparency logs, I noped out and just do a DNS challenge for a wildcard for almost everything. Now it’s have a nice script that distributes my key automatically to 20 or so hosts and apps and have a real SSL cert on everything from my UDM Pro to my Synology to random Raspberry Pis running containers. Most of which have domain names that only resolve on my local network. This is made possible by a fairly robust DNS setup that consists of not only giving A records to all my hosts automatically, but also adding in CNAMEs for services and blocking almost all outbound DNS, DNS over TLS, DoH, etc.
    - dopp0 2y ago
      > fairly robust DNS setup that consists of not only giving A records to all my hosts looks nice, can you give more details on this? tks!
- magicalhippo 2y ago
  I decided to try split DNS to avoid leaking the internal IPs, but it turned out a bit more fragile than I imagined. Especially Android is finicky, ignoring your DNS server if it doesn't like your setup. For example, if it gets an IPv6 address, it requires the DNS server to also have an IPv6 address, or it'll use Google's DNS servers. It works now but I'm not convinced it's worth it for me.
  - Hamuko 2y ago
    I use CNAME records and it works on everything except Windows, where it works sometimes. Basically, CNAME record from service.myserver.com to myserver.internal on a public DNS server, A record from myserver.internal to 1.2.3.4 on private DNS server. I think I could maybe get it working on Windows too by tweaking the TTLs. Currently both DNS servers are automatically setting the TTL and I think Windows freaks out about that.
    - ebb_earl_co 2y ago
      This seems like a good technique. What DNS software do you use?
      - Hamuko 2y ago
        I just use the one built into my UniFi router. Public DNS side is Cloudflare, which allows easy DNS validation for Let's Encrypt.
  - capitol_ 2y ago
    Split DNS causes lots headaches, it also makes it really hard to root cause analysis of failures when they involve DNS.
- ndsipa_pomu 2y ago
  > be aware that these external names you're using for internal devices become a matter of public record this way Yes, I sometimes think about that, but have come to the conclusion that it's not likely to make any difference. If someone is trying to infiltrate my home network, then it's not going to really help them to know internal IP addresses as by the time they get to use them, they're already in.
  - qwertox 2y ago
    You don't need to publish the IP addresses publicly if you use an internal DNS server. I think even Pi-hole could do this.
  - dspillett 2y ago
    > If someone is trying to infiltrate my home network I don't think the publishing of host names was mentioned as a concern for small home networks, but more for larger organisations that might be subject to a coordinated break-in or simply have trade secrets¹² that might be hinted at by careless naming of resources. ---- [1] Their next big product/enhancement, as yet unannounced even within the company, for instance. [2] Hmm, checking what is recorded against one of DayJob's domains I see clues as to who some of our clients are. Not really a significant issue for security at all, but I know at least some of our contracts say we shouldn't openly talk about that we provide services to that client³ so I'll drop a message to the ISC to suggest we discuss if we need to care about the matter… [3] Though that is mostly in the form of not using their logos in our advertising and such.
    - xena 2y ago
      That's why you have an internal domain that's not related to the company. Something like "packet-flinging.ninja". Everything's a tradeoff though.
      - deltaburnt 2y ago
        It seems really easy to associate a company with its internal domain though? Unless the company treats it as a secret only known between machines.
- deleted 2y ago
  [deleted]
- prmoustache 2y ago
  you can use a wildcard of type *.internal.example.com or use names that do not relate to the service name if you want to obfuscate the tech stack used. The only thing public is that you may have an internal network with nodes.
- Eikon 2y ago
  Shameless plug: https://www.merklemap.com/ https://www.merklemap.com/
- js2 2y ago
  Using a wildcard cert doesn't leak anything much. I went with `*.home.example.com` for my internal stuff.
djhworld 2y ago
I last looked at LetsEncrypt maybe 8-9 years ago, I thought it was awesome but not suitable for my internal stuff due to the http challenge requirement, so I went down the self signed CA route and stuck with that, and didn’t really keep up with developments in the space It was only until recently someone told me about the DNS challenge and I immediately ported everything over with a wildcard cert - its been great!
- bmicraft 2y ago
  They've introduced the dns challenge almost 9 years ago, you must have barely missed it!
candiddevmike 2y ago
Obligatory if DNS validation is good enough, DANE should've been too. Yes, MITM things could potentially ensue on untrusted networks without DNSSEC, but that's perfect being the enemy of good territory IMO. This would allow folks to have .internal with auto-discovered, decentralized, trusted PKI. It would also enable something like a DNSSEC on/off toggle switch for IoT devices to allow owners to MITM them and provide local functionality for their cloud services.
- ndsipa_pomu 2y ago
  I hadn't heard of DANE, so looked it up and found the wikipedia entry: https://en.wikipedia.org/wiki/DNS-based_Authentication_of_Named_Entities https://en.wikipedia.org/wiki/DNS-based_Authentication_of_Na... According to that, it's not supported by Chrome, nor Firefox.
  - tptacek 2y ago
    It was, once, and then got pulled.
  - teddyh 2y ago
    It’s customarily used for e-mail transport.
    - tptacek 2y ago
      No, it isn't; it can't be, because none of the major email domains are DNSSEC-signed to begin with. Let me know if I've misunderstood your point, and there some other widespread niche usage DANE finds in SMTP.
      - teddyh 2y ago
        “Customarily” must not necessarily mean “major email domains”. As I understand it, DANE is commonly used by organizations wanting to secure e-mail between them; DANE enforcement is agreed by both parties, and then used without issue.
Tepix 2y ago
How do you automate it?
- globular-toast 2y ago
  Not OP but I have a couple of implementations: one using caddyserver[0] as a reverse proxy in a docker-compose set up, and the other is a Kubernetes cluster using cert-manager[1]. [0] https://caddyserver.com/ https://caddyserver.com/ [1] https://cert-manager.io/ https://cert-manager.io/
- ndsipa_pomu 2y ago
  I use Dynu.com as my DNS provider (they're cheap, provide APIs and very fast to update which is great for home IP addresses that may change). Then, to get the certificates, I use https://github.com/acmesh-official/acme.sh https://github.com/acmesh-official/acme.sh which is a shell script that supports multiple certificate and DNS providers. Copying the certificates to the relevant machines is done by a custom BASH script that runs the relevant acme.sh commands. One advantage of DNS challenge is that it can be run anywhere (i.e. doesn't need to run on the webserver) - it just needs the relevant credentials to add a DNS TXT record. I've got my automation wrapped up into a Docker container.
lolinder 2y ago
Yep. I tried the custom-root-CA approach for a long time, but there were just too many problems with it: * Loading it into every device was more work than it sounds. We have Android, iOS, Mac, Windows, and Linux, all of which have their own rules. * Even once loaded, some applications come with their own set of root CAs. Some of those have a custom way of adding a new one (Firefox), others you just had to accept the invalid cert each time, and still others just refused to work. * I deploy my self-hosted stuff with Docker, which means that not only does each device need to have the root CA added to it but every Docker image that talks to the internal network needs to have it as well. This ends up being a mix of the previous two problems, as I now have to figure out how to mount the CA on an eclectic bunch of distros and I often then have to figure out why the dockerized application isn't using the CA. In the end I settled on a DNS-challenge wildcard SSL cert loaded into Caddy, with Caddy terminating TLS for everything that's on my home server. It's way simpler to configure the single server (or even 2-3 servers) than every single client.
- poincaredisk 2y ago
  Historically, before wildcard certificates were suddenly available for free, this leaked all internal domains to the internet, but now it's mostly a solved problem.
  - vegardx 2y ago
    I don't understand why that is such a huge problem. The alternatives have much more severe problems, all from reusing a wildcard in many places to running your own PKI.
    - NovemberWhiskey 2y ago
      It depends on your risk profile, but there are definitely people who'd rather run their own PKI than permit threat actor reconnaissance by publishing internal hostnames to CT logs.
      - vegardx 2y ago
bombcar 2y ago
Just get a wildcard cert from letsencyrpt and copy it internally. Then you don’t even need to DNS leak names to LE.
giobox 2y ago
LetsEncrypt + DNS challenge + DNS provider with letsencrpyt compatible API for modifying records works fantastically well for getting "real" https/SSL working for private IP addresses, the automatic renewals make it largely set and forget with very little config or setup required. I've had working validly signed SSL on literally all my private home self-hosted services and load-balancers internally for years this way. It also easily switches to a production like setup if you later did decide to host something on the public internet.
thatcherc 2y ago
This sounds like something I'd want to do! Is the idea that you'd have a public domain name like "internal.thatcherc.com" resolve to an internal IP address like 10.0.10.5? I've wondered about setting this up for some local services I have but I wasn't sure if it was a commonly-done thing.
- AdamJacobMuller 2y ago
  I've been doing this for a year or two with k3s + cert-manager. Works great. In my case everything points to a tailscale operator endpoint, which goes to nginx ingress, which routes to the appropriate pods. It's very much a set-and-forget solution.
wkat4242 2y ago
Yeah that's what I do. If you use anything other than Cloudflare its really really hard to get the authentication plugins going on every different web server though. Every server supports a different subset of providers and usually you have to install the plugins separately. It's a bit of a nightmare. But once it's dialled in it's ok. I didn't like this approach because I don't like to leak information about my internal setup but I found that you don't even have to register your servers on a public DNS so it's ok. Just the domain has to exist. It does create very temporary TXT records though.

I prefer to assign an external name to an internal device and grab a free SSL cert from LetsEncrypt, but using DNS challenge instead as internal IP addresses aren't reachable by their servers.

DandyDev 2y ago

I do this as well, but be aware that these external names you're using for internal devices become a matter of public record this way. If that's okay for you (it is for me), then this is a good solution. The advantage is also that you run no risk of name clashes because you actually own the domain

xfer 2y ago
Or use a wildcard cert for all internal certs.
- ndsipa_pomu 2y ago
  That could be a good idea, though it means that the certificate/key has to be well guarded.
- project2501a 2y ago
  Please don't. Technical debt accumulates by force of practice.
  - qwertox 2y ago
    It's working good for me. My technical debt is to always make sure that I'm able to renew a certificate and that the distribution occurs successfully. I don't see how other solutions are less problematic.
- pridkett 2y ago
  This is exactly what I do. After seeing how much of my internal network was exposed in certificate transparency logs, I noped out and just do a DNS challenge for a wildcard for almost everything. Now it’s have a nice script that distributes my key automatically to 20 or so hosts and apps and have a real SSL cert on everything from my UDM Pro to my Synology to random Raspberry Pis running containers. Most of which have domain names that only resolve on my local network. This is made possible by a fairly robust DNS setup that consists of not only giving A records to all my hosts automatically, but also adding in CNAMEs for services and blocking almost all outbound DNS, DNS over TLS, DoH, etc.
  - dopp0 2y ago
    > fairly robust DNS setup that consists of not only giving A records to all my hosts looks nice, can you give more details on this? tks!
magicalhippo 2y ago
I decided to try split DNS to avoid leaking the internal IPs, but it turned out a bit more fragile than I imagined. Especially Android is finicky, ignoring your DNS server if it doesn't like your setup. For example, if it gets an IPv6 address, it requires the DNS server to also have an IPv6 address, or it'll use Google's DNS servers. It works now but I'm not convinced it's worth it for me.
- Hamuko 2y ago
  I use CNAME records and it works on everything except Windows, where it works sometimes. Basically, CNAME record from service.myserver.com to myserver.internal on a public DNS server, A record from myserver.internal to 1.2.3.4 on private DNS server. I think I could maybe get it working on Windows too by tweaking the TTLs. Currently both DNS servers are automatically setting the TTL and I think Windows freaks out about that.
  - ebb_earl_co 2y ago
    This seems like a good technique. What DNS software do you use?
    - Hamuko 2y ago
      I just use the one built into my UniFi router. Public DNS side is Cloudflare, which allows easy DNS validation for Let's Encrypt.
- capitol_ 2y ago
  Split DNS causes lots headaches, it also makes it really hard to root cause analysis of failures when they involve DNS.
ndsipa_pomu 2y ago
> be aware that these external names you're using for internal devices become a matter of public record this way Yes, I sometimes think about that, but have come to the conclusion that it's not likely to make any difference. If someone is trying to infiltrate my home network, then it's not going to really help them to know internal IP addresses as by the time they get to use them, they're already in.
- qwertox 2y ago
  You don't need to publish the IP addresses publicly if you use an internal DNS server. I think even Pi-hole could do this.
- dspillett 2y ago
  > If someone is trying to infiltrate my home network I don't think the publishing of host names was mentioned as a concern for small home networks, but more for larger organisations that might be subject to a coordinated break-in or simply have trade secrets¹² that might be hinted at by careless naming of resources. ---- [1] Their next big product/enhancement, as yet unannounced even within the company, for instance. [2] Hmm, checking what is recorded against one of DayJob's domains I see clues as to who some of our clients are. Not really a significant issue for security at all, but I know at least some of our contracts say we shouldn't openly talk about that we provide services to that client³ so I'll drop a message to the ISC to suggest we discuss if we need to care about the matter… [3] Though that is mostly in the form of not using their logos in our advertising and such.
  - xena 2y ago
    That's why you have an internal domain that's not related to the company. Something like "packet-flinging.ninja". Everything's a tradeoff though.
    - deltaburnt 2y ago
      It seems really easy to associate a company with its internal domain though? Unless the company treats it as a secret only known between machines.
deleted 2y ago
[deleted]
prmoustache 2y ago
you can use a wildcard of type *.internal.example.com or use names that do not relate to the service name if you want to obfuscate the tech stack used. The only thing public is that you may have an internal network with nodes.
Eikon 2y ago
Shameless plug: https://www.merklemap.com/ https://www.merklemap.com/
js2 2y ago
Using a wildcard cert doesn't leak anything much. I went with `*.home.example.com` for my internal stuff.

djhworld 2y ago

I last looked at LetsEncrypt maybe 8-9 years ago, I thought it was awesome but not suitable for my internal stuff due to the http challenge requirement, so I went down the self signed CA route and stuck with that, and didn’t really keep up with developments in the space It was only until recently someone told me about the DNS challenge and I immediately ported everything over with a wildcard cert - its been great!

bmicraft 2y ago
They've introduced the dns challenge almost 9 years ago, you must have barely missed it!

candiddevmike 2y ago

Obligatory if DNS validation is good enough, DANE should've been too. Yes, MITM things could potentially ensue on untrusted networks without DNSSEC, but that's perfect being the enemy of good territory IMO. This would allow folks to have .internal with auto-discovered, decentralized, trusted PKI. It would also enable something like a DNSSEC on/off toggle switch for IoT devices to allow owners to MITM them and provide local functionality for their cloud services.

ndsipa_pomu 2y ago
I hadn't heard of DANE, so looked it up and found the wikipedia entry: https://en.wikipedia.org/wiki/DNS-based_Authentication_of_Named_Entities https://en.wikipedia.org/wiki/DNS-based_Authentication_of_Na... According to that, it's not supported by Chrome, nor Firefox.
- tptacek 2y ago
  It was, once, and then got pulled.
- teddyh 2y ago
  It’s customarily used for e-mail transport.
  - tptacek 2y ago
    No, it isn't; it can't be, because none of the major email domains are DNSSEC-signed to begin with. Let me know if I've misunderstood your point, and there some other widespread niche usage DANE finds in SMTP.
    - teddyh 2y ago
      “Customarily” must not necessarily mean “major email domains”. As I understand it, DANE is commonly used by organizations wanting to secure e-mail between them; DANE enforcement is agreed by both parties, and then used without issue.

Tepix 2y ago

How do you automate it?

globular-toast 2y ago
Not OP but I have a couple of implementations: one using caddyserver[0] as a reverse proxy in a docker-compose set up, and the other is a Kubernetes cluster using cert-manager[1]. [0] https://caddyserver.com/ https://caddyserver.com/ [1] https://cert-manager.io/ https://cert-manager.io/
ndsipa_pomu 2y ago
I use Dynu.com as my DNS provider (they're cheap, provide APIs and very fast to update which is great for home IP addresses that may change). Then, to get the certificates, I use https://github.com/acmesh-official/acme.sh https://github.com/acmesh-official/acme.sh which is a shell script that supports multiple certificate and DNS providers. Copying the certificates to the relevant machines is done by a custom BASH script that runs the relevant acme.sh commands. One advantage of DNS challenge is that it can be run anywhere (i.e. doesn't need to run on the webserver) - it just needs the relevant credentials to add a DNS TXT record. I've got my automation wrapped up into a Docker container.

lolinder 2y ago

Yep. I tried the custom-root-CA approach for a long time, but there were just too many problems with it: * Loading it into every device was more work than it sounds. We have Android, iOS, Mac, Windows, and Linux, all of which have their own rules. * Even once loaded, some applications come with their own set of root CAs. Some of those have a custom way of adding a new one (Firefox), others you just had to accept the invalid cert each time, and still others just refused to work. * I deploy my self-hosted stuff with Docker, which means that not only does each device need to have the root CA added to it but every Docker image that talks to the internal network needs to have it as well. This ends up being a mix of the previous two problems, as I now have to figure out how to mount the CA on an eclectic bunch of distros and I often then have to figure out why the dockerized application isn't using the CA. In the end I settled on a DNS-challenge wildcard SSL cert loaded into Caddy, with Caddy terminating TLS for everything that's on my home server. It's way simpler to configure the single server (or even 2-3 servers) than every single client.

poincaredisk 2y ago
Historically, before wildcard certificates were suddenly available for free, this leaked all internal domains to the internet, but now it's mostly a solved problem.
- vegardx 2y ago
  I don't understand why that is such a huge problem. The alternatives have much more severe problems, all from reusing a wildcard in many places to running your own PKI.
  - NovemberWhiskey 2y ago
    It depends on your risk profile, but there are definitely people who'd rather run their own PKI than permit threat actor reconnaissance by publishing internal hostnames to CT logs.
    - vegardx 2y ago

bombcar 2y ago

Just get a wildcard cert from letsencyrpt and copy it internally. Then you don’t even need to DNS leak names to LE.

giobox 2y ago

LetsEncrypt + DNS challenge + DNS provider with letsencrpyt compatible API for modifying records works fantastically well for getting "real" https/SSL working for private IP addresses, the automatic renewals make it largely set and forget with very little config or setup required. I've had working validly signed SSL on literally all my private home self-hosted services and load-balancers internally for years this way. It also easily switches to a production like setup if you later did decide to host something on the public internet.

thatcherc 2y ago

This sounds like something I'd want to do! Is the idea that you'd have a public domain name like "internal.thatcherc.com" resolve to an internal IP address like 10.0.10.5? I've wondered about setting this up for some local services I have but I wasn't sure if it was a commonly-done thing.

AdamJacobMuller 2y ago
I've been doing this for a year or two with k3s + cert-manager. Works great. In my case everything points to a tailscale operator endpoint, which goes to nginx ingress, which routes to the appropriate pods. It's very much a set-and-forget solution.

wkat4242 2y ago

Yeah that's what I do. If you use anything other than Cloudflare its really really hard to get the authentication plugins going on every different web server though. Every server supports a different subset of providers and usually you have to install the plugins separately. It's a bit of a nightmare. But once it's dialled in it's ok. I didn't like this approach because I don't like to leak information about my internal setup but I found that you don't even have to register your servers on a public DNS so it's ok. Just the domain has to exist. It does create very temporary TXT records though.