9 ms·

My biggest frustration with .internal is that it requires a private certificate authority. Lots of organizations struggle to fully set up trust for the private

My biggest frustration with .internal is that it requires a private certificate authority. Lots of organizations struggle to fully set up trust for the private CA on all internal systems. When you add BYOD or contractor systems, it's a mess. Using a publicly valid domain offers a number of benefits, like being able to use a free public CA like Lets Encrypt. Every machine will trust your internal certificates out of the box, so there is minimal toil. Last year I built getlocalcert [1] as a free way to automate this approach. It allows you to register a subdomain, publish TXT records for ACME DNS certificate validation, and use your own internal DNS server for all private use. [1] https://www.getlocalcert.net/ https://www.getlocalcert.net/

xer0x 2y ago
Oh neat, thanks for sharing this idea
TheRealPomax 2y ago
I'm pretty sure that if letsencrypt localhost certs work, they'll work fine with .internal too?
- merb 2y ago
  let’s encrypt does not support certain for localhost.
mschuster91 2y ago
> Lots of organizations struggle to fully set up trust for the private CA on all internal systems. Made worse by the fact phone OSes have made it very difficult to install CAs.
- booi 2y ago
  And in on some platforms and configurations, impossible. Same with the .dev domain
  - jhardy54 2y ago
    .dev isn’t a TLD for internal use though, do you have the same problem when you use .test?
    - dijit 2y ago
      gonna go ahead and cast shade at Google because of how they handled that. Their original application for .dev was written to "ensure its reserved use for internal projects - since it is a common internal TLD for development" - then once granted a few years later they started selling domains with it. ** WITH HSTS PRELOADING ** ensuring that all those internal dev sites they were aware of would break.
  - kortilla 2y ago
    .dev is a real domain
yjftsjthsd-h 2y ago
Do you mean to say that your biggest frustration with HTTPS on .internal is that it requires a private certificate authority? Because I'm running plain HTTP to .internal sites and it works fine.
- lysace 2y ago
  There's some every packet shall be encrypted, even in minimal private VPCs lore going on. I'm blaming PCI-DSS.
  - bruce511 2y ago
    The big problem with running unencrypted HTTP on a LAN is that it's terribly easy for (most) LANs to be compromised. Let's start with the obvious; wifi. If you're visiting a company and ask the receptionist for the wifi password you'll likely get it. Next are eternity ports. Sitting waiting in a meeting room, plug your laptop into the ethernet port and you're in. And of course it's not just hardware, any software running on any machine makes the LAN just as vulnerable. Sure, you can design a LAN to be secure. You can make sure there's no way to get onto it. But the -developer- and -network maintainer- are 2 different guys, or more likely different departments. As a developer are you convinced the LAN will be as secure in 10 years as it is today? 5 years? 1 year after that new intern arrives and takes over maintainence 6 weeks in? What starts out as "minimal private VPC" grows, changes, is fluid. Treating it as secure today is one thing. Trusting it to remain secure 10 years from now is another. In 99.9% of cases your LAN traffic should be secure. This us the message -developers- need to hear. Don't rely on some other department to secure your system. Do it yourself.
    - gorgoiler 2y ago
      Well said. I used to be of the mindset that if I ran VLANs I could at least segregate the good guys from the evil AliExpress wifi connected toasters. Now everything feels like it could become hostile at any moment and so, on that basis, we all share the same network with shields up as if it were the plain, scary Internet. It feels a lot safer. I guess my toaster is going to hack my printer someday, but at least it won’t get into my properly-secured laptop that makes no assumptions the local network is “safe”.
wkat4242 2y ago
The problem with internal CAs is also that it's really hard to add them on some OSes now. Especially on android since version 7 IIRC, you can no longer get certs into the system store, and every app is free to ignore the user store (I think it was even the default to ignore it). So a lot of apps will not work with it.
- Terr_ 2y ago
  Speculating a bit out of my depth here, but I'm under the impression that most of those sometimes-configurable OS-level CA lists are treated as "trust anything consistent with this data", as opposed to "only trust this CA record for these specific domain-patterns because that's the narrow purpose I chose to install it for." So there are a bunch of cases where we only want the second (simpler, lower-risk) case, but we have to incur all the annoyance and risk and locked-down-ness of the first use-case.
  - 8organicbits 2y ago
    Yes! Context specific CA trust would be great, but AFAIK isn't possible yet. Even name constraints, which are domain name limitations a CA or intermediate cert place on itself, are slowly being supported by relevant software [1]. As a contractor, I'll create a per-client VM for each contract and install any client network CAs only within that VM. [1] https://alexsci.com/blog/name-non-constraint/ https://alexsci.com/blog/name-non-constraint/
- thaumasiotes 2y ago
  > The problem with internal CAs is also that it's really hard to add them on some OSes now. Especially on android since version 7 IIRC That's because the purpose of certificate pinning is to protect software from the user. Letting you supply your own certificates would defeat the purpose of having them.
  - okanat 2y ago
    Protect the software from the user? Why are you giving them the software then?
jacooper 2y ago
This is why I'm using a FQDN for my home lab, I'm not going to setup a private CA for this, I can just use ACME-dns and get a cert that will work everywhere, for free!
prussian 2y ago
Just be mindful that any certs you issue in this way will be public information[1] so make sure the domain names don't give away any interesting facts about your infrastructure or future product ideas. I did this at my last job as well and I can still see them renewing them, including an unfortunate wildcard cert which wasn't me. [1] https://crt.sh/ https://crt.sh/
- Helmut10001 2y ago
  Just use wildcard certs and internal subdomains remain internal information.
  - qmarchi 2y ago
    There's a larger risk that if someone breaches a system with a wildcard cert, then you can end up with them being able to impersonate _every_ part of your domain, not just the one application.
    - politelemon 2y ago
      It's the opposite - there is a risk, but not a larger risk. Environment traversal is easier through a certificate transparency log, there is almost zero work to do. Through a wildcard compromise, the environment is not immediately visible. It's much safer to do wildcard for certs for internal use.
      - ixfo 2y ago
        Environment visibility is easy to get. If you pwn a box which has foo.internal, you can now impersonate foo.internal. If you pwn a box which has *.internal, you can now impersonate super-secret.internal and everything else, and now you're a DNS change away from MITM across an entire estate. Security by obscurity while making the actual security of endpoints weaker is not an argument in favour of wildcards...
    - eru 2y ago
      Can't you have a limited wildcard? Something like *.for-testing-only.company.com?
seb1204 2y ago
https://letsencrypt.org/ https://letsencrypt.org/ does not work?
- francislavoie 2y ago
  No, that's a public CA. No public domain registrars will be allowed to sell .internal domains so no public DNS servers will resolve .internal and that's a requirement for let's encrypt to validate that you control the domain. So you must use a private CA (one that you create yourself, with something like Smallstep, Caddy, or OpenSSL commands) and you'll need to install that CA's root certificate on any devices you want to be able to connect to your server(s) that use .internal
AndyMcConachie 2y ago
If you read the document that originally lead the ICANN Board to reserve .INTERNAL (SAC113) you will find this exact sentiment. The SSAC's recommendation is to only use .INTERNAL if using a publicly registered domain name is not an option. See Section 4.2. https://itp.cdn.icann.org/en/files/security-and-stability-advisory-committee-ssac-reports/sac-113-en.pdf https://itp.cdn.icann.org/en/files/security-and-stability-ad...
7bit 2y ago
> My biggest frustration with .internal is that it requires a private certificate authority So don't use it?
layer8 2y ago
I don’t understand the frustration. The use of .internal is explicitly for when you don’t want a publicly valid domain. Nobody is forcing anyone to use .internal otherwise.
- pas 2y ago
  the frustration comes when non-corporate-provisoned clients get on the .internal network and have trouble using the services because of TLS errors (or the problem is lack of TLS) and the recommendation is to simply do "*.internal.example.com" with LetsEncrypt (using DNS-01 validation), so every client gets the correct CA cert "for free" ... obviously if you want mTLS, then this doesn't help much. (but still, it's true that using a public domain has many advantages, as having an airgapped network too)
  - layer8 2y ago
    You’re basically saying that .internal can cause frustration when it is used without good reason. Fair enough, but also not surprising. When it is used for the intended reasons though, then there’s just no other solution. It’s a trade-off between conflicting goals. “Simply do X instead” doesn’t remove the trade-off.
    - nightpool 2y ago
      What do you see as the intended reasons with no other solutions?
      - 8organicbits 2y ago
        The biggest benefit of .internal IMO is that it is free to use. Free domains used to be a thing, but after the fall of Freenom you're stuck with free subdomains.
        powersnail 2y ago
        If `.internal` is for private-use only, they must be resolved by some sort of private or internal DNS. In that case, all domains are free for private-use anyway.
derefr 2y ago
It would be impossible for .internal domains to be publicly CAed, because they're non-unique; the whole point of .internal domains is that, just like private-use IP space, anyone can reuse the same .internal DNS names within their own organization. X.509 trust just doesn't work if multiple entities can get a cert for the same CN under the same root-of-trust, as then one of the issuees can impersonate the other. If public issuers would sign .internal certs, then presuming you have access to a random org's intranet, you could MITM any machine in that org by first setting up your own intranet with its own DNS, creating .internal records in it, getting a public issuer to issue certs for those domains, and then using those certs to impersonate the .internal servers in the org-intranet you're trying to attack.

xer0x 2y ago

Oh neat, thanks for sharing this idea

TheRealPomax 2y ago

I'm pretty sure that if letsencrypt localhost certs work, they'll work fine with .internal too?

merb 2y ago
let’s encrypt does not support certain for localhost.

mschuster91 2y ago

> Lots of organizations struggle to fully set up trust for the private CA on all internal systems. Made worse by the fact phone OSes have made it very difficult to install CAs.

booi 2y ago
And in on some platforms and configurations, impossible. Same with the .dev domain
- jhardy54 2y ago
  .dev isn’t a TLD for internal use though, do you have the same problem when you use .test?
  - dijit 2y ago
    gonna go ahead and cast shade at Google because of how they handled that. Their original application for .dev was written to "ensure its reserved use for internal projects - since it is a common internal TLD for development" - then once granted a few years later they started selling domains with it. ** WITH HSTS PRELOADING ** ensuring that all those internal dev sites they were aware of would break.
- kortilla 2y ago
  .dev is a real domain

yjftsjthsd-h 2y ago

Do you mean to say that your biggest frustration with HTTPS on .internal is that it requires a private certificate authority? Because I'm running plain HTTP to .internal sites and it works fine.

lysace 2y ago
There's some every packet shall be encrypted, even in minimal private VPCs lore going on. I'm blaming PCI-DSS.
- bruce511 2y ago
  The big problem with running unencrypted HTTP on a LAN is that it's terribly easy for (most) LANs to be compromised. Let's start with the obvious; wifi. If you're visiting a company and ask the receptionist for the wifi password you'll likely get it. Next are eternity ports. Sitting waiting in a meeting room, plug your laptop into the ethernet port and you're in. And of course it's not just hardware, any software running on any machine makes the LAN just as vulnerable. Sure, you can design a LAN to be secure. You can make sure there's no way to get onto it. But the -developer- and -network maintainer- are 2 different guys, or more likely different departments. As a developer are you convinced the LAN will be as secure in 10 years as it is today? 5 years? 1 year after that new intern arrives and takes over maintainence 6 weeks in? What starts out as "minimal private VPC" grows, changes, is fluid. Treating it as secure today is one thing. Trusting it to remain secure 10 years from now is another. In 99.9% of cases your LAN traffic should be secure. This us the message -developers- need to hear. Don't rely on some other department to secure your system. Do it yourself.
  - gorgoiler 2y ago
    Well said. I used to be of the mindset that if I ran VLANs I could at least segregate the good guys from the evil AliExpress wifi connected toasters. Now everything feels like it could become hostile at any moment and so, on that basis, we all share the same network with shields up as if it were the plain, scary Internet. It feels a lot safer. I guess my toaster is going to hack my printer someday, but at least it won’t get into my properly-secured laptop that makes no assumptions the local network is “safe”.

wkat4242 2y ago

The problem with internal CAs is also that it's really hard to add them on some OSes now. Especially on android since version 7 IIRC, you can no longer get certs into the system store, and every app is free to ignore the user store (I think it was even the default to ignore it). So a lot of apps will not work with it.

Terr_ 2y ago
Speculating a bit out of my depth here, but I'm under the impression that most of those sometimes-configurable OS-level CA lists are treated as "trust anything consistent with this data", as opposed to "only trust this CA record for these specific domain-patterns because that's the narrow purpose I chose to install it for." So there are a bunch of cases where we only want the second (simpler, lower-risk) case, but we have to incur all the annoyance and risk and locked-down-ness of the first use-case.
- 8organicbits 2y ago
  Yes! Context specific CA trust would be great, but AFAIK isn't possible yet. Even name constraints, which are domain name limitations a CA or intermediate cert place on itself, are slowly being supported by relevant software [1]. As a contractor, I'll create a per-client VM for each contract and install any client network CAs only within that VM. [1] https://alexsci.com/blog/name-non-constraint/ https://alexsci.com/blog/name-non-constraint/
thaumasiotes 2y ago
> The problem with internal CAs is also that it's really hard to add them on some OSes now. Especially on android since version 7 IIRC That's because the purpose of certificate pinning is to protect software from the user. Letting you supply your own certificates would defeat the purpose of having them.
- okanat 2y ago
  Protect the software from the user? Why are you giving them the software then?

jacooper 2y ago

This is why I'm using a FQDN for my home lab, I'm not going to setup a private CA for this, I can just use ACME-dns and get a cert that will work everywhere, for free!

prussian 2y ago

Just be mindful that any certs you issue in this way will be public information[1] so make sure the domain names don't give away any interesting facts about your infrastructure or future product ideas. I did this at my last job as well and I can still see them renewing them, including an unfortunate wildcard cert which wasn't me. [1] https://crt.sh/ https://crt.sh/

Helmut10001 2y ago
Just use wildcard certs and internal subdomains remain internal information.
- qmarchi 2y ago
  There's a larger risk that if someone breaches a system with a wildcard cert, then you can end up with them being able to impersonate _every_ part of your domain, not just the one application.
  - politelemon 2y ago
    It's the opposite - there is a risk, but not a larger risk. Environment traversal is easier through a certificate transparency log, there is almost zero work to do. Through a wildcard compromise, the environment is not immediately visible. It's much safer to do wildcard for certs for internal use.
    - ixfo 2y ago
      Environment visibility is easy to get. If you pwn a box which has foo.internal, you can now impersonate foo.internal. If you pwn a box which has *.internal, you can now impersonate super-secret.internal and everything else, and now you're a DNS change away from MITM across an entire estate. Security by obscurity while making the actual security of endpoints weaker is not an argument in favour of wildcards...
  - eru 2y ago
    Can't you have a limited wildcard? Something like *.for-testing-only.company.com?

seb1204 2y ago

https://letsencrypt.org/ https://letsencrypt.org/ does not work?

francislavoie 2y ago
No, that's a public CA. No public domain registrars will be allowed to sell .internal domains so no public DNS servers will resolve .internal and that's a requirement for let's encrypt to validate that you control the domain. So you must use a private CA (one that you create yourself, with something like Smallstep, Caddy, or OpenSSL commands) and you'll need to install that CA's root certificate on any devices you want to be able to connect to your server(s) that use .internal

AndyMcConachie 2y ago

If you read the document that originally lead the ICANN Board to reserve .INTERNAL (SAC113) you will find this exact sentiment. The SSAC's recommendation is to only use .INTERNAL if using a publicly registered domain name is not an option. See Section 4.2. https://itp.cdn.icann.org/en/files/security-and-stability-advisory-committee-ssac-reports/sac-113-en.pdf https://itp.cdn.icann.org/en/files/security-and-stability-ad...

7bit 2y ago

> My biggest frustration with .internal is that it requires a private certificate authority So don't use it?

layer8 2y ago

I don’t understand the frustration. The use of .internal is explicitly for when you don’t want a publicly valid domain. Nobody is forcing anyone to use .internal otherwise.

pas 2y ago
the frustration comes when non-corporate-provisoned clients get on the .internal network and have trouble using the services because of TLS errors (or the problem is lack of TLS) and the recommendation is to simply do "*.internal.example.com" with LetsEncrypt (using DNS-01 validation), so every client gets the correct CA cert "for free" ... obviously if you want mTLS, then this doesn't help much. (but still, it's true that using a public domain has many advantages, as having an airgapped network too)
- layer8 2y ago
  You’re basically saying that .internal can cause frustration when it is used without good reason. Fair enough, but also not surprising. When it is used for the intended reasons though, then there’s just no other solution. It’s a trade-off between conflicting goals. “Simply do X instead” doesn’t remove the trade-off.
  - nightpool 2y ago
    What do you see as the intended reasons with no other solutions?
    - 8organicbits 2y ago
      The biggest benefit of .internal IMO is that it is free to use. Free domains used to be a thing, but after the fall of Freenom you're stuck with free subdomains.
      - powersnail 2y ago
        If `.internal` is for private-use only, they must be resolved by some sort of private or internal DNS. In that case, all domains are free for private-use anyway.

derefr 2y ago

It would be impossible for .internal domains to be publicly CAed, because they're non-unique; the whole point of .internal domains is that, just like private-use IP space, anyone can reuse the same .internal DNS names within their own organization. X.509 trust just doesn't work if multiple entities can get a cert for the same CN under the same root-of-trust, as then one of the issuees can impersonate the other. If public issuers would sign .internal certs, then presuming you have access to a random org's intranet, you could MITM any machine in that org by first setting up your own intranet with its own DNS, creating .internal records in it, getting a public issuer to issue certs for those domains, and then using those certs to impersonate the .internal servers in the org-intranet you're trying to attack.