4 ms·

> but internal services with have internal CA signed certs with long expire times because of the number of crappy apps that make using certs a pain in the ass.

by xienze 2y ago

> but internal services with have internal CA signed certs with long expire times because of the number of crappy apps that make using certs a pain in the ass. Introduce them to the idea of having something like Caddy sit in front of apps solely for the purpose of doing TLS termination... Caddy et al can update the certs automatically.

pixl97 2y ago
Unless they are web/tech companies they aren't doing that. Banks, finance, large manufacturing are all terminating at F5's and AVI's. I'm pretty sure those update certs just fine, but it's not really what I do these days so I don't have a direct answer.
- xienze 2y ago
  Sure. The point is, don't bother letting the apps themselves do TLS termination. Too much work that's better handled by something else.
  - hedora 2y ago
    Also, moving termination off the endpoint server makes it much easier for three letter agencies to intercept + log.
    - qmarchi 2y ago
      Most responsible orgs do TLS termination on the public side of a connection, but will still make a backend connection protected by TLS, just with a internal CA.
- tikkabhuna 2y ago
  F5s don't support ACME, which has been a pain for us.
  - EvanAnderson 2y ago
    Exactly. According to posters here you should just throw them away and buy hardware from a vendor who does. >sigh< Don't expect firmware / software updates to enable ACME-type functionality for tons of gear. At best it'll be treated as an excuse by vendors to make Customers forklift and replace otherwise working gear. Corporate hardware lifecycles are longer than the proposed timeline for these changes. This feels like an ill thought-out initiative by bureaucrats working in companies who build their own infrastructure (in their white towers). Meanwhile, we plebs who work in less-than-Fortune 500 companies stuck with off-the-shelf solutions will be forced to suffer.
  - cpach 2y ago
    It might be possible to run an ACME client on another host in your environment. (IMHO, the DNS-01 challenge is very useful for this.) Then you can (probably) transfer the cert+key to BIG IP, and activate it, via the REST API. I haven’t used BIG IP in a long while, so take this with a grain of salt, but it seems to me that it might not be impossible to get something going – despite the fact that BIG IP itself doesn’t have native support for ACME. Two pointers that might be of interest: https://community.f5.com/discussions/technicalforum/upload-ltm-ssl-certificate-and-key-using-rest-api/324404 https://community.f5.com/discussions/technicalforum/upload-l... https://clouddocs.f5.com/api/icontrol-rest/APIRef_tm_sys_crypto_cert.html https://clouddocs.f5.com/api/icontrol-rest/APIRef_tm_sys_cry...
    - dijit 2y ago
      Sounds suspiciously similar to a rube goldberg machine. Those tend to be quite brittle in reality. What’s the old adage about engineering vs architecture again? Something like this I think: https://www.reddit.com/r/PeterExplainsTheJoke/comments/16141tk/all_comments_were_in_foreign_language/ https://www.reddit.com/r/PeterExplainsTheJoke/comments/16141...
      - cpach 2y ago
        Obviously it would be much better if BIG IP had native support for ACME. And F5 might implement it some day, but I wouldn’t hold my breath. For some companies, it might be worth it to throw away a $100000 device and buy something better. For others it might not be worth it.
cryptonym 2y ago
You now have to build and self-shot a complete CA/PKI. Or request a certificate over the public internet, for an internal service. Your hostname must be exposed to the web and will be publicly visible in transparency reports.
- stackskipton 2y ago
  You could always ask for wildcard for internal subdomain and use that instead so you will leak your internal FQDN but not individual hosts.
  - pixl97 2y ago
    I'm pretty sure every bank will auto fail wildcard certs these days, at least the ones I've worked with. Key loss on one of those is like a takeover of an entire chunk of hostnames. Really opens you up.
- mox1 2y ago
  Companies have software to manage this for you. We utilize https://www.cyberark.com/products/machine-identity-security/ https://www.cyberark.com/products/machine-identity-security/
- JoshTriplett 2y ago
  > Or request a certificate over the public internet, for an internal service. Your hostname must be exposed to the web and will be publicly visible in transparency reports. That doesn't seem like the end of the world. It means you shouldn't have `secret-plans-for-world-takeover.example.com`, but it's already the case that secret projects should use opaque codenames. Most internal domain names would not actually leak any information of value.

by xienze 2y ago

pixl97 2y ago

Unless they are web/tech companies they aren't doing that. Banks, finance, large manufacturing are all terminating at F5's and AVI's. I'm pretty sure those update certs just fine, but it's not really what I do these days so I don't have a direct answer.

xienze 2y ago
Sure. The point is, don't bother letting the apps themselves do TLS termination. Too much work that's better handled by something else.
- hedora 2y ago
  Also, moving termination off the endpoint server makes it much easier for three letter agencies to intercept + log.
  - qmarchi 2y ago
    Most responsible orgs do TLS termination on the public side of a connection, but will still make a backend connection protected by TLS, just with a internal CA.
tikkabhuna 2y ago
F5s don't support ACME, which has been a pain for us.
- EvanAnderson 2y ago
  Exactly. According to posters here you should just throw them away and buy hardware from a vendor who does. >sigh< Don't expect firmware / software updates to enable ACME-type functionality for tons of gear. At best it'll be treated as an excuse by vendors to make Customers forklift and replace otherwise working gear. Corporate hardware lifecycles are longer than the proposed timeline for these changes. This feels like an ill thought-out initiative by bureaucrats working in companies who build their own infrastructure (in their white towers). Meanwhile, we plebs who work in less-than-Fortune 500 companies stuck with off-the-shelf solutions will be forced to suffer.
- cpach 2y ago
  It might be possible to run an ACME client on another host in your environment. (IMHO, the DNS-01 challenge is very useful for this.) Then you can (probably) transfer the cert+key to BIG IP, and activate it, via the REST API. I haven’t used BIG IP in a long while, so take this with a grain of salt, but it seems to me that it might not be impossible to get something going – despite the fact that BIG IP itself doesn’t have native support for ACME. Two pointers that might be of interest: https://community.f5.com/discussions/technicalforum/upload-ltm-ssl-certificate-and-key-using-rest-api/324404 https://community.f5.com/discussions/technicalforum/upload-l... https://clouddocs.f5.com/api/icontrol-rest/APIRef_tm_sys_crypto_cert.html https://clouddocs.f5.com/api/icontrol-rest/APIRef_tm_sys_cry...
  - dijit 2y ago
    Sounds suspiciously similar to a rube goldberg machine. Those tend to be quite brittle in reality. What’s the old adage about engineering vs architecture again? Something like this I think: https://www.reddit.com/r/PeterExplainsTheJoke/comments/16141tk/all_comments_were_in_foreign_language/ https://www.reddit.com/r/PeterExplainsTheJoke/comments/16141...
    - cpach 2y ago
      Obviously it would be much better if BIG IP had native support for ACME. And F5 might implement it some day, but I wouldn’t hold my breath. For some companies, it might be worth it to throw away a $100000 device and buy something better. For others it might not be worth it.

cryptonym 2y ago

You now have to build and self-shot a complete CA/PKI. Or request a certificate over the public internet, for an internal service. Your hostname must be exposed to the web and will be publicly visible in transparency reports.

stackskipton 2y ago
You could always ask for wildcard for internal subdomain and use that instead so you will leak your internal FQDN but not individual hosts.
- pixl97 2y ago
  I'm pretty sure every bank will auto fail wildcard certs these days, at least the ones I've worked with. Key loss on one of those is like a takeover of an entire chunk of hostnames. Really opens you up.
mox1 2y ago
Companies have software to manage this for you. We utilize https://www.cyberark.com/products/machine-identity-security/ https://www.cyberark.com/products/machine-identity-security/
JoshTriplett 2y ago
> Or request a certificate over the public internet, for an internal service. Your hostname must be exposed to the web and will be publicly visible in transparency reports. That doesn't seem like the end of the world. It means you shouldn't have `secret-plans-for-world-takeover.example.com`, but it's already the case that secret projects should use opaque codenames. Most internal domain names would not actually leak any information of value.