6 ms·

I'm all for it -- it's hard to understate the extent to which LetsEncrypt has improved the WebPKI situation. Although the effective single-vendor situation isn'

I'm all for it -- it's hard to understate the extent to which LetsEncrypt has improved the WebPKI situation. Although the effective single-vendor situation isn't great, the "this is just something you only do via an automated API" approach is absolutely the right one. And certificate lifetimes measured in days work just fine with that. The only things that continue to amaze me are the number of (mostly "enterprise") software products that simply won't get with the times (or get it wrong, like renewing the cert, but continuing to use the old one until something is manually restarted), and the countless IT departments that still don't support any kind of API for their internal domains...

schmuckonwheels 10mo ago
> The only things that continue to amaze me are the number of (mostly "enterprise") software products that simply won't get with the times Yeah, no one's rewriting a bunch of software to support automating a specific, internet-facing, sometimes-reliable CA. Yes it's ACME, a standard you say. A standard protocol with nonstop changing profile requirements at LE's whim. Who's going to keep updating the software every 3 months to keep up? When the WebPKI sneeze in a different direction and change their minds yet again. Because 45 will become 30 will become 7 and they won't stop till the lifetime is 6 hours. "Enterprise" products are more often than not using internal PKI so it's a waste. I would like to see the metrics on how much time and resources are wasted babysitting all this automation vs. going in and updating a certificate manually once a year and not having to worry the automation will fail in a week.
- ZeroConcerns 10mo ago
  > I would like to see the metrics Well, I could regale you with my anecdotes on how all my 'grab a LetsEncrypt cert on Docker deploy and whenever remaining lifetime goes below 75%' services have not had a single TLS-related outage in at least a year, and how we just had a multi-day meltdown caused by a load-bearing cert that everyone forgot about expiring, but I doubt you'll be interested. I'm not here to take away your right to set up a few more 5-year+ internal CAs (or, hey, how about an entire self-signed hierarchy? can't be that hard...), and neither is LetsEncrypt. But on the Big Bad Internet (where Google is the leading cause of More Security, and LetsEncrypt is merely the leading vendor catering to that), things have moved on slightly.
- happymellon 10mo ago
  > I would like to see the metrics on how much time and resources are wasted babysitting all this automation vs. going in and updating a certificate manually once a year and not having to worry the automation will fail in a week. I have multiple systems. Legacy that I've inherited, and modern that are able to automatically update their certs. The automated updates require almost zero maintenance, there was a week a couple of years ago when I had to switch some configuration for a new root cert. The legacy system requires manual installation and multiple weeks of my time every year because of the number of environments, and since generation of the certs requires me to file a request for someone to manually create it, they invariably typo something and it has to be redone everywhere. So multiple engineers, over multiple weeks? Manual process at a guess is £50k pa, while the automated is close to an annual £0?
- crote 10mo ago
  The change in validity does not in any way alter the protocol itself. As mentioned in the linked blog post: if you've already got automated cert renewal, it'll almost certainly require zero change. After all, the logical approach is to schedule a daily cron job for your renewal process, which only contacts the server when the current cert has less than X days of validity remaining. Scheduling a one-off cron job every, say, 60 days would be extremely error-prone! With regards to future reductions: the point is to force companies to automate renewal, as they have proven to be unable to renew in time during incidents due to dozens of layers of bureaucracy. 45 days means one renewal a month, which is probably sufficient for that. Anything below 7 days isn't likely to happen, because at that point the industry considers certification revocation unnecessary. Internal PKI is not relevant here. Anyone with weird internal cert use shouldn't be using public certs anyways, so moving those people to certs backed by self-signed internal CAs is a win. They are free to use whatever validity they want with those.
  - basscomm 10mo ago
    > the point is to force companies to automate renewal Cool. I'm a small-time webmaster with a couple of hobby sites with no more than a handful of visitors. Why do I need to set up automation to renew certs every 45 days, too?
    - nickf 10mo ago
      Is the certificate you use on your website any different to that on google.com? Does/could a browser know this and act differently?
    - Shish2k 10mo ago
      I'm a small time webmaster and I haven't "set up" any automation - for my shared-hosting sites, the host has it built in; and for my self-hosted sites, the web server has it built in
      - iggldiggl 10mo ago
        The problem is that this breaks down if you don't want to leak any obscure subdomains you might be using via CT-logs – shared hosting rarely supports DNS-based certificate renewals for wildcard certificates, and even less so for domains hosted by an external registrar. (Even for a fully self-hosted system you'd still have to figure out how to interface the certificate renewal mechanism with your DNS provider, so not as easy to set up as individual certificates for each subdomain.)
- homebrewer 10mo ago
  In circles I'm running in, automatic certificate renewal has not caused a single problem over 7 years of using it, and whatever time was spent on setting it up, has paid many times over, both in saving effort on renewal, and in putting out fires when (not if) someone forgets to renew a certificate. You just have to be careful picking your automation — I haven't been impressed with certbot, for example. Also, everything is using https now. Living in a low-income country, certificates were too expensive to use them where they weren't absolutely required, but not anymore. This is orthogonal to automation, I'm just pointing out that LE is not as demonic as you make it out to be. I'm afraid enterprise users are on their own, probably approximately no-one else is interested in going back to the old ways of doing it. (Maybe embedded.)
  - imtringued 10mo ago
    Forcing automation would be fine if the default software package (certbot) was any good but from my experience certbot is simply not fit for purpose. Certbot doesn't support the industry standard PKCS#12 format, which makes it extremely brittle for anyone using a Java based webserver. Instead it uses the non-standard PEM format which requires conversion before usage. That conversion step breaks all the time and requires manual intervention. It's ridiculous.
    - arccy 10mo ago
      at this point PEM is more standard and prevalent than pkcs#12
    - nine_k 10mo ago
      PEM is very standard. Calling `openssl pkcs12` also should not be hard; IDK about certbot, but there is a hook for acmetool (which I use) that does just that for you: https://github.com/dlitz/acmetool-pkcs12-hooks https://github.com/dlitz/acmetool-pkcs12-hooks
    - cpach 10mo ago
      I hear ya. I’m also not fond of certbot and other existing clients. The best solution I’ve found so far was to implement a custom cert manager using the formidable acmez library.
- arp242 10mo ago
  > A standard protocol with nonstop changing profile requirements at LE's whim. Who's going to keep updating the software every 3 months to keep up? It really doesn't change that often. And whether this is a "breaking" change is something that's debatable (you shouldn't hard-code the cert lifetime, but I suspect many programs do, so de-facto it may or may not be a breaking change). If you look at the Go implementation for example (https://github.com/golang/crypto/commits/master/acme https://github.com/golang/crypto/commits/master/acme), then there haven't been any major changes since it was first written in 2018: just bugfixes and basic maintenance tasks. Maybe I missed a commit but it certainly doesn't require 3-month update cycles. You can probably take that 2018 code and it'll work fine today. And since it doesn't hard-code the 90 days it will continue to work in 2028 when this change is made. So it's more "you need to keep updating the software every decade to keep up".
- ForHackernews 10mo ago
  I'll happily take failed automation with errors I can fix over being totally screwed when we find out the guy who did it every year left the company this year and nobody else knows anything about it.
- patmorgan23 10mo ago
  All the enterprise software needs to do is create an API for configuring the certificate in the product. Then they can integrate Certbot or one of the many other ACME solutions, or let the customer do it (they are enterprises after all).
- rainsford 10mo ago
  > I would like to see the metrics on how much time and resources are wasted babysitting all this automation vs. going in and updating a certificate manually once a year and not having to worry the automation will fail in a week. I would also like to see those metrics, because I strongly suspect the cost is dramatically in favor of automation, especially when you consider the improved security posture you get from short lived certificates. My personal experience with Let's Encrypt has been that the only hassle is in systems that don't implement automation, so I get the worst of both worlds having to effectively manually renew short lived certificates every 90 days. CRL based revocation is largely a failure, both because implementation is a problem and because knowing when to revoke a certificate is hard. So the only way to achieve the goal of security in WebPKI that's resilient to private key compromise is short-lived certificates. And the only way to implement those is with automated renewal.
crote 10mo ago
It's not single-vendor. The ACME protocol is also supported by the likes of GlobalSign, Sectigo, and Digicert. You've got to remember that the reduction to a 45-day duration is industry-wide - driven by the browsers. Any CA not offering automated renewal (which in practice means ACME) is going to lose a lot of customers over the next few years.
- ZeroConcerns 10mo ago
  Effectively single-vendor. I'm not aware of any ACME-compatible CAs that don't have pernicious limits on their free plans (and if there are, I'd love to hear!), and here in the EU we've even recently lost a rather big player...
  - dmurray 10mo ago
    "multiple vendors, but only one of them is nice enough to give the product away for free" is not "effectively single-vendor". The other CAs aren't prohibitively priced for anyone who has a business need for lots of certificates, in case Let's Encrypt disappears or goes rogue.
    - ZeroConcerns 10mo ago
      > other CAs aren't prohibitively priced Not if you look at the per-cert pricing, but if you factor in the cost of "dealing with incompetent sales" and "convincing accounting to keep the contract going", they absolutely are.
      - toast0 10mo ago
        When I was working with Digicert a decade ago, it was expensive, but they also had knowledgable support and with a wildcard cert, they would issue all sorts of 'custom duplicates' by request that were super handy. No incompetent sales, but certainly you do need to make sure accounting will pay.
        10mo ago
riffic 10mo ago
There's a slew of RFC documents that cover these related protocols so imagine that now means "requests for compliance".

schmuckonwheels 10mo ago

> The only things that continue to amaze me are the number of (mostly "enterprise") software products that simply won't get with the times Yeah, no one's rewriting a bunch of software to support automating a specific, internet-facing, sometimes-reliable CA. Yes it's ACME, a standard you say. A standard protocol with nonstop changing profile requirements at LE's whim. Who's going to keep updating the software every 3 months to keep up? When the WebPKI sneeze in a different direction and change their minds yet again. Because 45 will become 30 will become 7 and they won't stop till the lifetime is 6 hours. "Enterprise" products are more often than not using internal PKI so it's a waste. I would like to see the metrics on how much time and resources are wasted babysitting all this automation vs. going in and updating a certificate manually once a year and not having to worry the automation will fail in a week.

ZeroConcerns 10mo ago
> I would like to see the metrics Well, I could regale you with my anecdotes on how all my 'grab a LetsEncrypt cert on Docker deploy and whenever remaining lifetime goes below 75%' services have not had a single TLS-related outage in at least a year, and how we just had a multi-day meltdown caused by a load-bearing cert that everyone forgot about expiring, but I doubt you'll be interested. I'm not here to take away your right to set up a few more 5-year+ internal CAs (or, hey, how about an entire self-signed hierarchy? can't be that hard...), and neither is LetsEncrypt. But on the Big Bad Internet (where Google is the leading cause of More Security, and LetsEncrypt is merely the leading vendor catering to that), things have moved on slightly.
happymellon 10mo ago
> I would like to see the metrics on how much time and resources are wasted babysitting all this automation vs. going in and updating a certificate manually once a year and not having to worry the automation will fail in a week. I have multiple systems. Legacy that I've inherited, and modern that are able to automatically update their certs. The automated updates require almost zero maintenance, there was a week a couple of years ago when I had to switch some configuration for a new root cert. The legacy system requires manual installation and multiple weeks of my time every year because of the number of environments, and since generation of the certs requires me to file a request for someone to manually create it, they invariably typo something and it has to be redone everywhere. So multiple engineers, over multiple weeks? Manual process at a guess is £50k pa, while the automated is close to an annual £0?
crote 10mo ago
The change in validity does not in any way alter the protocol itself. As mentioned in the linked blog post: if you've already got automated cert renewal, it'll almost certainly require zero change. After all, the logical approach is to schedule a daily cron job for your renewal process, which only contacts the server when the current cert has less than X days of validity remaining. Scheduling a one-off cron job every, say, 60 days would be extremely error-prone! With regards to future reductions: the point is to force companies to automate renewal, as they have proven to be unable to renew in time during incidents due to dozens of layers of bureaucracy. 45 days means one renewal a month, which is probably sufficient for that. Anything below 7 days isn't likely to happen, because at that point the industry considers certification revocation unnecessary. Internal PKI is not relevant here. Anyone with weird internal cert use shouldn't be using public certs anyways, so moving those people to certs backed by self-signed internal CAs is a win. They are free to use whatever validity they want with those.
- basscomm 10mo ago
  > the point is to force companies to automate renewal Cool. I'm a small-time webmaster with a couple of hobby sites with no more than a handful of visitors. Why do I need to set up automation to renew certs every 45 days, too?
  - nickf 10mo ago
    Is the certificate you use on your website any different to that on google.com? Does/could a browser know this and act differently?
  - Shish2k 10mo ago
    I'm a small time webmaster and I haven't "set up" any automation - for my shared-hosting sites, the host has it built in; and for my self-hosted sites, the web server has it built in
    - iggldiggl 10mo ago
      The problem is that this breaks down if you don't want to leak any obscure subdomains you might be using via CT-logs – shared hosting rarely supports DNS-based certificate renewals for wildcard certificates, and even less so for domains hosted by an external registrar. (Even for a fully self-hosted system you'd still have to figure out how to interface the certificate renewal mechanism with your DNS provider, so not as easy to set up as individual certificates for each subdomain.)
homebrewer 10mo ago
In circles I'm running in, automatic certificate renewal has not caused a single problem over 7 years of using it, and whatever time was spent on setting it up, has paid many times over, both in saving effort on renewal, and in putting out fires when (not if) someone forgets to renew a certificate. You just have to be careful picking your automation — I haven't been impressed with certbot, for example. Also, everything is using https now. Living in a low-income country, certificates were too expensive to use them where they weren't absolutely required, but not anymore. This is orthogonal to automation, I'm just pointing out that LE is not as demonic as you make it out to be. I'm afraid enterprise users are on their own, probably approximately no-one else is interested in going back to the old ways of doing it. (Maybe embedded.)
- imtringued 10mo ago
  Forcing automation would be fine if the default software package (certbot) was any good but from my experience certbot is simply not fit for purpose. Certbot doesn't support the industry standard PKCS#12 format, which makes it extremely brittle for anyone using a Java based webserver. Instead it uses the non-standard PEM format which requires conversion before usage. That conversion step breaks all the time and requires manual intervention. It's ridiculous.
  - arccy 10mo ago
    at this point PEM is more standard and prevalent than pkcs#12
  - nine_k 10mo ago
    PEM is very standard. Calling `openssl pkcs12` also should not be hard; IDK about certbot, but there is a hook for acmetool (which I use) that does just that for you: https://github.com/dlitz/acmetool-pkcs12-hooks https://github.com/dlitz/acmetool-pkcs12-hooks
  - cpach 10mo ago
    I hear ya. I’m also not fond of certbot and other existing clients. The best solution I’ve found so far was to implement a custom cert manager using the formidable acmez library.
arp242 10mo ago
> A standard protocol with nonstop changing profile requirements at LE's whim. Who's going to keep updating the software every 3 months to keep up? It really doesn't change that often. And whether this is a "breaking" change is something that's debatable (you shouldn't hard-code the cert lifetime, but I suspect many programs do, so de-facto it may or may not be a breaking change). If you look at the Go implementation for example (https://github.com/golang/crypto/commits/master/acme https://github.com/golang/crypto/commits/master/acme), then there haven't been any major changes since it was first written in 2018: just bugfixes and basic maintenance tasks. Maybe I missed a commit but it certainly doesn't require 3-month update cycles. You can probably take that 2018 code and it'll work fine today. And since it doesn't hard-code the 90 days it will continue to work in 2028 when this change is made. So it's more "you need to keep updating the software every decade to keep up".
ForHackernews 10mo ago
I'll happily take failed automation with errors I can fix over being totally screwed when we find out the guy who did it every year left the company this year and nobody else knows anything about it.
patmorgan23 10mo ago
All the enterprise software needs to do is create an API for configuring the certificate in the product. Then they can integrate Certbot or one of the many other ACME solutions, or let the customer do it (they are enterprises after all).
rainsford 10mo ago
> I would like to see the metrics on how much time and resources are wasted babysitting all this automation vs. going in and updating a certificate manually once a year and not having to worry the automation will fail in a week. I would also like to see those metrics, because I strongly suspect the cost is dramatically in favor of automation, especially when you consider the improved security posture you get from short lived certificates. My personal experience with Let's Encrypt has been that the only hassle is in systems that don't implement automation, so I get the worst of both worlds having to effectively manually renew short lived certificates every 90 days. CRL based revocation is largely a failure, both because implementation is a problem and because knowing when to revoke a certificate is hard. So the only way to achieve the goal of security in WebPKI that's resilient to private key compromise is short-lived certificates. And the only way to implement those is with automated renewal.

crote 10mo ago

It's not single-vendor. The ACME protocol is also supported by the likes of GlobalSign, Sectigo, and Digicert. You've got to remember that the reduction to a 45-day duration is industry-wide - driven by the browsers. Any CA not offering automated renewal (which in practice means ACME) is going to lose a lot of customers over the next few years.

ZeroConcerns 10mo ago
Effectively single-vendor. I'm not aware of any ACME-compatible CAs that don't have pernicious limits on their free plans (and if there are, I'd love to hear!), and here in the EU we've even recently lost a rather big player...
- dmurray 10mo ago
  "multiple vendors, but only one of them is nice enough to give the product away for free" is not "effectively single-vendor". The other CAs aren't prohibitively priced for anyone who has a business need for lots of certificates, in case Let's Encrypt disappears or goes rogue.
  - ZeroConcerns 10mo ago
    > other CAs aren't prohibitively priced Not if you look at the per-cert pricing, but if you factor in the cost of "dealing with incompetent sales" and "convincing accounting to keep the contract going", they absolutely are.
    - toast0 10mo ago
      When I was working with Digicert a decade ago, it was expensive, but they also had knowledgable support and with a wildcard cert, they would issue all sorts of 'custom duplicates' by request that were super handy. No incompetent sales, but certainly you do need to make sure accounting will pay.
      - 10mo ago

riffic 10mo ago

There's a slew of RFC documents that cover these related protocols so imagine that now means "requests for compliance".