4 ms·

This idea we seem to have moved towards where every applications ALSO includes their own ACME support really annoys me actually. I much prefer the idea that th

This idea we seem to have moved towards where every applications ALSO includes their own ACME support really annoys me actually. I much prefer the idea that there's well written clients who's job it is to do the ACME handling. Is my Postfix mailserver soon going to have an ACME shoehorned in? I've already seen GitHub issues for AdGuardHome (a DNS server that supports blocklists) to have an ACME client built in, thankfully thus far ignored. Proxmox (a VM Hypervisor!) has an ACME Client built in. I realise of course the inclusion of an ACME client in a product doesn't mean I need to use their implementation, I'm free to keep using my own independant client. But it seems to me adding ACME clients to everything is going to cause those projects more PRs, more baggage to drag forward etc. And confusion for users as now there's multiple places they could/should be generating certificates. Anyway, grumpy old man rant over. It just seems Zawinski's Law "Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can." can be replaced these days with MuppetMan's law of "Every program attempts to expand until it can issue ACME certificates."

atomicnumber3 1y ago
I personally think nginx is the kind of project I'd allow to have its own acme client. It's extremely extremely widely used software and I would be surprised if less than 50% of the certs LE issues are not exclusively served via nginx. Now if Jenkins adds acme support then yes I'll say maybe that one is too far.
- muppetman 1y ago
  But it's a webserver. I'm sure it farms out sending emails from forms it serves, I doubt it has a PHP library built in, surely it farms that out to php-fpm? It doesn't have a REDIS library or NodeJS built in. Why's ACME different?
  - tuckerman 1y ago
    I get what you are saying but surely obtaining a certificate is much closer to being considered a core part of a web server related to transport, especially in 2025 when browsers throw up "doesn’t support a secure connection with HTTPS" messages left and right, than those other examples. I think there is also clearly demand: caddy is very well liked and often recommended for hobbyists and I think a huge part of that is the built in certificate management.
  - firesteelrain 1y ago
    To your point, we use Venafi and it has clients that act as orchestrators to deploy the new cert and restart the web service. Webservice itself doesn’t need to be ACME aware. Venafi supports ACME protocol so it can be the ACME server like Let’s Encrypt I am speaking purely on prem non internet connect scenario
  - andmarios 1y ago
    Nginx (and Apache, etc) is not just a web server; it is also a reverse proxy, a TLS termination proxy, a load balancer, etc. The key service here is "TLS termination proxy", so being able to issue certificates automatically was pretty high on the wish list.
  - dividuum 1y ago
    Well, it already has, among a ton of other modules, a memcached and a JavaScript module (njs), so you’re actually not that far off. An optional ACME module sounds fitting.
  - banashark 1y ago
    Well you say that.... https://openresty.org/en/ https://openresty.org/en/ "Real-world applications of OpenResty® range from dynamic web portals and web gateways, web application firewalls, web service platforms for mobile apps/advertising/distributed storage/data analytics, to full-fledged dynamic web applications and web sites. The hardware used to run OpenResty® also ranges from very big metals to embedded devices with very limited resources. It is not uncommon for our production users to serve billions of requests daily for millions of active users with just a handful of machines."
- chrisweekly 1y ago
  "surprised if less than 50% of the certs LE issues are not..." triple-negative, too hard to parse
EvanAnderson 1y ago
I'm with you on this. I run my ACME clients as least-privileged standalone applications. On a machine where you're only running a webserver I suppose having Nginx do it the ACME renewal makes sense. On many of the machines I support I also need certificates for other services, too. In many cases I also have to distribute the certificate to multiple machines. I find it easy to manage and troubleshoot a single application handling the ACME process. I can't imagine having multiple logs to review and monitor would be easier.
nottorp 1y ago
That's okay, next step is to fold both nginx and the acme client into systemd.
- throw_a_grenade 1y ago
  Unironicaly, I think having systemd-something util that would provide TLS certs for .services upon encountering specific config knob in [Service] section would be much better that having multitude uncoordinated ACME clients that will quickly burn through allowed rate limits. Even just as a courtesy to LE/ISRG's computational resources.
  - Spivak 1y ago
    A systemd-certd would actually kinda slap. One cert store to rule them all for clients, a way to define certs and specify where they're supposed to be placed with automatic reload using the systemd dependency solver, a way to mount certs into services privately, a unified interface for interacting with the cert store.
    - nottorp 1y ago
      So ... not only would your system take ages to boot without the internets(tm) because that's how systemd works, it will be extended in the same spirit to not boot at all if letsencrypt is down. Sounds enterprise. Also, you people forgot that my proposal is to also fold the http server in, and ideally all the scripting languages and all of npm just in case.
      - throw_a_grenade 1y ago
        ExecStart=/usr/bin/python3 -m http.server WorkingDirectory=/srv/www ?
      - Spivak 1y ago
        Well I mean if you configured your system in a manner that requires one of the wait-online services that's kinda on you. It's not required for anything by default. It would be the same for certd. If you configure your system to hold up booting waiting for a cert then that's your choice but there's plenty of ways to have it not.
oliwarner 1y ago
The idea that the thing that needs the certificate, gets the certificate doesn't seem that perverse to me. The interface/port-bound httpd needs to known what domains it's serving, what certificates it's using. Automating this is pure benefit to those that want it, and a non-issue to those who don't — just don't use it.
Ajedi32 1y ago
It makes sense to me. If an application needs a signed certificate to function properly, why shouldn't it include code to obtain that certificate automatically when possible? Maybe if there were OS level features for doing the same thing you could argue the applications should call out to those instead, but at least on Linux that's not really the case. Why should admins need to install and configure a separate application just to get basic functionality working?
renewiltord 1y ago
You just don't load the module and use certbot and that will work which is what I'm doing. People get carried away with this stuff. The software is quite modular. It's fine for people to simplify it. For a bunch of tech-aware people the inability for you all here to modify your software to meet your needs is insane. As a 14 year old I was using the ck patch series to have a better (for me) scheduler in the kernel. Every other teenager could do this shit. In my 30s I have a low friction set up where each bit of software only does one thing and it's easy for me to replicate. Teenagers can do this too. Somehow you guys can't do either of these things. I don't get it. Are you stupid? Just don't load the module. Use stunnel. Use certbot. None of these things are disappearing. I much prefer. I much prefer. I much prefer. Christ. Never seen a userbase that moans as much about software (I moan about moaning - different thing) while being unable to do anything about it as HN.
- mikestorrent 1y ago
  The unix philosophy is still alive... and by that I mean complaining on newsgroups about things, not "do one thing well"
9dev 1y ago
I’m of the opposite opinion, really: Automatic TLS certificate requests are just an implementation detail of software able to advertise as accepting encrypted connections. Similarly many applications include an OAuth client that automatically takes care of requesting access tokens and refreshing them automatically, all using a discovery URI and client credentials. Lots of apps should support this automatically, with no intervention necessary, and just communicate securely with each other. And ACME is the way to enable that.
- imiric 1y ago
  Why should every software need to support encrypted connections? That is a rabbit hole of complexity which can easily be implemented incorrectly, and is a security risk of its own. Instead, it would make more sense for TLS to be handled centrally by a known and trusted implementation, which proxies the communication with each backend. This is a common architecture we've used for decades. It's flexible, more secure, keeps complexity compartmentalized, and is much easier to manage.
  - tuckerman 1y ago
    Isn't nginx one of the de facto choices (alongside HAProxy) for such a proxy and therefore it makes sense to include an ACME client? (This might be what you already had in mind but given the top level comment of the thread we are in I wasn't sure)
    - imiric 1y ago
      Yeah, I'm fine with web servers like nginx supporting TLS, ACME, or whatever protocol is required for encryption, since they can be used as proxies. I understood GP to have the opinion that most apps should have this support built-in, which is what I'm arguing against.
mholt 1y ago
Integrated ACME clients have proven to be more robust, more resilient, more automatic, and easier to use than exposing multiple moving parts: https://github.com/https-dev/docs/blob/master/acme-ops.md#certificate-management-features-should-be-embedded-within-the-application-using-the-certificates https://github.com/https-dev/docs/blob/master/acme-ops.md#ce... To avoid a splintered/disjoint ecosystem, library code can be reused across many applications.
dizhn 1y ago
I believe caddy was the first standalone software to include automated acme. It's a web server (and a proxy) so it's a very good fit. One software many domains. Proxmox likewise is a hypervisor hosting many VMs (hence domains). Another good fit. Though as far as I know they don't provide the service for the VMs "yet".
jdboyd 1y ago
Proxmox is not a hypervisor. It is a Linux distribution. As such it has a web server, kvm, zfs, and many other pieces. Maybe the acme client is built in to the web server. Maybe the acme client is built into their custom management software. Maybe they're just scripting around certbot. I do tend to find that I need multiple services with tls on the same machine, such as a web server and RabbitMQ, or postfix and dovecot. I don't know how having every program have its own acme client would end up working out. That seems like it could be a mess. On the other hand, I have been having trouble getting them all to take updated certificates correctly without me manually restarting services after cert bots cron job does an update.

atomicnumber3 1y ago

I personally think nginx is the kind of project I'd allow to have its own acme client. It's extremely extremely widely used software and I would be surprised if less than 50% of the certs LE issues are not exclusively served via nginx. Now if Jenkins adds acme support then yes I'll say maybe that one is too far.

muppetman 1y ago
But it's a webserver. I'm sure it farms out sending emails from forms it serves, I doubt it has a PHP library built in, surely it farms that out to php-fpm? It doesn't have a REDIS library or NodeJS built in. Why's ACME different?
- tuckerman 1y ago
  I get what you are saying but surely obtaining a certificate is much closer to being considered a core part of a web server related to transport, especially in 2025 when browsers throw up "doesn’t support a secure connection with HTTPS" messages left and right, than those other examples. I think there is also clearly demand: caddy is very well liked and often recommended for hobbyists and I think a huge part of that is the built in certificate management.
- firesteelrain 1y ago
  To your point, we use Venafi and it has clients that act as orchestrators to deploy the new cert and restart the web service. Webservice itself doesn’t need to be ACME aware. Venafi supports ACME protocol so it can be the ACME server like Let’s Encrypt I am speaking purely on prem non internet connect scenario
- andmarios 1y ago
  Nginx (and Apache, etc) is not just a web server; it is also a reverse proxy, a TLS termination proxy, a load balancer, etc. The key service here is "TLS termination proxy", so being able to issue certificates automatically was pretty high on the wish list.
- dividuum 1y ago
  Well, it already has, among a ton of other modules, a memcached and a JavaScript module (njs), so you’re actually not that far off. An optional ACME module sounds fitting.
- banashark 1y ago
  Well you say that.... https://openresty.org/en/ https://openresty.org/en/ "Real-world applications of OpenResty® range from dynamic web portals and web gateways, web application firewalls, web service platforms for mobile apps/advertising/distributed storage/data analytics, to full-fledged dynamic web applications and web sites. The hardware used to run OpenResty® also ranges from very big metals to embedded devices with very limited resources. It is not uncommon for our production users to serve billions of requests daily for millions of active users with just a handful of machines."
chrisweekly 1y ago
"surprised if less than 50% of the certs LE issues are not..." triple-negative, too hard to parse

EvanAnderson 1y ago

I'm with you on this. I run my ACME clients as least-privileged standalone applications. On a machine where you're only running a webserver I suppose having Nginx do it the ACME renewal makes sense. On many of the machines I support I also need certificates for other services, too. In many cases I also have to distribute the certificate to multiple machines. I find it easy to manage and troubleshoot a single application handling the ACME process. I can't imagine having multiple logs to review and monitor would be easier.

nottorp 1y ago

That's okay, next step is to fold both nginx and the acme client into systemd.

throw_a_grenade 1y ago
Unironicaly, I think having systemd-something util that would provide TLS certs for .services upon encountering specific config knob in [Service] section would be much better that having multitude uncoordinated ACME clients that will quickly burn through allowed rate limits. Even just as a courtesy to LE/ISRG's computational resources.
- Spivak 1y ago
  A systemd-certd would actually kinda slap. One cert store to rule them all for clients, a way to define certs and specify where they're supposed to be placed with automatic reload using the systemd dependency solver, a way to mount certs into services privately, a unified interface for interacting with the cert store.
  - nottorp 1y ago
    So ... not only would your system take ages to boot without the internets(tm) because that's how systemd works, it will be extended in the same spirit to not boot at all if letsencrypt is down. Sounds enterprise. Also, you people forgot that my proposal is to also fold the http server in, and ideally all the scripting languages and all of npm just in case.
    - throw_a_grenade 1y ago
      ExecStart=/usr/bin/python3 -m http.server WorkingDirectory=/srv/www ?
    - Spivak 1y ago
      Well I mean if you configured your system in a manner that requires one of the wait-online services that's kinda on you. It's not required for anything by default. It would be the same for certd. If you configure your system to hold up booting waiting for a cert then that's your choice but there's plenty of ways to have it not.

oliwarner 1y ago

The idea that the thing that needs the certificate, gets the certificate doesn't seem that perverse to me. The interface/port-bound httpd needs to known what domains it's serving, what certificates it's using. Automating this is pure benefit to those that want it, and a non-issue to those who don't — just don't use it.

Ajedi32 1y ago

It makes sense to me. If an application needs a signed certificate to function properly, why shouldn't it include code to obtain that certificate automatically when possible? Maybe if there were OS level features for doing the same thing you could argue the applications should call out to those instead, but at least on Linux that's not really the case. Why should admins need to install and configure a separate application just to get basic functionality working?

renewiltord 1y ago

You just don't load the module and use certbot and that will work which is what I'm doing. People get carried away with this stuff. The software is quite modular. It's fine for people to simplify it. For a bunch of tech-aware people the inability for you all here to modify your software to meet your needs is insane. As a 14 year old I was using the ck patch series to have a better (for me) scheduler in the kernel. Every other teenager could do this shit. In my 30s I have a low friction set up where each bit of software only does one thing and it's easy for me to replicate. Teenagers can do this too. Somehow you guys can't do either of these things. I don't get it. Are you stupid? Just don't load the module. Use stunnel. Use certbot. None of these things are disappearing. I much prefer. I much prefer. I much prefer. Christ. Never seen a userbase that moans as much about software (I moan about moaning - different thing) while being unable to do anything about it as HN.

mikestorrent 1y ago
The unix philosophy is still alive... and by that I mean complaining on newsgroups about things, not "do one thing well"

9dev 1y ago

I’m of the opposite opinion, really: Automatic TLS certificate requests are just an implementation detail of software able to advertise as accepting encrypted connections. Similarly many applications include an OAuth client that automatically takes care of requesting access tokens and refreshing them automatically, all using a discovery URI and client credentials. Lots of apps should support this automatically, with no intervention necessary, and just communicate securely with each other. And ACME is the way to enable that.

imiric 1y ago
Why should every software need to support encrypted connections? That is a rabbit hole of complexity which can easily be implemented incorrectly, and is a security risk of its own. Instead, it would make more sense for TLS to be handled centrally by a known and trusted implementation, which proxies the communication with each backend. This is a common architecture we've used for decades. It's flexible, more secure, keeps complexity compartmentalized, and is much easier to manage.
- tuckerman 1y ago
  Isn't nginx one of the de facto choices (alongside HAProxy) for such a proxy and therefore it makes sense to include an ACME client? (This might be what you already had in mind but given the top level comment of the thread we are in I wasn't sure)
  - imiric 1y ago
    Yeah, I'm fine with web servers like nginx supporting TLS, ACME, or whatever protocol is required for encryption, since they can be used as proxies. I understood GP to have the opinion that most apps should have this support built-in, which is what I'm arguing against.

mholt 1y ago

Integrated ACME clients have proven to be more robust, more resilient, more automatic, and easier to use than exposing multiple moving parts: https://github.com/https-dev/docs/blob/master/acme-ops.md#certificate-management-features-should-be-embedded-within-the-application-using-the-certificates https://github.com/https-dev/docs/blob/master/acme-ops.md#ce... To avoid a splintered/disjoint ecosystem, library code can be reused across many applications.

dizhn 1y ago

I believe caddy was the first standalone software to include automated acme. It's a web server (and a proxy) so it's a very good fit. One software many domains. Proxmox likewise is a hypervisor hosting many VMs (hence domains). Another good fit. Though as far as I know they don't provide the service for the VMs "yet".

jdboyd 1y ago

Proxmox is not a hypervisor. It is a Linux distribution. As such it has a web server, kvm, zfs, and many other pieces. Maybe the acme client is built in to the web server. Maybe the acme client is built into their custom management software. Maybe they're just scripting around certbot. I do tend to find that I need multiple services with tls on the same machine, such as a web server and RabbitMQ, or postfix and dovecot. I don't know how having every program have its own acme client would end up working out. That seems like it could be a mess. On the other hand, I have been having trouble getting them all to take updated certificates correctly without me manually restarting services after cert bots cron job does an update.