6 ms·

Is this a good opportunity to plug an open source project I've been working on for 5 years that basically solves this problem? https://caddyserver.com https://c

by mholt 7y ago

Is this a good opportunity to plug an open source project I've been working on for 5 years that basically solves this problem? https://caddyserver.com https://caddyserver.com will keep your certs renewed for free, automatically, without extra tooling, dependencies, or moving parts. I really hope more sites will use it because this stuff happens. If you've heard of Caddy but haven't used Caddy 2 yet, we've made some huge improvements with it, and it's capable of managing tens of thousands of certificates at a time. As of next week's beta it can handle certificates with lifetimes as short as a few minutes. It works as a load balancer, in a cluster behind load balancers, and with Docker with its new dynamic config API. There's no technical reasons I know of why sites like TechCrunch can't use it.

vinaypai 7y ago
What does it do that certbot doesn't? Personally, I don't see the benefit of bundling certificate renewals with the HTTPS server.
- mholt 7y ago
  A quick perusal of the Let's Encrypt forums will reveal a few issues. For one, it's easy to misconfigure. There's a lot of surface area it has to cover in the web server and there's a lot that can go wrong. It can also be tricky to install since it is separate from your web server and has a number of complex depeneldency requirements (like many large Python programs). And because it's an external dependency, there's no way the server can react to errors in CertBot, only CertBot can control the server, so you don't get the benefit of duplexed interactions. Caddy is written in Go, a memory safe language. I cannot overstate how significant it is that most servers like ngixn, Apache, and HAProxy are written in C and cannot offer the same security guarantees. Caddy will also staple OCSP, in the most robust way compared to other servers. It has weathered outages that took Apache and NGINX sites down (in Firefox). We've also seen CertBot consume high amounts of resources at scale, whereas Caddy can handle thousands of certs no problem. That's why some companies have talked to me about why they switched. Caddy can coordinate cert management in a cluster. This happens automatically when configured with the same storage backend. CertBot doesn't do that. Caddy works great behind reverse proxies, or as the reverse proxy. Perhaps most importantly, Caddy is the only server to use HTTPS by default without needing any explicit configuration. You simplify your deployment workflows and have less room for things to go wrong. There's also a values statement here... If you think privacy and security are important, you choose software that enables privacy by default because it aligns with your values. Sure, any number of solutions can get you TLS and a few even get you TLS via ACME, but Caddy (2) is highly optimized to handle the edge cases and scaling requirements a lot of sites have these days.
  - vinaypai 7y ago
    I don't meant to pooh-pooh your project, but Go's memory safety isn't some kind of automatic security guarantee... especially when it comes with a giant downside of having to run as root. Every other production-grade server drops privileges as soon as possible, for a very good reason.
    - mholt 7y ago
      Go does bounds checking and doesn't have dangling pointers, so it's inherently safe from buffer overflows. You don't have to run Caddy as root in production, even to bind to low ports. (None of my sites do.) https://memorysafety.fail https://memorysafety.fail for more info on memory unsafety.
      - vinaypai 7y ago
        Thank you for your condescending reply. I understand Go's memory model quite well (and have used Go quite a bit). News flash: you can have bugs of other kinds besides buffer overflows.
        mholt 7y ago
        Er, sorry, I wasn't being condescending. I intended to clarify some relevant facts, please don't take it personally. You had said: > Go's memory safety isn't some kind of automatic security guarantee... But in some cases, it is. I'm not claiming more than that, nor am I saying that you can't have "bugs of other kinds" -- on which point I agree with you.
        vinaypai 7y ago
        > Er, sorry, I wasn't being condescending. I intended to clarify some relevant facts, please don't take it personally. Fair enough, it's easy to misunderstand the intended tone in online comments.
    - justinclift 7y ago
- discreditable 7y ago
  Fail to start if it can't reach the ACME server https://github.com/caddyserver/caddy/issues/1680 https://github.com/caddyserver/caddy/issues/1680
  - mholt 7y ago
    That was 3 years ago when ACME was still a new and unstandardized protocol. That was one of the worst days of my life, trying to fix that while finishing my NIPS paper. If you read the issue you'll see that we fixed that and learned from it.
plopz 7y ago
Is this the one that refuses to start if there is a valid cert but it cant renew it?
- mholt 7y ago
  Nope, Caddy issues certs in the background and will retry with exponential backoff for up to 30 days. It reverts to the staging endpoint during retries so it doesn't count against your rate limits. It also has internal throttling to allow multiple challenges to take place at once (in most cases), so it can obtain several certificates per second as CA rate limits allow, being one of the highest-performing ACME clients available, and one of the most robust, too. https://caddyserver.com/docs/automatic-https#errors https://caddyserver.com/docs/automatic-https#errors
  - irishsultan 7y ago
    Also, yes, it's the one that had this issue (presumably this is what plopz had in mind https://github.com/caddyserver/caddy/issues/1680 https://github.com/caddyserver/caddy/issues/1680, https://news.ycombinator.com/item?id=14375022 https://news.ycombinator.com/item?id=14375022)
    - mholt 7y ago
      Caddy is not the only one, though. Traefik had the same problem: https://github.com/containous/traefik/issues/791 https://github.com/containous/traefik/issues/791 So to say Caddy is "the one" that had that issue is a mis-characterization. See also: https://github.com/containous/traefik/issues/1631 https://github.com/containous/traefik/issues/1631, https://github.com/containous/traefik/issues/1641 https://github.com/containous/traefik/issues/1641, https://github.com/containous/traefik/issues/1637 https://github.com/containous/traefik/issues/1637, https://github.com/containous/traefik/issues/1636 https://github.com/containous/traefik/issues/1636, and https://github.com/containous/traefik/issues/1632 https://github.com/containous/traefik/issues/1632
notyourday 7y ago
> There's no technical reasons I know of why sites like TechCrunch can't use it. Of course there is - a web server that gets a certificate via LE is a single screw. A commercial internet-facing site is a high rise. One does not just replace a screw in a high rise -- it requires ensuring that the right tools are in every toolbelt to manipulate the screws. For Caddy to be useful in a non- oh look ma, i'm on the internet projects, it needs to fit into the tooling system. For example, it should be fetching storing certificates in a database that can be accessed by multiple servers. Rather than doing everything behind the scenes, it should be emitting events indicating its desire to renew a cert and let someone else handle ( maybe a different caddy ) handle the renewals, etc, etc, etc.
- mholt 7y ago
  Oh I understand that, believe me. That's an operational reason, though, and not as much a technical one. By the way, Caddy CAN store certs in a database accessed by multiple servers. It coordinates cert management when configured as a cluster, automatically. Caddy is the only server that does this (for free).
  - notyourday 7y ago
    > That's an operational reason, though, and not as much a technical one. Every single operational reason is a technical reason if it touches operation of the technology. > By the way, Caddy CAN store certs in a database accessed by multiple servers. It coordinates cert management when configured as a cluster, automatically. Caddy is the only server that does this (for free An edge web server should carry no knowledge of a concept of a cluster. It should simply be able to ask something for a cert. Otherwise it is a screw that insists that in order for it be used in high rise, it needs to be used in a specific pattern that the screw designer envisioned. If you want to make Caddy easily adoptable in high rises I suggest the following path: 1. Subscribe to events saying "fetch this cert from here" 2. Emit events saying "Need to renew X" when you need to renew. 3. Create ability to forward renew requests to renew requests processor. 4. Make request processor update the cert store and emit the event that the key pair is available.
laumars 7y ago
Unless I'm overlooking the obvious -- and I apologise in advance if I am -- but you don't clearly state who you are using as your CA.
- yalooze 7y ago
  It's Let's Encrypt. Mentioned in the Features sections: Security and Privacy > Certificates > AUTO OBTAIN
  - laumars 7y ago
    Ah yes, I see it now. That's buried deep though. Definitely not obvious.
    - mholt 7y ago
      It's because most people go to our website already knowing they need to use Let's Encrypt, and they've found it by searching for a server that works with Let's Encrypt. Hence, we don't make that detail our front-and-center. Instead, we're targeting people who don't already know what Caddy is. The information is readily available in the docs.
      - laumars 7y ago
        That's clearly not always the case otherwise we wouldn't be having this conversation to begin with. Also I'm not suggesting you make it front and centre but as it is currently there's only one mention on your landing page and it's buried so far deep it might as well not be mentioned at all. I did also look through some of the other docs and still missed it. If I missed it you can bet others have too. All I'm suggesting is that it's worth being a little more explicit about that because that will be a key detail for some people. You mention about it being written in Go. That seems less relevant (in my personal opinion) but that's a great place to add a line saying something like "Caddy uses the ACME protocol to automatically manage secure certificates issued by Let's Encrypt."
djsumdog 7y ago
There's also Traefik if all you want is a load balancer for services run Docker containers. It does Lets Encrypt as well. Right now I use the official HAProxy container + the official certbot container and a script that renews all my domains and then sends a sig-HUP to HAProxy to reload the certs. It mostly works but there have been a few hickups (HAProxy not starting because of the README file in the live LetsEncrypt folder; however if you send an HUP, it ignores it fine). Has anyone used Traefik? Is it worth switching to over HAProxy+Certbot? https://docs.traefik.io/ https://docs.traefik.io/
- nodesocket 7y ago
  I use Traefik with Kubernetes clusters via Helm chart. Works amazingly. Just define a ingress resource and boom, everything just works. Note, if doing on-prem Kubernetes and cloud native load balances are not available, highly recommend the MetalLB[1] project. Traefik requires a load balancer resource in front of it. [1] https://metallb.universe.tf/ https://metallb.universe.tf/

by mholt 7y ago

vinaypai 7y ago

What does it do that certbot doesn't? Personally, I don't see the benefit of bundling certificate renewals with the HTTPS server.

mholt 7y ago
A quick perusal of the Let's Encrypt forums will reveal a few issues. For one, it's easy to misconfigure. There's a lot of surface area it has to cover in the web server and there's a lot that can go wrong. It can also be tricky to install since it is separate from your web server and has a number of complex depeneldency requirements (like many large Python programs). And because it's an external dependency, there's no way the server can react to errors in CertBot, only CertBot can control the server, so you don't get the benefit of duplexed interactions. Caddy is written in Go, a memory safe language. I cannot overstate how significant it is that most servers like ngixn, Apache, and HAProxy are written in C and cannot offer the same security guarantees. Caddy will also staple OCSP, in the most robust way compared to other servers. It has weathered outages that took Apache and NGINX sites down (in Firefox). We've also seen CertBot consume high amounts of resources at scale, whereas Caddy can handle thousands of certs no problem. That's why some companies have talked to me about why they switched. Caddy can coordinate cert management in a cluster. This happens automatically when configured with the same storage backend. CertBot doesn't do that. Caddy works great behind reverse proxies, or as the reverse proxy. Perhaps most importantly, Caddy is the only server to use HTTPS by default without needing any explicit configuration. You simplify your deployment workflows and have less room for things to go wrong. There's also a values statement here... If you think privacy and security are important, you choose software that enables privacy by default because it aligns with your values. Sure, any number of solutions can get you TLS and a few even get you TLS via ACME, but Caddy (2) is highly optimized to handle the edge cases and scaling requirements a lot of sites have these days.
- vinaypai 7y ago
  I don't meant to pooh-pooh your project, but Go's memory safety isn't some kind of automatic security guarantee... especially when it comes with a giant downside of having to run as root. Every other production-grade server drops privileges as soon as possible, for a very good reason.
  - mholt 7y ago
    Go does bounds checking and doesn't have dangling pointers, so it's inherently safe from buffer overflows. You don't have to run Caddy as root in production, even to bind to low ports. (None of my sites do.) https://memorysafety.fail https://memorysafety.fail for more info on memory unsafety.
    - vinaypai 7y ago
      Thank you for your condescending reply. I understand Go's memory model quite well (and have used Go quite a bit). News flash: you can have bugs of other kinds besides buffer overflows.
      - mholt 7y ago
        Er, sorry, I wasn't being condescending. I intended to clarify some relevant facts, please don't take it personally. You had said: > Go's memory safety isn't some kind of automatic security guarantee... But in some cases, it is. I'm not claiming more than that, nor am I saying that you can't have "bugs of other kinds" -- on which point I agree with you.
        vinaypai 7y ago
        > Er, sorry, I wasn't being condescending. I intended to clarify some relevant facts, please don't take it personally. Fair enough, it's easy to misunderstand the intended tone in online comments.
  - justinclift 7y ago
discreditable 7y ago
Fail to start if it can't reach the ACME server https://github.com/caddyserver/caddy/issues/1680 https://github.com/caddyserver/caddy/issues/1680
- mholt 7y ago
  That was 3 years ago when ACME was still a new and unstandardized protocol. That was one of the worst days of my life, trying to fix that while finishing my NIPS paper. If you read the issue you'll see that we fixed that and learned from it.

plopz 7y ago

Is this the one that refuses to start if there is a valid cert but it cant renew it?

mholt 7y ago
Nope, Caddy issues certs in the background and will retry with exponential backoff for up to 30 days. It reverts to the staging endpoint during retries so it doesn't count against your rate limits. It also has internal throttling to allow multiple challenges to take place at once (in most cases), so it can obtain several certificates per second as CA rate limits allow, being one of the highest-performing ACME clients available, and one of the most robust, too. https://caddyserver.com/docs/automatic-https#errors https://caddyserver.com/docs/automatic-https#errors
- irishsultan 7y ago
  Also, yes, it's the one that had this issue (presumably this is what plopz had in mind https://github.com/caddyserver/caddy/issues/1680 https://github.com/caddyserver/caddy/issues/1680, https://news.ycombinator.com/item?id=14375022 https://news.ycombinator.com/item?id=14375022)
  - mholt 7y ago
    Caddy is not the only one, though. Traefik had the same problem: https://github.com/containous/traefik/issues/791 https://github.com/containous/traefik/issues/791 So to say Caddy is "the one" that had that issue is a mis-characterization. See also: https://github.com/containous/traefik/issues/1631 https://github.com/containous/traefik/issues/1631, https://github.com/containous/traefik/issues/1641 https://github.com/containous/traefik/issues/1641, https://github.com/containous/traefik/issues/1637 https://github.com/containous/traefik/issues/1637, https://github.com/containous/traefik/issues/1636 https://github.com/containous/traefik/issues/1636, and https://github.com/containous/traefik/issues/1632 https://github.com/containous/traefik/issues/1632

notyourday 7y ago

> There's no technical reasons I know of why sites like TechCrunch can't use it. Of course there is - a web server that gets a certificate via LE is a single screw. A commercial internet-facing site is a high rise. One does not just replace a screw in a high rise -- it requires ensuring that the right tools are in every toolbelt to manipulate the screws. For Caddy to be useful in a non- oh look ma, i'm on the internet projects, it needs to fit into the tooling system. For example, it should be fetching storing certificates in a database that can be accessed by multiple servers. Rather than doing everything behind the scenes, it should be emitting events indicating its desire to renew a cert and let someone else handle ( maybe a different caddy ) handle the renewals, etc, etc, etc.

mholt 7y ago
Oh I understand that, believe me. That's an operational reason, though, and not as much a technical one. By the way, Caddy CAN store certs in a database accessed by multiple servers. It coordinates cert management when configured as a cluster, automatically. Caddy is the only server that does this (for free).
- notyourday 7y ago
  > That's an operational reason, though, and not as much a technical one. Every single operational reason is a technical reason if it touches operation of the technology. > By the way, Caddy CAN store certs in a database accessed by multiple servers. It coordinates cert management when configured as a cluster, automatically. Caddy is the only server that does this (for free An edge web server should carry no knowledge of a concept of a cluster. It should simply be able to ask something for a cert. Otherwise it is a screw that insists that in order for it be used in high rise, it needs to be used in a specific pattern that the screw designer envisioned. If you want to make Caddy easily adoptable in high rises I suggest the following path: 1. Subscribe to events saying "fetch this cert from here" 2. Emit events saying "Need to renew X" when you need to renew. 3. Create ability to forward renew requests to renew requests processor. 4. Make request processor update the cert store and emit the event that the key pair is available.

laumars 7y ago

Unless I'm overlooking the obvious -- and I apologise in advance if I am -- but you don't clearly state who you are using as your CA.

yalooze 7y ago
It's Let's Encrypt. Mentioned in the Features sections: Security and Privacy > Certificates > AUTO OBTAIN
- laumars 7y ago
  Ah yes, I see it now. That's buried deep though. Definitely not obvious.
  - mholt 7y ago
    It's because most people go to our website already knowing they need to use Let's Encrypt, and they've found it by searching for a server that works with Let's Encrypt. Hence, we don't make that detail our front-and-center. Instead, we're targeting people who don't already know what Caddy is. The information is readily available in the docs.
    - laumars 7y ago
      That's clearly not always the case otherwise we wouldn't be having this conversation to begin with. Also I'm not suggesting you make it front and centre but as it is currently there's only one mention on your landing page and it's buried so far deep it might as well not be mentioned at all. I did also look through some of the other docs and still missed it. If I missed it you can bet others have too. All I'm suggesting is that it's worth being a little more explicit about that because that will be a key detail for some people. You mention about it being written in Go. That seems less relevant (in my personal opinion) but that's a great place to add a line saying something like "Caddy uses the ACME protocol to automatically manage secure certificates issued by Let's Encrypt."

djsumdog 7y ago

There's also Traefik if all you want is a load balancer for services run Docker containers. It does Lets Encrypt as well. Right now I use the official HAProxy container + the official certbot container and a script that renews all my domains and then sends a sig-HUP to HAProxy to reload the certs. It mostly works but there have been a few hickups (HAProxy not starting because of the README file in the live LetsEncrypt folder; however if you send an HUP, it ignores it fine). Has anyone used Traefik? Is it worth switching to over HAProxy+Certbot? https://docs.traefik.io/ https://docs.traefik.io/

nodesocket 7y ago
I use Traefik with Kubernetes clusters via Helm chart. Works amazingly. Just define a ingress resource and boom, everything just works. Note, if doing on-prem Kubernetes and cloud native load balances are not available, highly recommend the MetalLB[1] project. Traefik requires a load balancer resource in front of it. [1] https://metallb.universe.tf/ https://metallb.universe.tf/