6 ms·
Let’s Encrypt: Stopping Issuance for Potential Incident – Resolved
- t1234s 5mo agoHow much of the internet is going to fail because of this?
- walrus01 5mo agoIt's an interesting thought experiment to consider how much of 'the internet' would still find a way to communicate with each other and fix the problem if somebody waved a magic wand and all http and https servers and clients magically disappeared worldwide instantly. For instance some of the folks who run core BGP at medium to large sized ISPs would revert back to a few legacy IRC channels and find each other to chat and figure out WTF is going on. "the internet" would still exist, a subset of the application layer stuff that runs on top it wouldn't...
- ben0x539 5mo agoI bet we'd see a bunch of unexpected breakage in presumed-to-be-lower-level-than-http[s] infrastructure so that eg. your legacy IRC server goes down because it's running on rented hardware and the hosting provider's operations rely on some internal http services.
- walrus01 5mo agoThis is extremely likely in the case of many automated provisioning, billing, and web interface control panel systems for shared hosting platforms, VPS, virtual machine service providers that likely do something https to https internally to communicate between tooling. In my intentionally absurd theoretical scenario, what would remain up would be the bare metal in colocation in certain service providers' environments...
- nicolas_17 5mo agoNone, unless someone is renewing their certificates only 2 hours before they expire, which is a dumb thing to do.
- esseph 5mo agoSome other internet things going on to Discord, Cloudflare, and others. Unsure if related in any way.
- noplacelikehome 5mo agoHere's hoping it's not another security nightmare...
- mcherm 5mo agoThere is one little-discussed down side to ever shorter-lived certificates...
- devrand 5mo agoIf you're using ACME to handle certificate rotation, can't you just configure multiple providers?
- pseudalopex 5mo agohttps://news.ycombinator.com/item?id=48071607 https://news.ycombinator.com/item?id=48071607
- Analemma_ 5mo agoOnly if you’re reissuing right before expiration, which is a stupid thing to do. If you have a 47-day cert, best practice is to reissue on day 30, meaning LE would need to be down for more than two weeks before anything went wrong. If this outage breaks your system, that’s entirely on you, not Let’s Encrypt.
- mark_round 5mo agoThat's really not good. Fortunately I'm not using any short-lived certificates like the recently announced 6 day certs, so have some breathing room. Without further details, I'd imagine anyone with a short-lived cert is getting a bit sweaty right now. Let's Encrypt has become one of those pieces of critical Internet infrastructure that just quietly hums away in the background, the fact that they've stopped ALL issuance is deeply concerning.
- walrus01 5mo agoConsidering the open source nature of Letsencrypt, I wonder what the barriers/costs would be (theoretically) to a wealthy benefactor who wanted to duplicate its server side infrastructure and a core staffing level of persons, and fund a "parallel" equally trusted, alternative entity with a solid governing board. Same general idea how Acton funded the Signal foundation. Somewhere that none of the physical infrastructure/hosting environment overlapped with existing Letsencrypt stuff so that the failure of one entity would have zero blast radius affecting the other. I know there's a long and complicated process to go through to become a trusted root CA and get your CA public cert auto-installed in every OS and browser trust store. Indeed in the early days of letsencrypt I recall their root CA certs were signed by other older root CAs.
- JCTheDenthog 5mo ago[dead]
- dochtman 5mo agoA lot of Let’s Encrypt is not the software but a bunch of auditing and process that ensure compliance and make it legible to the required auditors.
- walrus01 5mo agoI understand there's probably a big thorny problem of duplicating the corporate process/policies on the human level that ensure compliance, but is the back-end software pipelining stuff to CT logs not also something that can be replicated? Or is it not part of the server side stuff which has been open sourced? https://letsencrypt.org/docs/ct-logs/ https://letsencrypt.org/docs/ct-logs/
- kalmarv 5mo agoHopefully it's just a technical issue and not something like a key compromise. This could have disastrous effects considering how much of the web runs on LE certs these days. Granted if it's configured properly everyone should have 30 days of leeway before having to issue new certs...
- mark_round 5mo ago"We have been made aware of a potential incident and are shutting down all issuance" seems to lean towards the latter and not simply a technical issue :(
- tptacek 5mo agoJosh Aas is on the thread. It's a compliance issue, they expect to be issuing shortly.
- rvnx 5mo agoWhat if they get kicked out of trusted roots because non-compliant ?
- nicolas_17 5mo agoThat's why they take incidents like this seriously and stop issuance until it's fixed. They could get kicked out of trusted roots otherwise.
- nijave 5mo agoChange your config to ZeroSSL or another free ACME provider?
- wolrah 5mo agoYou don't get kicked out of trusted roots for non-compliance, you get kicked out for continuing to knowingly issue non-compliant certs, failing to revoke non-compliant certs in a timely fashion once discovered, etc. Pausing issuance immediately upon discovery of a compliance issue is the absolute correct response so as long as they do their followup appropriately there is absolutely zero risk of being distrusted.
- cedws 5mo agoDiscord is out too right now, probably unrelated though.
- aroman 5mo agoJust speculating, but I don't think it's unrelated. Discord heavily utilizes Cloudflare, and Cloudflare uses Let's Encrypt for a certificate issuance. If they happened to have a certificate signing dependency in some operational rollout today, I think it could explain it. Certainly the timing is very correlated.
- everfrustrated 5mo agoCloudflare doesn't issue let's encrypt certs
- cedws 5mo agoI guess we'll find out but it would be surprising if they use Let's Encrypt for their backend services. The front door is issued by Google Trust Services.
- reaperducer 5mo agoJust speculating Then why post? HN is for informed discussion, not every random thought in someone's head. Certainly the timing is very correlated. I had chocolate ice cream for breakfast. Certainly the timing is very corrolated [sic].
- winstonwinston 5mo agoOn my account they always serve Google issued certificates. There is also Let’s encrypt certificate but it is not used though. I guess that’s a fail-safe.
- nijave 5mo agoIn Cloudflare Enterprise you can pick either or leave it on auto. Iirc there's a 3rd option but I don't know if it's still supported (Terraform and SDKs used to have it in the enum) https://developers.cloudflare.com/ssl/reference/certificate-authorities/ https://developers.cloudflare.com/ssl/reference/certificate-...
- jstyles 5mo agoHopefully just a minor mississuance incident and not something more serious.
- bravetraveler 5mo agoIt's certainly an incident when ceasing to issue certificates... after doing absolutely everything, including limiting lifetime, to encourage their frequent renewal
- deleted 5mo ago[deleted]
- bstsb 5mo agoin other news, Digicert's Secure Site Pro certificates are down to only $5,880.00 yearly for one wildcard domain!
- jaas 5mo agoThis is a compliance incident, we should be issuing again shortly. Update: Issuance is back up. Update: Preliminary incident report: https://bugzilla.mozilla.org/show_bug.cgi?id=2038351 https://bugzilla.mozilla.org/show_bug.cgi?id=2038351
- gabeio 5mo ago> This is a compliance incident Uh. I don't know if I like the sound of that...
- walrus01 5mo agoIndeed. "Compliance" can mean some internal audit/monitoring system has tripped and requires in depth investigation and preservation of logging, or it can mean "federal law enforcement with badges are right now standing in our datacenter and/or NOC serving a court order".
- tptacek 5mo agoAt times like this it's worth remembering that message boards strongly favor whatever narrative is going to be most fun and exciting to talk about.
- walrus01 5mo agoI sincerely hope it's the most mundane and least spectacular explanation possible, just saying from my point above that compliance has a very wide range of possible meanings and interpretations (also depending on the background/career POV of the reader), until the incident is further explained..
- jaas 5mo agoIn that sense, prepare yourself to be bored.
- michaelt 5mo ago
- x86a 5mo agoThey had scheduled maintenance a few hours ago, https://letsencrypt.status.io/pages/maintenance/55957a99e800baa4470002da/69fe1ac74c661a0589f527c1 https://letsencrypt.status.io/pages/maintenance/55957a99e800...
- deleted 5mo ago[deleted]
- DerekL 5mo agoThe title is misspelled. It's “Let's Encrypt”, with an apostrophe.
- DerekL 5mo agoIt's fixed now. Thanks!
- baigy 5mo agodang I'll have to return to paid certs again?
- deleted 5mo ago[deleted]
- croemer 5mo agoIssuance was stopped almost 2 hours ago: May 8, 2026 18:37 UTC.
- hosteur 5mo agoRelated Cloudflare issue: https://www.cloudflarestatus.com/incidents/z3vgxxfvt3yb https://www.cloudflarestatus.com/incidents/z3vgxxfvt3yb