7 ms·
LetsEncrypt – Expiration Notification Service Has Ended
- leakycap 1y agoOne could say it expired. > Providing expiration notifications costs Let’s Encrypt tens of thousands of dollars per year, money that we believe can be better spent on other aspects of our infrastructure. Appreciate the honesty (they had other reasons, too! but emails are a pain and expensive at their scale)
- amenghra 1y agoThey should just build a mobile app for the purpose of receiving these notifications. Make the app $2.99. Turn the expense into a profit. /s
- tuananh 1y agowhat's the cost of sending notifications via mobile app? cheaper than email?
- tom1337 1y agoat least for iOS there are no costs associated with using Apple Push Notification Service (APNS) but depending on the way you use it you either need to pay for the infrastructure that sends your notifications to Apple or for a service like OneSignal which does that for you. Not sure what the volume of LE is but I am pretty sure it's a smart move to focus on their core "business" (providing certificates) and let other handle expiration notifications.
- b112 1y agoMobile app? Now they need to develop that, keep up to date with OS version changes, and far far worse, support end users and their bugs? And worse of all, worry about Apple and Google's arbitrary rejections? This seems far more costly than email. Just having one dev keeping those apps going, is likely 20x or more than their email costs per year.
- Hamuko 1y agoI imagine this is best left to third parties like the recommended service linked in the post. I assume that there's also a whole deluge of other services that have similar offerings.
- nikolayasdf123 1y agothat's quite a good idea..
- bayindirh 1y agoAs an other option, you install a cron job on your server, and send push notifications via pushover or ntfy.sh whenever it fails to renew. Pushover is $5 once for personal use, ntfy.sh can be completely self-hostable if you prefer. I have written a small tool which utilizes pushover for these reasons. You can receive the notifications on your browser/mobile for free afterwards.
- 0x073 1y agoOr just a cronjob that fetches the tls certs and look at the expiration date and then send a mail or X. So it's even work if you don't have control about the le client.
- unilynx 1y agoExactly this. Don't look at the renewal proces, look at its output. It'll work for all certificate sources and catch other potential errors too (eg the webserver reporting success but not presenting the new certificate)
- bayindirh 1y agoThat'll work too. The idea was to put your own infra in place if you really need that, and it's not very hard to do it, even with completely self-hosted stack.
- genewitch 1y agotens of thousands of dollars? that's it? No one can just write them a check? switchgear costs more than that!
- leakycap 1y agoIf someone were to hand them a check, with that $10,000, they would like to do something other than send reminder emails That's the point
- jbverschoor 1y agoNot sure why, but many large companies that rely heavily on any open source/free initiative don’t donate. It’s sickening tbh
- szszrk 1y agoWhy discuss it here? Let's Encrypt has a shitload of corporate sponsorship. Look at their main page.
- udev4096 1y agoWhy not? The sponsorship they get is far from enough. For such a significant CA, it should be a lot more than that
- szszrk 1y agoDo you have some links to their financial reports for last years? I'd love to see that. > Why not? Because I believe it is "sickening" expressing how "sickening (...) it is that large companies don't support open source/free initiative" when discussing one of the projects that do global-scale operations purely on corporate and personal donations. Somehow Let's Encrypt themself don't express that sickening in their blogs and websites. They do thank their sponsors, though.
- eszed 1y ago
- cbenskxk 1y agowill email still be recuired for getting certs?
- dizhn 1y agoAs far as I know. No.
- TonyTrapp 1y agoI don't think it ever was? I never gave my email address to LetsEncrypt but I'm also not using their official client.
- cpach 1y agoAccount registration is done by sending a public key to a certain API endpoint. The key will become associated with an account URL that looks like this: https://acme-v02.api.letsencrypt.org/acme/acct/4277968575 https://acme-v02.api.letsencrypt.org/acme/acct/4277968575
- wordofx 1y ago[flagged]
- gleenn 1y agoSending single custom emails is much more effort than bulk-mailing a huge list operationally. Sending bulk can be accomplished by uploading a csv of emails to some enail bulk sender versus code to run at the correct time for the correct user... way easier in bulk and way cheaper
- Y_Y 1y agoIs it truly much more difficult? At worst you could batch them by week and registered email, a one-liner can generate the list of destinations, and then you send that to your newsletter-sender-service and call the email "your cert is expiring next week".
- leakycap 1y agoIt is easy to think something like this is easy until you attempt to do it. Are you really questioning a free SSL Certificate system when it says something is too complex and not worth it? If you ever set up a free SSL before LetsEncrypt, you'd know they're amazing and you can trust them not to lie to you, especially about this where they've outlined the reasons clearly.
- szszrk 1y agoOf course it's more difficult. You are talking of a volume of around 600 000 000 domains (based on a plot on their website) that try to renew at best after 8 weeks. And that's just default profile, there are 160h certs profiles now [0]. You think they will ever send nearly as much as (at least) 75 million newsletter mails weekly? Sendgrid's highest value in their pricing slider is 1,25 mil a week. - [0] https://letsencrypt.org/docs/profiles/ https://letsencrypt.org/docs/profiles/
- wordofx 1y agoIt has nothing to do with complexity. > Providing expiration notification emails means that we have to _retain millions of email addresses_ connected to issuance records. As an organization that _values privacy_, removing this requirement is important to us. A mailing list. Is still retaining emails somewhere. Doesn’t matter if it’s stored in a text file on a usb drive in a vault. It’s still retaining an email list.
- Jazgot 1y agoThis pushed me to automate certificate renewal for all my domains. This is much better than waiting for any kind of notifications, and it was very easy. I think this is a very good decision on their part.
- toast0 1y agoThese emails were handy to detect when the automation failed.
- tialaramex 1y agoI strongly recommend building affirmative detection. A script which checks everything is OK and either tells everybody "Yeah, everything is OK" or "Here are the problems" means when, inevitably, that script doesn't fire, you don't get the false impression everything is OK. All "silent success" detection systems will also silently fail and so they're worse than useless in my experience.
- nikolayasdf123 1y agois there a Slack bot for expiry checks?
- general1726 1y agoWrite a lambda in some cloud provider framework and run it every 1 hour to 24 hours and if it finds expired certificate, it will use webhooks to send you a message on Slack or Gotify or whatever. Or you can just periodically renew the certificate on server using Task Scheduler + win-acme or Cron and certbot.
- nikolayasdf123 1y agohaha, I was asking if somebody build that or recommends good open sourced version of exactly this
- TekMol 1y agoCertificates are still a pain in the butt. One of the most cumbersome aspects of the web. Especially domain wide certs which need DNS auth. DNS auth would be okish if it was simply tied to a txt entry in the DNS and valid as long as the txt entry is there. Why does LetsEncrypt expire the cert while the acme DNS entry is still there? Which attack vector does this prevent? Also, why not support file based auth in .well-known/acme-challenge/... for domain wide certs? Which attack vector does that prevent?
- matharmin 1y agoCertificates have a static expiry date by design - it's not "LetsEncrypt expiring the cert". There is no way to avoid expiring a cert if the DNS entry is still there - all you can do is make it easier to renew the cert. That means it must be automated, in which case it doesn't matter if you need to re-create a DNS entry. In my experience, it takes a little effort to set things up the first time, but from then on it just works.
- rjst01 1y agoI think the parent commenter would be satisfied if they could authorize their DNS by creating a DNS challenge entry one time, and then continue to renew their certificate as long as that entry still existed. And I'm sympathetic to the concerns that automating this type of thing is hard - many of the simpler DNS tools - which otherwise more than cover the needs for 90% of users - do not support API control or have other compromises with doing so. That said, I do think LE's requirements here are reasonable given how dangerous wildcard certs can be.
- jeroenhd 1y ago> many of the simpler DNS tools -...- do not support API control That's on the DNS provider in my opinion. They can, if they want to, make things easy and automatic for their customers, but they choose not to. There's a whole list of provider-specific plugins (https://eff-certbot.readthedocs.io/en/stable/using.html#dns-plugins https://eff-certbot.readthedocs.io/en/stable/using.html#dns-...) with many more unofficial ones available (https://pypi.org/search/?q=certbot-dns- https://pypi.org/search/?q=certbot-dns-*). Generic ones, like the DirectAdmin one, will work for many web hosts that don't have their own APIs. If you like to stick with whatever domain provider you picked and still want to use Let's Encrypt DNS validation, you can create a CNAME to another domain on a domain provider that does have API control. For instance, you could grab one of those free webhosting domains (rjst01.weirdfreewebhostthatputsadsinyourhtml.biz) with DirectAdmin access, create a TXT record there, and CNAME the real domain to that free web host. Janky, but it'll let you keep using the bespoke, API-less registrar. I imagine you could set up a small DNS service offering this kind of DNS access for a modest fee ($1 per year?) just to host API-controllable CNAME DNS validation records. Then again, most of the time the people picking weird, browser-only domain registrars do so because it allows them to save a buck, so unless it's free the service will probably not see much use.
- jgaa 1y agoWhen I received the first warning email about this, I wrote a simple library and cli to validate all my certs for me. https://github.com/jgaa/openvalify https://github.com/jgaa/openvalify
- samlinnfer 1y agoI just have a cronjob that does: #!/usr/bin/env bash cert_check() { server=$1 host=$2 port=$3 str=`ssh "$server" "echo | openssl s_client -servername $host -connect localhost:$port | openssl x509 -noout -checkend 604800"` || true if ! echo "$str" | grep -q 'Certificate will not expire' ; then echo "$str" | ./send-email.py "Certificate \"$host\" on $server will expire in 7 days" \ fi } cert_check name myserver.com 443
- masklinn 1y agoIf you’re automating the check why not automate the renewal directly?
- detaro 1y agowho says they don't have the renewal automated?
- jeroenhd 1y agoI've missed expired certificates because of a configuration issue that broke the certbot automation. Granted, I could've read the certbot journalctl output, but 99.9% of the time that's a waste of time. Not like there was anything mission-critical on there.
- throw0101b 1y ago> https://github.com/jgaa/openvalify https://github.com/jgaa/openvalify I don't begrudge people writing a tool to learn, but it should be noted that this wheel has already been invented: * https://github.com/matteocorti/check_ssl_cert https://github.com/matteocorti/check_ssl_cert * https://exchange.nagios.org/directory/Plugins/Security/check_ssl-2Ftls/details https://exchange.nagios.org/directory/Plugins/Security/check... * https://github.com/narbehaj/ssl-checker https://github.com/narbehaj/ssl-checker * https://github.com/Matty9191/ssl-cert-check https://github.com/Matty9191/ssl-cert-check
- whatever1 1y agoIs it the right time to rant about the cert expiration as a concept? I understand why certs might be revoked. But expire?
- unilynx 1y agoCan't remove a certificate from the revocation lists until it's expired, leading to boundless growth of those lists. Risk of private keys/certificates from old backup media being leaked (remembering the adobe password leak...) and then suddenly coming back online and working until someone figures out how to revoke them
- borplk 1y agoAt a minimum I consider it like an automatic "garbage collection" mechanism that prevents dead and abandoned things to remain "valid forever". It also helps with things such as change of ownership so after a certain period of time you can have the peace of mind that certs potentially issued by the previous owners are not lingering around as active (I understand things such as revoking and pinning can help with this too but It's nice to have a plain time based expiry too).
- em-bee 1y agorevoking certs does not work. it is so bad that the end result is that by 2029 certificates will not be allowed to be valid longer than 47 days: https://news.ycombinator.com/item?id=43693900 https://news.ycombinator.com/item?id=43693900
- scrapheap 1y agoThis makes sense to me. You should never rely on your CA to let you know that a certificate is due to expire soon, you should have your own monitoring in place that actively checks this for you.
- kassner 1y agoI do agree with you, and setting up your own monitoring is key. I have that. Yet it was still valuable to find those that fell through the cracks. At work, the emails prevented a couple of outages by expired cert, because a dev that left was renewing them by hand and we only found out when they left and the catch-all started to bubble them up to support. Things fall through the cracks, or people are in a pinch and just forget to add the cert to the in-house monitoring system. The emails were a wonderful failsafe. I wish I could just query LE to tell me all existing certain where the account is under my domain name. Extremely helpful to assemble a SBOM.
- bo1024 1y agoAs a hobbyist without a lot of time for sysadmin, it would be nice if basic email monitoring was a standard package (apt install letsencrypt-monitors or something).
- bravesoul2 1y agoUse one of the myriad uptime monitoring services.
- johnisgood 1y agoJust use certbot. It automatically sets up a scheduled task to renew your SSL/TLS certificates in the background, typically using a systemd timer that runs twice a day. I do not know why people using LetsEncrypt would not set up certbot along with it, that is how I do it. Some nginx config + certbot.
- Walf 1y agoMaybe the situation's improved, but I found certbot from system package managers would diverge from latest version, sometimes significantly, like support for DNS challenge APIs breaking. I switched to ‘acme.sh’ for most machines and haven't looked back. It no longer has Let's Encrypt as its default issuer, but you can set it back to LE with one config command.
- udev4096 1y agoI can't believe they didn't end it soon. Majority of the users have automatic renewals in place which makes this completely unnecessary
- bkolobara 1y agoIt still helps sometimes. I was changing some settings on the server and messed up the automatic renewal. Getting an email from them saved me a lot of issues.
- weird-eye-issue 1y agoA company like Postmark should have just given them a free account on the condition they mentioned them at the bottom of emails or something It's a valuable service for the average person to get these emails without having to set up separate monitoring
- bravesoul2 1y agoAt least there is this: > For those who would like to continue receiving expiration notifications, we recommend using a third party service such as Red Sift Certificates Lite (formerly Hardenize). Red Sift’s monitoring service providing expiration emails is free of charge for up to 250 certificates. More monitoring options can be found here.
- jojobas 1y agoThey could just include a local notifier in their scripts, would probably satisfy the majority of users.
- kevincox 1y agoI don't think this is nearly as effective of a solution. It stops working if the script is crashing before hitting the notification, it doesn't work if you break your email setup (which for an email that almost never sends is likely to go unnoticed forever) it also fails if you accidentally stop running a script or remove the certificate from your config entirely.
- jojobas 1y agoYou could have a weekly canary email as well.
- weird-eye-issue 1y agoYeah and people could just use rsync instead of Dropbox
- cosmodev 1y agoI was using this with Certbot for 17 different domains it's a bit sad to see it go. I’m not even sure if I ever relied on the notifications, but just knowing it existed gave some peace of mind.
- sltr 1y agoDIY monitoring: $ curl https://example.com -vI --stderr - | grep "expire date" * expire date: Jan 15 23:59:59 2026 GMT
- Aachen 1y agoI hope they don't send another 20 emails at random intervals across two months to notify me of this now...
- smjburton 1y agoIt's unfortunate to see this go away, but understandable given the costs involved. Another option is to run Caddy as a web server, which provides automatic cert renewal (https://caddyserver.com/docs/automatic-https https://caddyserver.com/docs/automatic-https). If notifications are still important, they also provide an event subscription system (https://github.com/caddyserver/certmagic#events https://github.com/caddyserver/certmagic#events) so you can subscribe to cert-related events, run custom code, trigger event handlers, etc.
- builtsimple 1y agoThis is a smart move. The amount of infrastructure complexity for what's essentially a band-aid for poor automation practices wasn't worth it. We migrated ~800 domains to LE back in 2019 and initially relied heavily on those expiration emails as a safety net. But honestly, they became more of a crutch than a help. Once we implemented proper monitoring with Prometheus + cert-manager, we haven't had a single cert expire unexpectedly. The privacy angle is interesting too. I hadn't considered how much PII they were sitting on just for this feature. With GDPR and similar regulations, that's a significant liability for what amounts to "your cron job didn't run" notifications. For anyone panicking about this: if you're still depending on email notifications for cert renewal in 2025, this is your wake-up call to implement actual monitoring. Even a simple bash script that checks cert expiry dates and posts to a Slack webhook would be more reliable than email notifications. Curious what their infrastructure costs actually were for this. "Tens of thousands per year" seems low for managing millions of emails, but I guess if it's just queuing jobs to an email service provider, that tracks.