8 ms·

A Unix shell script implementing ACME client protocol

125 points by ausjke 9y ago

daurnimator 9y ago
But it does have dependencies (as any shell script tends to). e.g. it requires openssl for computing hmacs; and it needs either curl or wget for doing http requests. Also, Why would someone prefer this over https://github.com/lukas2511/dehydrated https://github.com/lukas2511/dehydrated ?
- deleted 9y ago
  [deleted]
- drudru11 9y ago
  +1 for dehydrate - been using it for a long time
- jtl999 9y ago
  Indeed. For a long time I was using my own fork of simp_le, but the code wasn't easy to split the file writing logic into per domain directories for certificates, etc.
- moviuro 9y ago
  L1: #!/usr/bin/env bash Bash is GPL Licensed and does not ship with most UNIX-like systems. *BSDs don't ship it, macOS is stuck in the past, etc. As a BSD user myself, I install curl(1), but not bash(1) on my machines (or jails). OpenSSL (an alias of LibreSSL when applicable - e.g. OpenBSD) is standard and available on virtually all systems.
deleted 9y ago
[deleted]
tlrobinson 9y ago
A simple (6000 line) shell script.
- bigiain 9y ago
  "With sufficient thrust, pigs fly just fine. However, this is not necessarily a good idea." (hat tip to the authors of RFC1925)
  - jwilk 9y ago
    https://tools.ietf.org/html/rfc1925 https://tools.ietf.org/html/rfc1925
- bb88 9y ago
  I just read through the top 1000 or so lines. Yes it's shell script. No it's not grokable if something goes wrong with it. E.g.: _supported_vtypes="$(echo "$response" | _egrep_o "\"challenges\":\[[^]]*]" | tr '{' "\n" | grep type | cut -d '"' -f 4 | tr "\n" ' ')"
- oblio 9y ago
  + extensions, see the DNS stuff.
ilikepi 9y ago
I'm curious to know if anyone else has looked specifically at the stateless mode this script uses. I've pondered it a bit, and it seems reasonable to me superficially, but I haven't invested much time into deeply understanding ACME. I'm wondering if using stateless mode opens one up to any sort of risk that is not present in the other modes. EDIT: clarification; typo
- tialaramex 9y ago
  Yes, this adds a risk which you might judge worth taking. ACME http-01 validations involve asking an HTTP server for a resource in the .well-known/ reserved URI space with an arbitrary token name, and expecting a reply which contains the token AND a magic value associated with an ACME account. Ordinarily one configures the server manually each time to respond to requests for a token you know will be used for a single ACME validation you want to succeed. "Stateless" mode configures the web server to always reply saying the validation is OK for your ACME account, to any request. Bad Guys can't just use this stateless configuration to get certificates because they don't own your ACME account, if they try to use _their_ ACME account, the validations fail because "stateless" is configured for a single account. However, if bad guys get your ACME account private key or trick you into configuring one they know, with "stateless" mode they can request certificates at any time and your server will validate the requests automatically.
donpdonp 9y ago
Launching those binaries (openssl, curl) seems like a point of failure, not to mention the cough unique challenges of doing simple things like managing an array of strings in a language like bash. My idea of the simplest letsencrypt client is lego. One binary to rule them all. https://github.com/xenolf/lego https://github.com/xenolf/lego
- tehlike 9y ago
  Lego + NGINX + Kubernetes works very well for most projects.
- gpm 9y ago
  At the top of the readme > This is a work in progress. Please do NOT run this on a production server and please report any bugs you find! And it's been in development for at least two years per the dates on the files - with hundreds of commits made at a reasonably continuous pace judging by the code frequency graph. Either that's a ridiculously conservative warning, or I think there is a problem with something in this approach.
  - deleted 9y ago
    [deleted]
  - xenolf 9y ago
    It's a rediculously conservative warning. :)
- aplorbust 9y ago
  "One binary to rule them all." But if one is using a shell to launch that binary, then IMO there are two userland binaries: the shell and the binary being launched. In that case, the second binary depends on the first. Lets assume one really wants to use only a single binary for whatever reason. Could she use busybox? Does busybox have an openssl-like function? I am not sure. However I can confirm that the BSD equivalent of busybox can easily be compiled, statically, to include an http client, openssl and the other utilities needed for these shell scripts. I use such a binary for daily work. Note I am not a LetsEncrypt user and have no comment on ACME or these shell scripts or other programs. I am only commenting as an avid shell scripter and user of static, "multi-call" binaries.
rmetzler 9y ago
I use acme.sh in production on several servers. When I decided to use it, I was looking for something I could read and understand in less than a day. The one file approach helped this. Lots of code bases are designed for easy change, which often means many small files, so you find the right line easier if you know which file you have to go to. Because of historic reasons, I use it mainly with Apache, which I configure manually. I know there are options for automatic update of the config, but these weren't there when I read the script first. For me personally there are two problems - maybe these are even addressed in new versions of acme.sh, I haven't checked yet. 1. Apache won't load the vhost config for https if it can't find the key and cert files. Which now means, I have to use a http-only config and another full featured http/https when the challenge is done. 2. My typos in (sub-)domains are one of the main sources of confusion when I try to get a new certificate. I think using dig or host and curl it should be possible to warn the user in this circumstances.
- reacweb 9y ago
  Maybe you could write a small script that checks the parameters using your rules, then calls acme.sh passing these parameters?
mortenlarsen 9y ago
I have written a small plugin for the acme.sh dnsapi that works with my DNS setup. I run acme.sh in a FreeBSD jail (acme-client). It writes files that are picked up by another jail (acme-dns) that runs nsd. This jail is NOT one of my main authoritative name servers. It only runs _acme.mydomain.tld containing records like: _acme-challenge.test IN TXT XXXXXXXXXXXXXXX These records are pointed to from my main name servers with records like: _acme-challenge.test NS acme-dns.mydomain.tld. (CNAME seems to work too, I will probably switch to that) All this gets me the following benefits: a) Everything runs restricted by jails that only run for about 10 seconds when issuing or renewing certificates. b) No need for the service to be publicly accessible. (no issues with firewalls, no need for public IP's) c) No need for the service to be some kind of web-server (think smtp, imap, irc, xmpp, etc.) d) The service does not need a public "A" record. e) No risk of me or the script messing up any live/production configuration. f) The only thing that needs to accept inbound connections is the "acme-dns" jail on port 53 for about 10 seconds when it is running. There are still some things I need to find a good solution for. Like easier distribution of certs, keys, etc. I also want to generate the private keys elsewhere and only give the CSR's to the acme-client jail. (If this is possible with ACME. I think it is.) This setup is not yet complete and I am still experimenting, but it seems to work well.
- voltagex_ 9y ago
  That sounds like there's a lot of moving parts but it's interesting. Is there any part of it you can share?
  - mortenlarsen 9y ago
    Maybe when it is a bit more complete. Currently too much is hardcoded for it to be useful outside my setup. But if you look at one of the existing dnsapi plugins you will notice that only two functions need to be implemented. The rest is described above. BTW: Remember to use letsencrypt-staging for testing. Also, have a look at: https://blog.crashed.org/letsencrypt-in-freebsd-org/ https://blog.crashed.org/letsencrypt-in-freebsd-org/ It was this that inspired me in the first place. I just added the separate subdomain and separate nameserver concept.
Emptysister 9y ago
what about https://github.com/diafygi/acme-tiny https://github.com/diafygi/acme-tiny ?
- Tepix 9y ago
  Yep. Works great and has a limit of 200 lines of code. Support for ACME2 will be a different project (possibly with a limit of 256 lines of code from what I've heard)
drinchev 9y ago
Wow 6k lines of bash script with 1.6k tests file [1]. Looks a bit nightmarish to maintain. 1 : https://github.com/Neilpang/acmetest/blob/master/letest.sh https://github.com/Neilpang/acmetest/blob/master/letest.sh
- zingmars 9y ago
  The test file seems fun, but overall it's not that bad. Bash is a fairly simple to parse syntax-wise, so as long as you know your coreutils and you know what you're looking for, making simple changes is fairly easy.
  - drinchev 9y ago
    It doesn't look so simple, but definitely looks quirky. Take a look at how the tests are invoked : for t in $(grep ^le_test_ $FILE_NAME | cut -d '(' -f 1) do if [ -z "$CASE" ] ; then __green "Progress: " [ "$_ret" = "0" ] && __green "$_ret" || __red "$_ret" __green "/$num/$total" printf "\n" num=$(_math $num + 1) fi ... It basically parses the very same file and checks for functions with the name `le_test`. This means that a random comment containing `le_test_` will break the whole script. You need lots of discipline in order to make things work. Bash works perfectly with "Write programs that do one thing and do it well". Wouldn't be so surprised if this was a bunch of small test cases separated by file.
    - LambdaComplex 9y ago
      Looks like it's only grepping for lines that start with le_test_. A comment would start with #
    - moviuro 9y ago
      ^ is beginning of line, so you'd need: le_test_xxx # <- this is not a comment anyway # le_test_xxx does something nice. # <- doesn't match ^le_test_
- defgeneric 9y ago
teekert 9y ago
Still wondering what is this adds over a script like this, running weekly using cron: ./letsencrypt-auto --renew nginx reload
- oblio 9y ago
  You mean certbot, the official client? It's not supported on Amazon Linux, for example.
- majewsky 9y ago
  What if your service is not HTTP? I use LE certs with nginx, Prosody (XMPP) and Mumble (voice chat).
cm2187 9y ago
The list of DNS APIs supported is impressive. This is the main problem with automating wildcard certificates (DNS provider specific APIs that makes using a single client challenging).
AdmiralAsshat 9y ago
I thought I new Bash pretty well, but the leading underlines in function names is throwing me for a loop. Is this a unique stylistic thing to the author, or is _foo() versus foo() as a function name following an established convention?
- rmgraham 9y ago
  I asked someone when I saw them doing similar things. They adopted the style after reading Google's style guide for shell scripts. Not sure if that's where this author got it from, but it's one possible source.
- xiaomai 9y ago
  I think this is the perl-style "private" convention. Those functions are helpers that aren't intended to be used on their own so he prefixes them with _.
TekMol 9y ago
ACME seems so overcomplicated. How about just making the domain owner publish which certs are valid on a url like /certificates.txt Then LetsEncrypt could periodically check if the certs they issued are still endorsed by the domain owner. And if not revoke them.
- gruez 9y ago
  >And if not revoke them. Certificate revocation is not guaranteed. Yes, there's CRLs and OSCP, but there are plenty of clients that don't do either of those. Not to mention how big CRLs will get if certificates can be issued and revoked for free.
- zaarn 9y ago
  Access to this /certificates.txt would need to be done over a TLS connection as an attacker could insert their own certs here. ACME avoids this by associating a specific CSR to a response, so an attacker could not insert their own certificate in the middle of the process and get it signed.
Sir_Cmpwn 9y ago
Acme clients are a space where there are a lot of bad choices. My favorite acme client is this: https://kristaps.bsd.lv/acme-client/ https://kristaps.bsd.lv/acme-client/ It is: - Small - Written in C - Can be statically linked against LibreSSL and curl No nonsense, does what it says on the tin, works on many operating systems.
brynet 9y ago
I probably wouldn't trust implementing important network speaking services as a shell script, potentially running as root. OpenBSD's acme-client [0] is a fork of Kristaps Dzonsons' project (formerly letskencrypt), it's a properly privilege separated ACME v1 client, written in C, using pledge(2) on OpenBSD, libseccomp on Linux. https://kristaps.bsd.lv/acme-client/ https://kristaps.bsd.lv/acme-client/ [0] https://man.openbsd.org/acme-client https://man.openbsd.org/acme-client
- Panino 9y ago
  acme-client is great and I use it for RSA certs, preferring it over acme-tiny [0]. However I prefer ECDSA (until we get EdDSA), so for those certs I use acme-tiny. https://github.com/diafygi/acme-tiny https://github.com/diafygi/acme-tiny
peterwwillis 9y ago
I am not going to hate on this tool for being big, because it's portable, that kind of comes with the territory. However, if you wanted something similar that was more portable with less dependencies, you could write it in Perl. There are pure perl libraries for whatever you need, including crypto, and Perl runs on systems like Windows without the need for an extra subsystem or environment.

125 points by ausjke 9y ago

But it does have dependencies (as any shell script tends to). e.g. it requires openssl for computing hmacs; and it needs either curl or wget for doing http requests. Also, Why would someone prefer this over https://github.com/lukas2511/dehydrated https://github.com/lukas2511/dehydrated ?

deleted 9y ago
[deleted]
drudru11 9y ago
+1 for dehydrate - been using it for a long time
jtl999 9y ago
Indeed. For a long time I was using my own fork of simp_le, but the code wasn't easy to split the file writing logic into per domain directories for certificates, etc.
moviuro 9y ago
L1: #!/usr/bin/env bash Bash is GPL Licensed and does not ship with most UNIX-like systems. *BSDs don't ship it, macOS is stuck in the past, etc. As a BSD user myself, I install curl(1), but not bash(1) on my machines (or jails). OpenSSL (an alias of LibreSSL when applicable - e.g. OpenBSD) is standard and available on virtually all systems.

deleted 9y ago

[deleted]

tlrobinson 9y ago

A simple (6000 line) shell script.

bigiain 9y ago
"With sufficient thrust, pigs fly just fine. However, this is not necessarily a good idea." (hat tip to the authors of RFC1925)
- jwilk 9y ago
  https://tools.ietf.org/html/rfc1925 https://tools.ietf.org/html/rfc1925
bb88 9y ago
I just read through the top 1000 or so lines. Yes it's shell script. No it's not grokable if something goes wrong with it. E.g.: _supported_vtypes="$(echo "$response" | _egrep_o "\"challenges\":\[[^]]*]" | tr '{' "\n" | grep type | cut -d '"' -f 4 | tr "\n" ' ')"
oblio 9y ago
+ extensions, see the DNS stuff.

ilikepi 9y ago

I'm curious to know if anyone else has looked specifically at the stateless mode this script uses. I've pondered it a bit, and it seems reasonable to me superficially, but I haven't invested much time into deeply understanding ACME. I'm wondering if using stateless mode opens one up to any sort of risk that is not present in the other modes. EDIT: clarification; typo

tialaramex 9y ago
Yes, this adds a risk which you might judge worth taking. ACME http-01 validations involve asking an HTTP server for a resource in the .well-known/ reserved URI space with an arbitrary token name, and expecting a reply which contains the token AND a magic value associated with an ACME account. Ordinarily one configures the server manually each time to respond to requests for a token you know will be used for a single ACME validation you want to succeed. "Stateless" mode configures the web server to always reply saying the validation is OK for your ACME account, to any request. Bad Guys can't just use this stateless configuration to get certificates because they don't own your ACME account, if they try to use _their_ ACME account, the validations fail because "stateless" is configured for a single account. However, if bad guys get your ACME account private key or trick you into configuring one they know, with "stateless" mode they can request certificates at any time and your server will validate the requests automatically.

donpdonp 9y ago

Launching those binaries (openssl, curl) seems like a point of failure, not to mention the cough unique challenges of doing simple things like managing an array of strings in a language like bash. My idea of the simplest letsencrypt client is lego. One binary to rule them all. https://github.com/xenolf/lego https://github.com/xenolf/lego

tehlike 9y ago
Lego + NGINX + Kubernetes works very well for most projects.
gpm 9y ago
At the top of the readme > This is a work in progress. Please do NOT run this on a production server and please report any bugs you find! And it's been in development for at least two years per the dates on the files - with hundreds of commits made at a reasonably continuous pace judging by the code frequency graph. Either that's a ridiculously conservative warning, or I think there is a problem with something in this approach.
- deleted 9y ago
  [deleted]
- xenolf 9y ago
  It's a rediculously conservative warning. :)
aplorbust 9y ago
"One binary to rule them all." But if one is using a shell to launch that binary, then IMO there are two userland binaries: the shell and the binary being launched. In that case, the second binary depends on the first. Lets assume one really wants to use only a single binary for whatever reason. Could she use busybox? Does busybox have an openssl-like function? I am not sure. However I can confirm that the BSD equivalent of busybox can easily be compiled, statically, to include an http client, openssl and the other utilities needed for these shell scripts. I use such a binary for daily work. Note I am not a LetsEncrypt user and have no comment on ACME or these shell scripts or other programs. I am only commenting as an avid shell scripter and user of static, "multi-call" binaries.

rmetzler 9y ago

I use acme.sh in production on several servers. When I decided to use it, I was looking for something I could read and understand in less than a day. The one file approach helped this. Lots of code bases are designed for easy change, which often means many small files, so you find the right line easier if you know which file you have to go to. Because of historic reasons, I use it mainly with Apache, which I configure manually. I know there are options for automatic update of the config, but these weren't there when I read the script first. For me personally there are two problems - maybe these are even addressed in new versions of acme.sh, I haven't checked yet. 1. Apache won't load the vhost config for https if it can't find the key and cert files. Which now means, I have to use a http-only config and another full featured http/https when the challenge is done. 2. My typos in (sub-)domains are one of the main sources of confusion when I try to get a new certificate. I think using dig or host and curl it should be possible to warn the user in this circumstances.

reacweb 9y ago
Maybe you could write a small script that checks the parameters using your rules, then calls acme.sh passing these parameters?

mortenlarsen 9y ago

I have written a small plugin for the acme.sh dnsapi that works with my DNS setup. I run acme.sh in a FreeBSD jail (acme-client). It writes files that are picked up by another jail (acme-dns) that runs nsd. This jail is NOT one of my main authoritative name servers. It only runs _acme.mydomain.tld containing records like: _acme-challenge.test IN TXT XXXXXXXXXXXXXXX These records are pointed to from my main name servers with records like: _acme-challenge.test NS acme-dns.mydomain.tld. (CNAME seems to work too, I will probably switch to that) All this gets me the following benefits: a) Everything runs restricted by jails that only run for about 10 seconds when issuing or renewing certificates. b) No need for the service to be publicly accessible. (no issues with firewalls, no need for public IP's) c) No need for the service to be some kind of web-server (think smtp, imap, irc, xmpp, etc.) d) The service does not need a public "A" record. e) No risk of me or the script messing up any live/production configuration. f) The only thing that needs to accept inbound connections is the "acme-dns" jail on port 53 for about 10 seconds when it is running. There are still some things I need to find a good solution for. Like easier distribution of certs, keys, etc. I also want to generate the private keys elsewhere and only give the CSR's to the acme-client jail. (If this is possible with ACME. I think it is.) This setup is not yet complete and I am still experimenting, but it seems to work well.

voltagex_ 9y ago
That sounds like there's a lot of moving parts but it's interesting. Is there any part of it you can share?
- mortenlarsen 9y ago
  Maybe when it is a bit more complete. Currently too much is hardcoded for it to be useful outside my setup. But if you look at one of the existing dnsapi plugins you will notice that only two functions need to be implemented. The rest is described above. BTW: Remember to use letsencrypt-staging for testing. Also, have a look at: https://blog.crashed.org/letsencrypt-in-freebsd-org/ https://blog.crashed.org/letsencrypt-in-freebsd-org/ It was this that inspired me in the first place. I just added the separate subdomain and separate nameserver concept.

Emptysister 9y ago

what about https://github.com/diafygi/acme-tiny https://github.com/diafygi/acme-tiny ?

Tepix 9y ago
Yep. Works great and has a limit of 200 lines of code. Support for ACME2 will be a different project (possibly with a limit of 256 lines of code from what I've heard)

drinchev 9y ago

Wow 6k lines of bash script with 1.6k tests file [1]. Looks a bit nightmarish to maintain. 1 : https://github.com/Neilpang/acmetest/blob/master/letest.sh https://github.com/Neilpang/acmetest/blob/master/letest.sh

zingmars 9y ago
The test file seems fun, but overall it's not that bad. Bash is a fairly simple to parse syntax-wise, so as long as you know your coreutils and you know what you're looking for, making simple changes is fairly easy.
- drinchev 9y ago
  It doesn't look so simple, but definitely looks quirky. Take a look at how the tests are invoked : for t in $(grep ^le_test_ $FILE_NAME | cut -d '(' -f 1) do if [ -z "$CASE" ] ; then __green "Progress: " [ "$_ret" = "0" ] && __green "$_ret" || __red "$_ret" __green "/$num/$total" printf "\n" num=$(_math $num + 1) fi ... It basically parses the very same file and checks for functions with the name `le_test`. This means that a random comment containing `le_test_` will break the whole script. You need lots of discipline in order to make things work. Bash works perfectly with "Write programs that do one thing and do it well". Wouldn't be so surprised if this was a bunch of small test cases separated by file.
  - LambdaComplex 9y ago
    Looks like it's only grepping for lines that start with le_test_. A comment would start with #
  - moviuro 9y ago
    ^ is beginning of line, so you'd need: le_test_xxx # <- this is not a comment anyway # le_test_xxx does something nice. # <- doesn't match ^le_test_
defgeneric 9y ago

teekert 9y ago

Still wondering what is this adds over a script like this, running weekly using cron: ./letsencrypt-auto --renew nginx reload

oblio 9y ago
You mean certbot, the official client? It's not supported on Amazon Linux, for example.
majewsky 9y ago
What if your service is not HTTP? I use LE certs with nginx, Prosody (XMPP) and Mumble (voice chat).

cm2187 9y ago

The list of DNS APIs supported is impressive. This is the main problem with automating wildcard certificates (DNS provider specific APIs that makes using a single client challenging).

AdmiralAsshat 9y ago

I thought I new Bash pretty well, but the leading underlines in function names is throwing me for a loop. Is this a unique stylistic thing to the author, or is _foo() versus foo() as a function name following an established convention?

rmgraham 9y ago
I asked someone when I saw them doing similar things. They adopted the style after reading Google's style guide for shell scripts. Not sure if that's where this author got it from, but it's one possible source.
xiaomai 9y ago
I think this is the perl-style "private" convention. Those functions are helpers that aren't intended to be used on their own so he prefixes them with _.

TekMol 9y ago

ACME seems so overcomplicated. How about just making the domain owner publish which certs are valid on a url like /certificates.txt Then LetsEncrypt could periodically check if the certs they issued are still endorsed by the domain owner. And if not revoke them.

gruez 9y ago
>And if not revoke them. Certificate revocation is not guaranteed. Yes, there's CRLs and OSCP, but there are plenty of clients that don't do either of those. Not to mention how big CRLs will get if certificates can be issued and revoked for free.
zaarn 9y ago
Access to this /certificates.txt would need to be done over a TLS connection as an attacker could insert their own certs here. ACME avoids this by associating a specific CSR to a response, so an attacker could not insert their own certificate in the middle of the process and get it signed.

Sir_Cmpwn 9y ago

Acme clients are a space where there are a lot of bad choices. My favorite acme client is this: https://kristaps.bsd.lv/acme-client/ https://kristaps.bsd.lv/acme-client/ It is: - Small - Written in C - Can be statically linked against LibreSSL and curl No nonsense, does what it says on the tin, works on many operating systems.

brynet 9y ago

I probably wouldn't trust implementing important network speaking services as a shell script, potentially running as root. OpenBSD's acme-client [0] is a fork of Kristaps Dzonsons' project (formerly letskencrypt), it's a properly privilege separated ACME v1 client, written in C, using pledge(2) on OpenBSD, libseccomp on Linux. https://kristaps.bsd.lv/acme-client/ https://kristaps.bsd.lv/acme-client/ [0] https://man.openbsd.org/acme-client https://man.openbsd.org/acme-client

Panino 9y ago
acme-client is great and I use it for RSA certs, preferring it over acme-tiny [0]. However I prefer ECDSA (until we get EdDSA), so for those certs I use acme-tiny. https://github.com/diafygi/acme-tiny https://github.com/diafygi/acme-tiny

peterwwillis 9y ago

I am not going to hate on this tool for being big, because it's portable, that kind of comes with the territory. However, if you wanted something similar that was more portable with less dependencies, you could write it in Perl. There are pure perl libraries for whatever you need, including crypto, and Perl runs on systems like Windows without the need for an extra subsystem or environment.