9 ms·

Writing Small CLI Programs in Common Lisp

174 points by reikonomusha 6y ago

e40 6y ago
In Allegro Common Lisp it's #! /path/to/mlisp-or-alisp -#! ...lisp code...
vram22 6y ago
There is also scsh, the Scheme shell.
lambdaba 6y ago
For those unfamiliar with Common Lisp there is also Babashka which allows for Clojure shell scripting without the JVM startup time. https://github.com/babashka/babashka https://github.com/babashka/babashka
- nerdponx 6y ago
  Common Lisp itself has Roswell, which I am disappointed to see is not even mentioned in the article. https://github.com/roswell/roswell https://github.com/roswell/roswell
  - Jach 6y ago
    It's sort of mentioned indirectly at the beginning in that source-code scripts that use a shebang, or make use of exit, can introduce development barriers. I've not written many CL scripts but I like Roswell too, and haven't found those barriers significant/odd to work around. I agree Roswell is definitely worth checking out. (Plus at least for me it gets even more use by letting me trivially run my tests with alternate CL implementations...) Like, I didn't even consider including "some ugly reader macros" to handle the shebang issue, since my approach has been to comment it out while developing and uncomment once I was ready to use it as a script. But I think this simple macro I put just now in my $HOME/.sbclrc file should suffice: (set-dispatch-macro-character #\# #\! (lambda (stream subchar arg) (declare (ignore subchar arg)) (read-line stream nil (values) t) (values))) And in some programs (not scripts) where I make use of uiop:quit, I just don't call those paths during dev, but this is the same approach the article has differentiating calling run vs toplevel.
    - vindarel 6y ago
      About uiop:quit, that sometimes are in the middle of a script: lately I have wrapped it around a check of the nature of the terminal: if the terminal is dumb ($TERM), we are very likely inside Emacs/Slime, so we don't quit. https://github.com/vindarel/termp https://github.com/vindarel/termp
- Kelamir 6y ago
  Judging from the logo, the name is derived from the Russian word Babushka, which stands for a granny.
  - lambdaba 6y ago
    Indeed, it's a lovely name
- bmitc 6y ago
  A quote from that link: > Life's too short to remember how to write Bash code. Hard agree.
  - idolaspecus 6y ago
    Life’s too short to not learn how to write bash code.
  - exdsq 6y ago
    I love writing bash!
    - deleted 6y ago
      [deleted]
  - fctorial 6y ago
    I'm astonished that we haven't moved past bash.
    - blacktriangle 6y ago
      I'm not. "Everything is a string" is an incredibly powerful worldview for tying systems together where those systems never put any thought into interoperability. Any realistic bash alternative would either need to rewrite the entire user land with a more structured exchange format like JSON or .NET objects, or ship with adapters for most every command line tool we use.
  - JasonFruit 6y ago
    I really feel bad that I don't write more Bash, but every time I learn some, it's gone. I cannot retain that language, and I don't understand why: I'm a language gourmand, in general. But I throw Python, Scheme, or even Nim at little scripts every time, because I have to look everything up if I do it in Bash.
    - azalemeth 6y ago
      I've always internally thought that there are some languages that our brains just _don't_ like. I used to think that there was "something wrong" with me for finding C++ verbose and C expressively dangerous, or to find myself reaching for multi-line awk, sed, and bash scripts that did _stupid_ things when I "knew" that a better solution existed but I didn't know how to _express_ that solution concisely. Perhaps a more concrete example would be this: I don't use python often and some of its syntactical quirks get me every time -- like, for example, None as a method to insert a new axis by convention in numpy (so much so that numpy define np.newaxis as an alias to it). It makes total sense but at the same time, the fact that the language likes having what almost looks like a C NULL as a method to insert a dimension seems...wrong, like it should be a syntax error. I've now just come to realise that hey: if you can't express this concept well in one language, and you _can_ in another, it's well worth prototyping the solution in the right domain and one could always write something in a "sensible" language later if it turns out to work.
abhinav22 6y ago
Does this work on Windows? If i recall, there were issues with lisp interfacing with windows (but uiop may have solved it already - I have been to get my programs to work in Windows without issue)
- aidenn0 6y ago
  You probably won't be using a Makefile and shell-scripts on windows for doing the building, but yes it works fine. SBCL had scary "windows not supported" messages for a long time, but that was mainly because none of the core sbcl team had windows machines, so any bugs that didn't reproduce under wine weren't going to get support from them. LispWorks and Allegro both have had first-class support for windows for decades. As far as UIOP: UIOP provides portability interfaces that make it less likely that something developed on linux will be DOA on windows, as well as smoothing out differences between implementations. If were developing on a single implementation for just windows nothing in UIOP is really necessary (but portability is always a "nice to have" feature that UIOP does provide).
  - abhinav22 6y ago
    Thanks
- pjmlp 6y ago
  LispWorks and Allegro work just fine on Windows.
Narishma 6y ago
I wish they'd mention the size of the binaries produced this way, and whether they are standalone.
- 13415 6y ago
  I was wondering about that, too. Last time I checked SBCL binaries weren't exactly small, more around 40MB.
  - wglb 6y ago
    Typically; if you use the :compress feature, more like 14 meg.
- aidenn0 6y ago
  The following data is for standalone executables (modulo things like libc and libm that most Linux executables depend on) Columns are: - apparent size (I run FS compression so the actual on-disk size is nearly the same between the compressed and uncompressed) - Executable name: $IMPLEMENTATION.{big|compress} implementation here is either SBCL or CCL. "Big" means I load the drakma library before saving the image (which pulls in things like unicode tables). "compress" means I attempt to enable image compression; the numbers make it looks my system's CCL does not support compression. - Mean time to run a program that quits immediately as a startup time proxy; sbcl pays a big penalty for compression as without compression it just MMAPs the image in so doesn't ever load most of the image when you quit immediately. /bin/true is included for a minimal "fast and tiny C program" as a comparison. 31K /bin/true 85% .0004 43M sbcl 99% .0036 29M ccl 93% .0053 58M sbcl.big 99% .0049 14M sbcl.compress 99% .2012 44M ccl.big 95% .0071 44M ccl.compress 95% .0071
  - nonameiguess 6y ago
    /bin/true will still forever irk me: https://twitter.com/rob_pike/status/966896123548872705 https://twitter.com/rob_pike/status/966896123548872705
  - aidenn0 6y ago
PuercoPop 6y ago
Another alternative for shell scripting in CL there is https://www.cliki.net/cl-launch https://www.cliki.net/cl-launch
verdverm 6y ago
I've recently been using Python3 + argparse for small single file utilities. Used to use bash but have probably converted now with how easy calling external programs is (compared to something like Go) I'd have a hard time selecting LISP for anything others would use (which is almost all the software I write) unless I worked in a company that was using LISP regularly.
- CraigJPerry 6y ago
  I read this entire thread thinking - this is neat and all but it’s a handful of lines of almost english language in a .py file. Click is a neat upgrade from argparse if you’re ever tempted (assuming something like “pip install —-user” is viable in your situation which isn’t true for everyone). I’ve seen, but haven’t used https://typer.tiangolo.com/ https://typer.tiangolo.com/ - i have used his FastAPI and thought that was nicely done (even if python async is a bit annoying to test). Talking of python cli libs I haven’t tried but will do one day - rich: https://github.com/willmcgugan/rich https://github.com/willmcgugan/rich
  - verdverm 6y ago
    I try to avoid pip and dependencies for these python scripts. This goal has it's limits. Click is nice if you want a command with multiple sub commands. I use Go + Cobra at that point via https://github.com/hofstadter-io/hofmod-cli https://github.com/hofstadter-io/hofmod-cli The human understanding is a big reason why python has gained my favor over bash
  - p_l 6y ago
    Well, but then you'd be writing python. (and having to maintain python runtime environment, which tends to be much less stable than CL)
- barbazoo 6y ago
  I recently switched to Click from argparse and like it better so far.
  - 6y ago
cb321 6y ago
Nim [1] has cligen [2]. An output binary size for the trivial: proc f = discard import cligen dispatch f is about 708 KiB on Linux, 621 KiB stripped. And if you like colors, hldiff [3] is not a terrible example. [1] https://nim-lang.org/ https://nim-lang.org/ [2] https://github.com/c-blake/cligen https://github.com/c-blake/cligen [3] https://github.com/c-blake/hldiff https://github.com/c-blake/hldiff
junke 6y ago
I would say: don't do it. Nowadays I'd rather use Common Lisp as a glue for other scripts rather than invoking it along with other programs with shell scripts. Plus the need for scripts diminues when you can write a function on typed objects (all scripts are parsers). I also try to seek other IPC mechanisms when possible, such as talking to daemons through DBus or JSON-RPC depending on what is available. edit: this story is currently on the front page of HN too, https://news.ycombinator.com/item?id=26491858 https://news.ycombinator.com/item?id=26491858 (Do not use redirection characters in your shell prompt), the neverending examples of how shell scripting makes your life miserable (sure it's helpful, but can't we have nicer things?)
mark_l_watson 6y ago
Nice writeup! I have fairly recently written CLI programs in Common Lisp, GAMBIT Scheme, and Swift. All good experiences. To be transparent, I don't write bash scripts more than a few lines. For something short, I would prefer Ruby. EDIT: in the last month or two, I have experimented with Babashka (mentioned in this thread) also.
vindarel 6y ago
Common Lisp is awesome: image-based development, compile-time type checking, self-contained executables, stability… https://github.com/CodyReichert/awesome-cl/ https://github.com/CodyReichert/awesome-cl/ https://lispcookbook.github.io/cl-cookbook/editor-support.html https://lispcookbook.github.io/cl-cookbook/editor-support.ht... (Atom, Sublime, VSCode, Jupyter…) https://lisp-lang.org/success/ https://lisp-lang.org/success/ (and https://github.com/azzamsa/awesome-lisp-companies https://github.com/azzamsa/awesome-lisp-companies)

174 points by reikonomusha 6y ago

e40 6y ago

In Allegro Common Lisp it's #! /path/to/mlisp-or-alisp -#! ...lisp code...

vram22 6y ago

There is also scsh, the Scheme shell.

lambdaba 6y ago

For those unfamiliar with Common Lisp there is also Babashka which allows for Clojure shell scripting without the JVM startup time. https://github.com/babashka/babashka https://github.com/babashka/babashka

nerdponx 6y ago
Common Lisp itself has Roswell, which I am disappointed to see is not even mentioned in the article. https://github.com/roswell/roswell https://github.com/roswell/roswell
- Jach 6y ago
  It's sort of mentioned indirectly at the beginning in that source-code scripts that use a shebang, or make use of exit, can introduce development barriers. I've not written many CL scripts but I like Roswell too, and haven't found those barriers significant/odd to work around. I agree Roswell is definitely worth checking out. (Plus at least for me it gets even more use by letting me trivially run my tests with alternate CL implementations...) Like, I didn't even consider including "some ugly reader macros" to handle the shebang issue, since my approach has been to comment it out while developing and uncomment once I was ready to use it as a script. But I think this simple macro I put just now in my $HOME/.sbclrc file should suffice: (set-dispatch-macro-character #\# #\! (lambda (stream subchar arg) (declare (ignore subchar arg)) (read-line stream nil (values) t) (values))) And in some programs (not scripts) where I make use of uiop:quit, I just don't call those paths during dev, but this is the same approach the article has differentiating calling run vs toplevel.
  - vindarel 6y ago
    About uiop:quit, that sometimes are in the middle of a script: lately I have wrapped it around a check of the nature of the terminal: if the terminal is dumb ($TERM), we are very likely inside Emacs/Slime, so we don't quit. https://github.com/vindarel/termp https://github.com/vindarel/termp
Kelamir 6y ago
Judging from the logo, the name is derived from the Russian word Babushka, which stands for a granny.
- lambdaba 6y ago
  Indeed, it's a lovely name
bmitc 6y ago
A quote from that link: > Life's too short to remember how to write Bash code. Hard agree.
- idolaspecus 6y ago
  Life’s too short to not learn how to write bash code.
- exdsq 6y ago
  I love writing bash!
  - deleted 6y ago
    [deleted]
- fctorial 6y ago
  I'm astonished that we haven't moved past bash.
  - blacktriangle 6y ago
    I'm not. "Everything is a string" is an incredibly powerful worldview for tying systems together where those systems never put any thought into interoperability. Any realistic bash alternative would either need to rewrite the entire user land with a more structured exchange format like JSON or .NET objects, or ship with adapters for most every command line tool we use.
- JasonFruit 6y ago
  I really feel bad that I don't write more Bash, but every time I learn some, it's gone. I cannot retain that language, and I don't understand why: I'm a language gourmand, in general. But I throw Python, Scheme, or even Nim at little scripts every time, because I have to look everything up if I do it in Bash.
  - azalemeth 6y ago
    I've always internally thought that there are some languages that our brains just _don't_ like. I used to think that there was "something wrong" with me for finding C++ verbose and C expressively dangerous, or to find myself reaching for multi-line awk, sed, and bash scripts that did _stupid_ things when I "knew" that a better solution existed but I didn't know how to _express_ that solution concisely. Perhaps a more concrete example would be this: I don't use python often and some of its syntactical quirks get me every time -- like, for example, None as a method to insert a new axis by convention in numpy (so much so that numpy define np.newaxis as an alias to it). It makes total sense but at the same time, the fact that the language likes having what almost looks like a C NULL as a method to insert a dimension seems...wrong, like it should be a syntax error. I've now just come to realise that hey: if you can't express this concept well in one language, and you _can_ in another, it's well worth prototyping the solution in the right domain and one could always write something in a "sensible" language later if it turns out to work.

abhinav22 6y ago

Does this work on Windows? If i recall, there were issues with lisp interfacing with windows (but uiop may have solved it already - I have been to get my programs to work in Windows without issue)

aidenn0 6y ago
You probably won't be using a Makefile and shell-scripts on windows for doing the building, but yes it works fine. SBCL had scary "windows not supported" messages for a long time, but that was mainly because none of the core sbcl team had windows machines, so any bugs that didn't reproduce under wine weren't going to get support from them. LispWorks and Allegro both have had first-class support for windows for decades. As far as UIOP: UIOP provides portability interfaces that make it less likely that something developed on linux will be DOA on windows, as well as smoothing out differences between implementations. If were developing on a single implementation for just windows nothing in UIOP is really necessary (but portability is always a "nice to have" feature that UIOP does provide).
- abhinav22 6y ago
  Thanks
pjmlp 6y ago
LispWorks and Allegro work just fine on Windows.

Narishma 6y ago

I wish they'd mention the size of the binaries produced this way, and whether they are standalone.

13415 6y ago
I was wondering about that, too. Last time I checked SBCL binaries weren't exactly small, more around 40MB.
- wglb 6y ago
  Typically; if you use the :compress feature, more like 14 meg.
aidenn0 6y ago
The following data is for standalone executables (modulo things like libc and libm that most Linux executables depend on) Columns are: - apparent size (I run FS compression so the actual on-disk size is nearly the same between the compressed and uncompressed) - Executable name: $IMPLEMENTATION.{big|compress} implementation here is either SBCL or CCL. "Big" means I load the drakma library before saving the image (which pulls in things like unicode tables). "compress" means I attempt to enable image compression; the numbers make it looks my system's CCL does not support compression. - Mean time to run a program that quits immediately as a startup time proxy; sbcl pays a big penalty for compression as without compression it just MMAPs the image in so doesn't ever load most of the image when you quit immediately. /bin/true is included for a minimal "fast and tiny C program" as a comparison. 31K /bin/true 85% .0004 43M sbcl 99% .0036 29M ccl 93% .0053 58M sbcl.big 99% .0049 14M sbcl.compress 99% .2012 44M ccl.big 95% .0071 44M ccl.compress 95% .0071
- nonameiguess 6y ago
  /bin/true will still forever irk me: https://twitter.com/rob_pike/status/966896123548872705 https://twitter.com/rob_pike/status/966896123548872705
- aidenn0 6y ago

PuercoPop 6y ago

Another alternative for shell scripting in CL there is https://www.cliki.net/cl-launch https://www.cliki.net/cl-launch

verdverm 6y ago

I've recently been using Python3 + argparse for small single file utilities. Used to use bash but have probably converted now with how easy calling external programs is (compared to something like Go) I'd have a hard time selecting LISP for anything others would use (which is almost all the software I write) unless I worked in a company that was using LISP regularly.

CraigJPerry 6y ago
I read this entire thread thinking - this is neat and all but it’s a handful of lines of almost english language in a .py file. Click is a neat upgrade from argparse if you’re ever tempted (assuming something like “pip install —-user” is viable in your situation which isn’t true for everyone). I’ve seen, but haven’t used https://typer.tiangolo.com/ https://typer.tiangolo.com/ - i have used his FastAPI and thought that was nicely done (even if python async is a bit annoying to test). Talking of python cli libs I haven’t tried but will do one day - rich: https://github.com/willmcgugan/rich https://github.com/willmcgugan/rich
- verdverm 6y ago
  I try to avoid pip and dependencies for these python scripts. This goal has it's limits. Click is nice if you want a command with multiple sub commands. I use Go + Cobra at that point via https://github.com/hofstadter-io/hofmod-cli https://github.com/hofstadter-io/hofmod-cli The human understanding is a big reason why python has gained my favor over bash
- p_l 6y ago
  Well, but then you'd be writing python. (and having to maintain python runtime environment, which tends to be much less stable than CL)
barbazoo 6y ago
I recently switched to Click from argparse and like it better so far.
- 6y ago

cb321 6y ago

Nim [1] has cligen [2]. An output binary size for the trivial: proc f = discard import cligen dispatch f is about 708 KiB on Linux, 621 KiB stripped. And if you like colors, hldiff [3] is not a terrible example. [1] https://nim-lang.org/ https://nim-lang.org/ [2] https://github.com/c-blake/cligen https://github.com/c-blake/cligen [3] https://github.com/c-blake/hldiff https://github.com/c-blake/hldiff

junke 6y ago

I would say: don't do it. Nowadays I'd rather use Common Lisp as a glue for other scripts rather than invoking it along with other programs with shell scripts. Plus the need for scripts diminues when you can write a function on typed objects (all scripts are parsers). I also try to seek other IPC mechanisms when possible, such as talking to daemons through DBus or JSON-RPC depending on what is available. edit: this story is currently on the front page of HN too, https://news.ycombinator.com/item?id=26491858 https://news.ycombinator.com/item?id=26491858 (Do not use redirection characters in your shell prompt), the neverending examples of how shell scripting makes your life miserable (sure it's helpful, but can't we have nicer things?)

mark_l_watson 6y ago

Nice writeup! I have fairly recently written CLI programs in Common Lisp, GAMBIT Scheme, and Swift. All good experiences. To be transparent, I don't write bash scripts more than a few lines. For something short, I would prefer Ruby. EDIT: in the last month or two, I have experimented with Babashka (mentioned in this thread) also.

vindarel 6y ago

Common Lisp is awesome: image-based development, compile-time type checking, self-contained executables, stability… https://github.com/CodyReichert/awesome-cl/ https://github.com/CodyReichert/awesome-cl/ https://lispcookbook.github.io/cl-cookbook/editor-support.html https://lispcookbook.github.io/cl-cookbook/editor-support.ht... (Atom, Sublime, VSCode, Jupyter…) https://lisp-lang.org/success/ https://lisp-lang.org/success/ (and https://github.com/azzamsa/awesome-lisp-companies https://github.com/azzamsa/awesome-lisp-companies)