5 ms·

Babashka: Fast native Clojure scripting runtime

204 points by zbentley 3y ago

livrem 3y ago
I like Clojure, but I never had any good opportunities to use it other than for a few small hobby projects. It is unfortunate that it is so huge with tons of dependencies and no simpler native implementation. I started looking at various LISPs and Schemes to find something lighter to use instead and ended up settling for Janet that I think is Clojure-like enough to be comfortable to use, but in a small native binary with no dependencies and can be embedded in other native programs. I am sure for big, real, projects that Clojure makes more sense, but for my hobby projects and scripts I do not think I will install it again. I am still happy for the things I learned from learning Clojure. It was a real eye-opener for an old OO-programmer. https://janet-lang.org/ https://janet-lang.org/
- sokoloff 3y ago
  My initial experience as a low-time Clojure programmer* is that the language feels rather small and tidy (in a good way). Not as small as Scheme, but still very easy to learn the entire syntax and the native data types. * - More than half of my time with Clojure has been with Advent of Code challenges the last 3 years, so I'm in the ~250 hour club.
zilti 3y ago
My question with Babashka is always: why? Why force Clojure into something it is not? Why force some interpreter through the heavy GraalVM machinery to have a binary interpreter for a language that wasn't meant for this? Just because "the syntax feels similar"? Might as well just use Scheme for such tasks, and invest the resources into other Clojure projects - Clojure has a need for that. Nonetheless it is an impressive achievement. Just one that I deem, like GraalVM's native compilation itself, an unnecessary one.
- panick21_ 3y ago
  You seem to suggest there is some platonic ideal use of clojure and this isn't it. But this simply isn't the case. Just because the first version of clojure was on the JVM doesn't mean all other uses are somehow 'lesser'. Why is simple, use the language you already know that has good libraries available for scripting.
  - zilti 3y ago
    The thing is that this - unlike ClojureJVM, ClojureScript, ClojureCLR - only acts as if it were Clojure, while not actually being it, by lacking some core features and having a out of technical necessity convoluted "pod" mechanism. We're using it at work, together with Clojure, it works just fine, but it is the weird sibling.
- sokoloff 3y ago
  I see it differently. Consider a dev organization who is considering adopting Clojure as a supported (or even default) language. The case for such adoption is strengthened if you can make the argument that much of the shell scripting tasks can be done in the same* language and in a way that's more ergonomic and safer. That helps drive clojure adoption in a way that suggesting that instead dev teams adopting clojure should also learn scheme and write some of their code in that seems not nearly as compelling. It's the same reason that clojurescript is supportive of the overall clojure community, rather than being a distraction and diversion of resources. * It's not 100% the same, but for what's needed to replace bash scripts, it rounds to 100%.
- akkad33 3y ago
  It allows people who know clojure use it for scripting without the load time of JVM. Most similar implementations of other languages is to overcome some limitation of the original implementation. Like Pypy for python. People do not have to invest time to learn a different language for every use case
andrenth 3y ago
Is anyone using Clojurescript for server-side Clojure with Node/Deno/Bun? I'm not even sure this is possible, but it seems to me like the most obvious candidate for a lighter-weight Clojure environment.
dang 3y ago
Related: Babashka Babooka: Write Command-Line Clojure - https://news.ycombinator.com/item?id=34261010 https://news.ycombinator.com/item?id=34261010 - Jan 2023 (14 comments) Babashka is a fast-starting scripting environment for Clojure - https://news.ycombinator.com/item?id=33909241 https://news.ycombinator.com/item?id=33909241 - Dec 2022 (42 comments) Babashka Clojure nREPL as a system interface - https://news.ycombinator.com/item?id=33761877 https://news.ycombinator.com/item?id=33761877 - Nov 2022 (23 comments) Clojure Scripting on Node.js - https://news.ycombinator.com/item?id=32339979 https://news.ycombinator.com/item?id=32339979 - Aug 2022 (33 comments) Ad-hoc ClojureScript scripting of Mac applications - https://news.ycombinator.com/item?id=29779353 https://news.ycombinator.com/item?id=29779353 - Jan 2022 (8 comments) Replacing my Octopress blog with 200 lines of Babashka - https://news.ycombinator.com/item?id=28982952 https://news.ycombinator.com/item?id=28982952 - Oct 2021 (27 comments) Clojure Babashka scripting notebook - https://news.ycombinator.com/item?id=25227419 https://news.ycombinator.com/item?id=25227419 - Nov 2020 (2 comments) Using Clojure in the command line with Babashka - https://news.ycombinator.com/item?id=24353476 https://news.ycombinator.com/item?id=24353476 - Sept 2020 (36 comments) Babashka: A quick example - https://news.ycombinator.com/item?id=22382381 https://news.ycombinator.com/item?id=22382381 - Feb 2020 (16 comments) Babashka – A Clojure for the grey areas of Bash - https://news.ycombinator.com/item?id=22254793 https://news.ycombinator.com/item?id=22254793 - Feb 2020 (65 comments) Babashka: a tiny Clojure in Clojure for shell scripting - https://news.ycombinator.com/item?id=20667509 https://news.ycombinator.com/item?id=20667509 - Aug 2019 (1 comment)
- akkad33 3y ago
  Sorry if this is a stupid question, but I see comments like this with previous posts on a big number of hackernews posts. Are these auto-generated and what is the point of them?
  - dang 3y ago
    Not automated but machine-augmented: https://news.ycombinator.com/item?id=35668525 https://news.ycombinator.com/item?id=35668525
- minebreaker 3y ago
pixelmonkey 3y ago
This is a particularly joyful piece of open source software. And I love this page describing it. I hadn't seen it before today. After seeing Michiel Brokent -- the project's original creator/author -- present on it at a meetup, I wrote a tidy overview of babashka (bb) which you can find here: https://amontalenti.com/2020/07/11/babashka https://amontalenti.com/2020/07/11/babashka
- sokoloff 3y ago
  Michiel Borkent, aka borkdude (transpose the r and o).
  - pixelmonkey 3y ago
    Ah, indeed. My mistake mis-spelling his last name.
hlship 3y ago
Babashka has unlocked so much power for me. Clojure and the JVM simply aren't set up for rapid start up, Babashka is instant. Bash scripting is alien, ugly, and hazard prone for me - Babashka makes it totally familiar, since it's virtually identical to Clojure. That is the definition of "have my cake and eat it too". The end result is that anything the least bit non-trivial becomes a Babashka script, and I've created tools containing dozens (or potentially, hundreds) of sub-commands. Joyful is the correct adjective.
pron 3y ago
If someone took the time to implement Clojure as a Truffle language [1] it could also be JITted essentially for free (Truffle automatically generates a JIT from an interpreter implementation; further information can make the JITted code more efficient). There seems to have been an academic attempt to do that (for a subset of the language) with promising results: https://epub.jku.at/obvulihs/download/pdf/501665 https://epub.jku.at/obvulihs/download/pdf/501665 [1]: https://www.graalvm.org/latest/graalvm-as-a-platform/language-implementation-framework/ https://www.graalvm.org/latest/graalvm-as-a-platform/languag...
- mike_hearn 3y ago
  I suspect you could also get a long way even without Truffle. Last time I looked at Clojure startup it was doing a lot of really inefficient stuff that would be optimizable with modern JVM features. For example most of Clojure's startup time goes into giant <clinit> class initializers that init endless sets of constants for the entire standard library, regardless of what's actually used. Newer bytecode features like constant dynamic, invokedynamic etc would be ideal for this stuff but the Clojure compiler doesn't use them. You can combine technologies. Compile the Clojure compiler to a native image as a library to make text->bytecode fast, then invoke it from the babashka wrapper if there's no JAR for the script inside a disk cache, otherwise run the JAR if found. If the Clojure bytecode were lazier and you combined it with AppCDS the startup time would be fine. I've got a similar sort of concept but using Kotlin. The scripting syntax is convenient for the use case of replacing bash and the generated bytecode is simple and efficient. Unfortunately compiling Kotlin is really slow, so whilst finished scripts start up nice and fast the edit/compile/run cycle is poor.
  - pron 3y ago
    Clojure's startup time on OpenJDK could certainly be improved considerably with Leyden with or without a Truffle implementation, and it is also true that Truffle is somewhat orthogonal — you could have Truffle Clojure running on OpenJDK or on Native Image. However, if you want to use Native Image, then I think Truffle is the only viable way to improve the runtime performance. Even if you NI the current Clojure compiler, there would still be no way to run it short of running the resulting classes on OpenJDK, and those classes would need to be intialised, so NI would buy you little. I don't think that loading and initialising the compiler is the dominant factor in Clojure's startup; I recall it being the loading of user/library classes.
tombert 3y ago
Interesting; I've been using GraalVM to compile my Clojure JAR files for command line tools. Doing a basic Unix `time` command, the startup time for that is about 20 milliseconds, well within my tolerance threshold of a command line application. That said, the executables for GraalVM are enormous, so I'm absolutely open to something comparable that avoids that. I'll give Babashka an install and play with it.
- funcDropShadow 3y ago
  Babashka is built using GraalVM. It allows you to write script-like programs in Clojure while having instant startup time.

204 points by zbentley 3y ago

I like Clojure, but I never had any good opportunities to use it other than for a few small hobby projects. It is unfortunate that it is so huge with tons of dependencies and no simpler native implementation. I started looking at various LISPs and Schemes to find something lighter to use instead and ended up settling for Janet that I think is Clojure-like enough to be comfortable to use, but in a small native binary with no dependencies and can be embedded in other native programs. I am sure for big, real, projects that Clojure makes more sense, but for my hobby projects and scripts I do not think I will install it again. I am still happy for the things I learned from learning Clojure. It was a real eye-opener for an old OO-programmer. https://janet-lang.org/ https://janet-lang.org/

sokoloff 3y ago
My initial experience as a low-time Clojure programmer* is that the language feels rather small and tidy (in a good way). Not as small as Scheme, but still very easy to learn the entire syntax and the native data types. * - More than half of my time with Clojure has been with Advent of Code challenges the last 3 years, so I'm in the ~250 hour club.

zilti 3y ago

My question with Babashka is always: why? Why force Clojure into something it is not? Why force some interpreter through the heavy GraalVM machinery to have a binary interpreter for a language that wasn't meant for this? Just because "the syntax feels similar"? Might as well just use Scheme for such tasks, and invest the resources into other Clojure projects - Clojure has a need for that. Nonetheless it is an impressive achievement. Just one that I deem, like GraalVM's native compilation itself, an unnecessary one.

panick21_ 3y ago
You seem to suggest there is some platonic ideal use of clojure and this isn't it. But this simply isn't the case. Just because the first version of clojure was on the JVM doesn't mean all other uses are somehow 'lesser'. Why is simple, use the language you already know that has good libraries available for scripting.
- zilti 3y ago
  The thing is that this - unlike ClojureJVM, ClojureScript, ClojureCLR - only acts as if it were Clojure, while not actually being it, by lacking some core features and having a out of technical necessity convoluted "pod" mechanism. We're using it at work, together with Clojure, it works just fine, but it is the weird sibling.
sokoloff 3y ago
I see it differently. Consider a dev organization who is considering adopting Clojure as a supported (or even default) language. The case for such adoption is strengthened if you can make the argument that much of the shell scripting tasks can be done in the same* language and in a way that's more ergonomic and safer. That helps drive clojure adoption in a way that suggesting that instead dev teams adopting clojure should also learn scheme and write some of their code in that seems not nearly as compelling. It's the same reason that clojurescript is supportive of the overall clojure community, rather than being a distraction and diversion of resources. * It's not 100% the same, but for what's needed to replace bash scripts, it rounds to 100%.
akkad33 3y ago
It allows people who know clojure use it for scripting without the load time of JVM. Most similar implementations of other languages is to overcome some limitation of the original implementation. Like Pypy for python. People do not have to invest time to learn a different language for every use case

andrenth 3y ago

Is anyone using Clojurescript for server-side Clojure with Node/Deno/Bun? I'm not even sure this is possible, but it seems to me like the most obvious candidate for a lighter-weight Clojure environment.

dang 3y ago

Related: Babashka Babooka: Write Command-Line Clojure - https://news.ycombinator.com/item?id=34261010 https://news.ycombinator.com/item?id=34261010 - Jan 2023 (14 comments) Babashka is a fast-starting scripting environment for Clojure - https://news.ycombinator.com/item?id=33909241 https://news.ycombinator.com/item?id=33909241 - Dec 2022 (42 comments) Babashka Clojure nREPL as a system interface - https://news.ycombinator.com/item?id=33761877 https://news.ycombinator.com/item?id=33761877 - Nov 2022 (23 comments) Clojure Scripting on Node.js - https://news.ycombinator.com/item?id=32339979 https://news.ycombinator.com/item?id=32339979 - Aug 2022 (33 comments) Ad-hoc ClojureScript scripting of Mac applications - https://news.ycombinator.com/item?id=29779353 https://news.ycombinator.com/item?id=29779353 - Jan 2022 (8 comments) Replacing my Octopress blog with 200 lines of Babashka - https://news.ycombinator.com/item?id=28982952 https://news.ycombinator.com/item?id=28982952 - Oct 2021 (27 comments) Clojure Babashka scripting notebook - https://news.ycombinator.com/item?id=25227419 https://news.ycombinator.com/item?id=25227419 - Nov 2020 (2 comments) Using Clojure in the command line with Babashka - https://news.ycombinator.com/item?id=24353476 https://news.ycombinator.com/item?id=24353476 - Sept 2020 (36 comments) Babashka: A quick example - https://news.ycombinator.com/item?id=22382381 https://news.ycombinator.com/item?id=22382381 - Feb 2020 (16 comments) Babashka – A Clojure for the grey areas of Bash - https://news.ycombinator.com/item?id=22254793 https://news.ycombinator.com/item?id=22254793 - Feb 2020 (65 comments) Babashka: a tiny Clojure in Clojure for shell scripting - https://news.ycombinator.com/item?id=20667509 https://news.ycombinator.com/item?id=20667509 - Aug 2019 (1 comment)

akkad33 3y ago
Sorry if this is a stupid question, but I see comments like this with previous posts on a big number of hackernews posts. Are these auto-generated and what is the point of them?
- dang 3y ago
  Not automated but machine-augmented: https://news.ycombinator.com/item?id=35668525 https://news.ycombinator.com/item?id=35668525
minebreaker 3y ago

pixelmonkey 3y ago

This is a particularly joyful piece of open source software. And I love this page describing it. I hadn't seen it before today. After seeing Michiel Brokent -- the project's original creator/author -- present on it at a meetup, I wrote a tidy overview of babashka (bb) which you can find here: https://amontalenti.com/2020/07/11/babashka https://amontalenti.com/2020/07/11/babashka

sokoloff 3y ago
Michiel Borkent, aka borkdude (transpose the r and o).
- pixelmonkey 3y ago
  Ah, indeed. My mistake mis-spelling his last name.

hlship 3y ago

Babashka has unlocked so much power for me. Clojure and the JVM simply aren't set up for rapid start up, Babashka is instant. Bash scripting is alien, ugly, and hazard prone for me - Babashka makes it totally familiar, since it's virtually identical to Clojure. That is the definition of "have my cake and eat it too". The end result is that anything the least bit non-trivial becomes a Babashka script, and I've created tools containing dozens (or potentially, hundreds) of sub-commands. Joyful is the correct adjective.

pron 3y ago

If someone took the time to implement Clojure as a Truffle language [1] it could also be JITted essentially for free (Truffle automatically generates a JIT from an interpreter implementation; further information can make the JITted code more efficient). There seems to have been an academic attempt to do that (for a subset of the language) with promising results: https://epub.jku.at/obvulihs/download/pdf/501665 https://epub.jku.at/obvulihs/download/pdf/501665 [1]: https://www.graalvm.org/latest/graalvm-as-a-platform/language-implementation-framework/ https://www.graalvm.org/latest/graalvm-as-a-platform/languag...

mike_hearn 3y ago
I suspect you could also get a long way even without Truffle. Last time I looked at Clojure startup it was doing a lot of really inefficient stuff that would be optimizable with modern JVM features. For example most of Clojure's startup time goes into giant <clinit> class initializers that init endless sets of constants for the entire standard library, regardless of what's actually used. Newer bytecode features like constant dynamic, invokedynamic etc would be ideal for this stuff but the Clojure compiler doesn't use them. You can combine technologies. Compile the Clojure compiler to a native image as a library to make text->bytecode fast, then invoke it from the babashka wrapper if there's no JAR for the script inside a disk cache, otherwise run the JAR if found. If the Clojure bytecode were lazier and you combined it with AppCDS the startup time would be fine. I've got a similar sort of concept but using Kotlin. The scripting syntax is convenient for the use case of replacing bash and the generated bytecode is simple and efficient. Unfortunately compiling Kotlin is really slow, so whilst finished scripts start up nice and fast the edit/compile/run cycle is poor.
- pron 3y ago
  Clojure's startup time on OpenJDK could certainly be improved considerably with Leyden with or without a Truffle implementation, and it is also true that Truffle is somewhat orthogonal — you could have Truffle Clojure running on OpenJDK or on Native Image. However, if you want to use Native Image, then I think Truffle is the only viable way to improve the runtime performance. Even if you NI the current Clojure compiler, there would still be no way to run it short of running the resulting classes on OpenJDK, and those classes would need to be intialised, so NI would buy you little. I don't think that loading and initialising the compiler is the dominant factor in Clojure's startup; I recall it being the loading of user/library classes.

tombert 3y ago

Interesting; I've been using GraalVM to compile my Clojure JAR files for command line tools. Doing a basic Unix `time` command, the startup time for that is about 20 milliseconds, well within my tolerance threshold of a command line application. That said, the executables for GraalVM are enormous, so I'm absolutely open to something comparable that avoids that. I'll give Babashka an install and play with it.

funcDropShadow 3y ago
Babashka is built using GraalVM. It allows you to write script-like programs in Clojure while having instant startup time.