3 ms·

If someone took the time to implement Clojure as a Truffle language [1] it could also be JITted essentially for free (Truffle automatically generates a JIT from

• mike_hearn 3y ago
  I suspect you could also get a long way even without Truffle. Last time I looked at Clojure startup it was doing a lot of really inefficient stuff that would be optimizable with modern JVM features. For example most of Clojure's startup time goes into giant <clinit> class initializers that init endless sets of constants for the entire standard library, regardless of what's actually used. Newer bytecode features like constant dynamic, invokedynamic etc would be ideal for this stuff but the Clojure compiler doesn't use them. You can combine technologies. Compile the Clojure compiler to a native image as a library to make text->bytecode fast, then invoke it from the babashka wrapper if there's no JAR for the script inside a disk cache, otherwise run the JAR if found. If the Clojure bytecode were lazier and you combined it with AppCDS the startup time would be fine. I've got a similar sort of concept but using Kotlin. The scripting syntax is convenient for the use case of replacing bash and the generated bytecode is simple and efficient. Unfortunately compiling Kotlin is really slow, so whilst finished scripts start up nice and fast the edit/compile/run cycle is poor.
  • pron 3y ago
    Clojure's startup time on OpenJDK could certainly be improved considerably with Leyden with or without a Truffle implementation, and it is also true that Truffle is somewhat orthogonal — you could have Truffle Clojure running on OpenJDK or on Native Image. However, if you want to use Native Image, then I think Truffle is the only viable way to improve the runtime performance. Even if you NI the current Clojure compiler, there would still be no way to run it short of running the resulting classes on OpenJDK, and those classes would need to be intialised, so NI would buy you little. I don't think that loading and initialising the compiler is the dominant factor in Clojure's startup; I recall it being the loading of user/library classes.
    • mike_hearn 3y ago
      Right, so in order of rollout difficulty for the end developer: 1. Make the Clojure compiler emit more efficient and lazier bytecode, so it doesn't init everything all at once at startup (the stdlib gets compiled into what is effectively one mega-class). No change required to anything, it just starts up faster once they upgrade to a new Clojure version. Constant Dynamic was made for this problem. 2. AppCDS, Leyden. No semantics change, only workflow. 3. A Truffle version that operates on source code. Some workflow changes. Have to switch to a new runtime, eliminate build steps and ship sourcecode instead of bytecode (which may not matter). 4. Compile end user programs themselves down to native with native-image. Semantics and workflow both change but all the dead code in the stdlib can be eliminated this way. Lots of options. (1) is the lowest hanging fruit but I suspect the intersection of people who know about the newest bytecode features, people who can modify the Clojure compiler, and people with the time and energy to do so, is currently the empty set.
      • pron 3y ago
        I agree, just note that a Truffle implementation doesn't require a new runtime (if by runtime you mean a Java runtime; Truffle works on OpenJDK) and that the implementation of Babashka already depends on a Clojure interpreter (just not one based on the Truffle framework) that's a different implementation from mainline Clojure (which uses a compiler to bytecode). My point wasn't so much to suggest to Clojure's stewards a course of action but to say that since Babashka already makes use of Native Image to get fast startup and already depends on a separate interpreter implementation that suffers from runtime performance, then Babashka may want to consider basing their interpreter on Truffle to improve runtime performance. If you already picked Native Image for your product and already decided to create a separate implementation of the language based on a source-code interpreter, might as well try basing it on Truffle.
        • mike_hearn 3y ago
          That's true. Adapting or rewriting their interpreter for Truffle would indeed solve these problems too. One of the side projects I keep wanting to try is to see how far I can drive GPT-4 to develop a Lisp interpreter for Truffle. Lisps are simple languages, and it does know at least some of the API.