4 ms·
I always wondered why we rarely see more efforts like an LLVM based Lisp or Scheme language, especially when some projects like Racket are looking to improve pe
by giancarlostoro 6y ago
I always wondered why we rarely see more efforts like an LLVM based Lisp or Scheme language, especially when some projects like Racket are looking to improve performance by switching their runtimes (and I assume other benefits that come from consolidating underlying programming efforts across multiple projects).
- rscho 6y agoThe main driver for Chez-based Racket was to have a Scheme runtime. Performance was an afterthought, but was anticipated due to the excellent performance of Chez and the superior expertise of many Racketeers with Scheme compared to C. The macro expander was a particularly desirable part to have in Scheme, as it since allowed more people to contribute to this complex but crucial component.
- Reelin 6y ago> an LLVM based Lisp or Scheme language Clasp is exactly that! (https://www.cliki.net/Clasp https://www.cliki.net/Clasp) There's also SICL, which includes the Cleavir framework (used by Clasp I think?). (https://cliki.net/SICL https://cliki.net/SICL) And there are also other non-Lisp languages that manage to attain similar levels of "meta" power and expressiveness. Terra is one example; it uses LLVM. (http://terralang.org/ http://terralang.org/)
- moonchild 6y agoMeh, sbcl[1] has an excellent JIT. You don't need a heavyweight AOT, and it kills your compile time. 1. http://sbcl.org/ http://sbcl.org/
- lispm 6y agoSBCL provides AOT compiler. A file to machine code compiler and a Lisp form to in-memory machine code compiler. It's not a JIT compiler.
- pankajdoharey 6y agoSBCL has JIT'ing capabilities sb-fasteval. Essentially as long as you can generate machine code, putting it to memory and making that piece of memory executable is all you need to JIT.
- moonchild 6y agoSBCL compiles lisp code, assembles it in-memory and executes it directly. That's a JIT in my book. It can also dump that code to a file ahead-of-time to be run later—true—even better! More flexibility!
- juki 6y agoWhether code is written to a file or just executed in memory is irrelevant. JIT compilers compile code at run time, and may also recompile the already compiled code again to optimize it based on run-time analysis. SBCL however has distinct compile-time and run-time. For example, you have a function that does a bunch of math without knowing the types of the arguments. A JIT compiler can first compile it with generic math, but later observe that it's almost always called with fixnums, and choose to create a specialized implementation for fixnums. SBCL relies on the programmer to implement and call the optimized version when appropriate.
- lispm 6y agoThat's what in the Lisp context might be called 'incremental in-memory machine-code compilation'. The compiler can take individual forms and create in-memory machine code. The interface to that is the function COMPILE. It's then often used in the READ EVAL PRINT LOOP. The EVAL function is then implemented by first compiling the source code passed (SBCL compiles it to machine code) and then calling the compiled code. This is then a tiny AOT compilation step in each REPL usage. That feature (REPL default incremental AOT machine code compilation) is available in some Lisp implementations at least since the 80s - but I don't know if it was used even earlier. Might be. Often interpreters were used in the REPL -> one of the reasons was reduced latency. Popular always compiling REPLs used a fast (and thus a bit dumb) compiler, then. Why should this not be called JIT in the SBCL case? Because even in this case, where the user enters code to a running Lisp, usually all of the source code (not byte code -> SBCL does not use a byte code engine) is unconditionally compiled to machine code before (!) it gets executed. What people usually (outside of Lisp) experience as a JIT compiler is actually something like a combination of an AOT byte code compilation step and runtime JIT machine code generation - usually under control of some byte code engine, which decides when and what to compile. SBCL does not do the moral equivalent of that. Now SBCL also has an optional source code (-> not byte code) interpreter, which might do fancy stuff when used.
- deleted 6y ago[deleted]
- afranchuk 6y agoI've been working on a lisp that's strongly typed and basically only has functions that are the llvm C API, including JIT. With that you can build macros, type systems, etc. Basically much of the code you write as a user can be what's typically implemented (in predetermined ways) in the compiler in other languages.
- giancarlostoro 6y agoThis sounds interesting is any of this public or publicly blogged about?
- afranchuk 6y agoI'd love to make it public when it is a bit more thorough. Right now it's just local. I really wanted to get to the point that it's self-hosted before going public, but naturally that's a little involved since it does involve building everything up from assembly (llvm IR). But it's fairly straightforward, albeit time consuming, to build simple abstractions on top of each other to implement features typically present in high-level languages.