9 ms·

Emacs Lisp JIT Compiler

172 points by unhammer 8y ago

gopalv 8y ago
That's quite a blast from the past. Good on Aleksey for picking it up and modernizing libjit to this point. And for everyone who thinks of LLVM in this context, here's a thread with some "primary sources" [1]. [1] - http://lists.gnu.org/archive/html/dotgnu-libjit/2004-05/msg00012.html http://lists.gnu.org/archive/html/dotgnu-libjit/2004-05/msg0...
- zellyn 8y ago
  Oh wow. I haven't seen those names in a loooong time. I used to lurk and follow the DotGNU mailing lists avidly :-)
- earenndil 8y ago
  AFAIK, llvm jit is incredibly slow, to the point that certain emulators that have used it have had to drop it for performance's sake.
  - fasquoika 8y ago
    IIRC LLVM relatively recently did a major overhaul of the JIT API, including better support for lazy compilation. That might improve compilation speed drastically
- avdicius 8y ago
  Thanks, Gopal. I didn't do much, just maintained it on life support. Recently some good contributors arrived: Jakob and Tom. Let's hope this really revives the project.
dangirsh 8y ago
Don't miss the later comments by Richard Stallman (creator of Emacs), and Tom's reply: Richard> I don't think a 3% speedup is worth those drawbacks. Richard> Or even a 10% speedup. Richard> A really big speedup would justify the costs. Tom> It is 3x, not 3%. Mic drop.
- chubot 8y ago
  More context: In some simple benchmarks, it is about 3x faster than the bytecode interpreter. I'm always skeptical of statements of these, because workloads vary so much. JITs seem to do well for numerical benchmarks, e.g. summing a list of numbers or the mandelbrot fractal. They seem to do worse with string-based workloads, because the bottleneck is in memory allocations, and I have yet to see a JIT that does anything about that (i.e. analyzing code to reduce allocations). I imagine that ELisp is used mostly for string workloads and not numeric workloads. So I won't be surprised if the 3x number doesn't hold up. I'm interested in hearing more details and happy be to be corrected.
  - chrisseaton 8y ago
    > I have yet to see a JIT that does anything about that (i.e. analyzing code to reduce allocations) You mean you've never seen a JIT that does anything about memory allocations for ELISP? Or do you mean you've never seen a JIT do anything at all about memory allocations? Because removing memory allocations through escape analysis and scalar replacement is a key feature of any sophisticated JIT, and there are definitely many JITs which do this. The JIT for Ruby I work on will effectively remove the allocation of string objects.
    - chubot 8y ago
      Hm yes I was wrong about that. It looks like v8 and PyPy do it (which is apparent from some Googling and grepping of the code). I have been reading some papers on JITs and I don't see escape analysis mentioned that often. In the PyPy paper (which is over a decade old) they mention it as future work. Still, I actually tried PyPy on a string-based workload and it was slower than CPython and used more memory. I don't know why but that contributes to my feeling that JITs are bad for string-based workloads. I'm interested in seeing any pointers to benchmarks that show the improvements resulting from escape analysis in JITs. I haven't seen anything like that and I've done a decent amount of research. A cursory look at this blog post makes me think it's not super straightforward: https://v8project.blogspot.com/2017/09/disabling-escape-analysis.html https://v8project.blogspot.com/2017/09/disabling-escape-anal... That post is less than a year old! i.e. the fact that v8 has been around for 10+ years and they're still updating escape analysis makes me wonder what the issue with it is. Is it hard to implement or does it not produce that much speedup? I appreciate any pointers.
sillysaurus3 8y ago
The reason it's difficult to make elisp fast is because the language defaults to dynamic scoping rather than lexical scoping. That means any function that uses `let` is no longer making a tailcall. Meaning you have to restore the value of each variable after you call whatever function would have been the tailcall. Stuff like that adds up. It's why Lua is so fast compared to JS, too.
- rurban 8y ago
  But this could be solved with escape analysis. Rarely do such let locals shadow outer locals, and as such don't need to be restored.
  - dan-robertson 8y ago
    You can’t know whether you are shadowing an outer variable. With dynamic scope: (defun get-x () x) (defun foo (let ((x 7)) (get-x))) (foo) ;; => 7 (not shadowing x) (let ((x 4)) (foo)) ;; => 7 (is shadowing x) (setq old-get-x #'get-x) (defun get-x () (let ((x 10)) (funcall old-get-x))) (foo) ;; => 10
    - rurban 8y ago
      Ouch, you are right. So the only strategy would be to switch to lexical compilation blockwise, while fixing the remaining dynamic extent vars.
      - pzone 8y ago
        Which is more or less the strategy in the Emacs codebase - the intention is to incrementally port to lexical binding on a per-file basis. (This is seen through the annotation -- lexical-binding:t -- at the top of many .el files.)
    - kazinator 8y ago
      Emacs Lisp has supported lexical scope in its byte code compiler as an option for eons, and is starting to make more use of that. No reason why a brand new JIT couldn't follow suit.
fusiongyro 8y ago
I hate to be "that guy," but what happened to the Guile-Emacs project? Guile performs way better than Elisp too, but (perhaps more importantly) raises the possibility of writing configuration and extensions in Lua and Scheme.
- rurban 8y ago
  Too slow strings still
- avar 8y ago
  I wrote a comment here over a year ago summarizing some of the state of it, it seems like there's been no changes since then: https://news.ycombinator.com/item?id=13885682 https://news.ycombinator.com/item?id=13885682
- rauhl 8y ago
  IMHO switching to Guile from C is moving in the wrong direction: rather than going from a poor-but-performant systems programming language like C to a pedagogic language like Scheme, it’d be a better idea to move to an rich-and-performant language meant for industrial systems programming like Common Lisp. Scheme’s a pretty little language, but it’s not meant for building large applications. Indeed, elisp’s weaknesses are IMHO mainly where it’s most akin to Scheme (e.g. its flat namespace). I also think it’d be a bad idea to make it too easy to write emacs extensions in anything but a Lisp, because that would fragment the ecosystem. If we’re going to replace elisp, it should be with a better Lisp, not with a lesser language (e.g. Lua, Python, Ruby … whatever). The great thing about a JIT is that we get a speedup without changing the actual implementation language or fragmenting the ecosystem.
  - fusiongyro 8y ago
    The idea was to replace Emacs's embedded Elisp interpreter with Guile's, not to rewrite Emacs in Guile Scheme.
    - eadmund 8y ago
vemv 8y ago
Tangentially related: applying some commonly-recommended optimizations, you can shave Emacs init time to about 2 seconds, from whatever your previous init time was (7 in my case, but I also read about 60s -> 2s improvements). Init time alone greatly affects how a given system is perceived.
- fusiongyro 8y ago
  What are these recommended optimizations?
  - kovrik 8y ago
    As far as I know, things like: changing gc-cons-threshold, setting file-name-handler-alist to nil, adjusting some jit-lock-* values. And using deferred package loading.
  - yankcrime 8y ago
    This section from the Doom Emacs distribution's FAQ covers some of the recommendations: https://github.com/hlissner/doom-emacs/wiki/FAQ#how-is-dooms-startup-so-fast https://github.com/hlissner/doom-emacs/wiki/FAQ#how-is-dooms...
  - dan-robertson 8y ago
    The extreme version is to reproduce a step from building Emacs where you produce a binary with the libraries you want already loaded into memory. This is like making a lisp-image except the image also includes a big C text editor
    - vemv 8y ago
      This sounds pretty neat (and something a CI server could routinely do) - has it been done in practice?
      - dan-robertson 8y ago
        Doing this requires various security features that operating systems have gained since the 1980s to be turned off so it’s not super advisable if you can avoid it
davidw 8y ago
> From: Tom Tromey Of course it is. That guy's an amazing wizard.
- deleted 8y ago
  [deleted]
microcolonel 8y ago
Cool to see that libjit can actually help with elisp, I've not really seen many other uses of libjit.
zokier 8y ago
The authors blog has some extra details (see also the comments) https://tromey.com/blog/?p=982 https://tromey.com/blog/?p=982
didibus 8y ago
Make my emacs faster, yes please!
mijoharas 8y ago
Absolutely love the message from stallman and the response (paraphrased below, full quote [0]): Richard> I don't think a 3% speedup is worth those drawbacks [ed: added complexity]. Or even a 10% speedup. A really big speedup would justify the costs. Tom> It is 3x, not 3%. [0] https://lists.gnu.org/archive/html/emacs-devel/2018-08/msg00439.html https://lists.gnu.org/archive/html/emacs-devel/2018-08/msg00...
vmsp 8y ago
The code is in branch feature/libjit of emacs' repo, if anyone wants to take a look. http://git.savannah.gnu.org/cgit/emacs.git/tree/?h=feature/libjit http://git.savannah.gnu.org/cgit/emacs.git/tree/?h=feature/l...
avdicius 8y ago
The development of libjit almost halted a while back. But thanks to the last GSoC it is set to receive a major update soon: https://github.com/ademakov/libjit/pull/14 https://github.com/ademakov/libjit/pull/14
- vkazanov 8y ago
  Is it about the new register allocator? How good is it?
  - avdicius 8y ago
    Yes, initial benchmark results: https://github.com/M4GNV5/GSoC2018 https://github.com/M4GNV5/GSoC2018
ezoe 8y ago
https://lists.gnu.org/archive/html/emacs-devel/2018-08/msg00437.html https://lists.gnu.org/archive/html/emacs-devel/2018-08/msg00... >To replace an interpreter by a JIT compiler means more oomplexity and > also more possible problems. (For example, if there are platforms > someday that libjit does not support.) Reading a Lisp interpreter is > very useful for learning. > If the plan is to add a jit and keep the Lisp interpreter as well, > we don't lose its advantages for study, and we can still support all > the platforms -- but we add complexity even more. > I don't think a 3% speedup is worth those drawbacks. Or even a 10% > speedup. A really big speedup would justify the costs. Why do they still let RMS allows to say something about technical today? No wonder Emacs is losing the users.
- vkazanov 8y ago
  While sometimes overruling he's completely reasonable here. There had been multiple approaches to jit-ing emacs bytecode, and all of them offerred a very limited speed-up. Jitting engines bring extra dependencies and one more layer of complexity to the C core of the editor. And not too many editors support a full language of their own. Notice that Emacs is mostly a volunteer effort and the resources are very limited. For example this particular engine seems to offer a good speed boost on numeric operations but one of the replies mentions that realistic complex code is sometimes slower than a pure byte-code VM. Probably a bug but still... After all those failed attempts certain level of scepticism is understandable.
- fantispug 8y ago
  A 3% increase in speed is a miniscule improvement. Emacs has a small dedicated group of maintainers, increasing the maintenance burden will stop other developements (such as the recent addition of async which is a much bigger performance issue). If emacs is losing users I would guess it is because to make it an IDE takes more configuration, and has more issues than competing products.
  - Latteland 8y ago
    it's 3x, 3 times, not 3%
tmalsburg2 8y ago
With JIT compilation, wouldn't byte compilation become redundant? If yes, a JIT could significantly reduce the complexity of Emacs (contrary to RMS's comment).
- gopalv 8y ago
  > wouldn't byte compilation become redundant? Bitcode is still useful, even with a JIT (just like in Java) - the JIT needs to do the heavy parsing, semantic analysis etc everytime it starts. However, the JIT means it can execute the bitcode faster than an interpreter can, because it can keep more intermediate steps in registers directly (like spill to an xmm register, instead of heap) & can do a tiny bit of cpu guided optimization (like is AES-NI available etc).
jkabrg 8y ago
Wouldn't it be nice if ELisp were implemented in RPython? What sort of difficulties would this face over the JIT compiler in the link? It would have the advantage that such an ELisp implementation would still be an interpreter (annotated and auto-JITted). Presumably, that would keep it "simple".
Myrmornis 8y ago
Cool! I just compiled it. How do I know whether the jit is enabled / I wonder how to convince myself what the speedup is for different usages. git clone git@github.com:emacs-mirror/emacs.git cd emacs git checkout feature/libjit # Instructions for macos from # https://stuff-things.net/2018/01/30/building-emacs-25-on-macos-high-sierra/ brew install autoconf automake texinfo export PATH="/usr/local/opt/texinfo/bin:$PATH" make configure ./configure --with-ns make install open nextstep/Emacs.app

172 points by unhammer 8y ago

That's quite a blast from the past. Good on Aleksey for picking it up and modernizing libjit to this point. And for everyone who thinks of LLVM in this context, here's a thread with some "primary sources" [1]. [1] - http://lists.gnu.org/archive/html/dotgnu-libjit/2004-05/msg00012.html http://lists.gnu.org/archive/html/dotgnu-libjit/2004-05/msg0...

zellyn 8y ago
Oh wow. I haven't seen those names in a loooong time. I used to lurk and follow the DotGNU mailing lists avidly :-)
earenndil 8y ago
AFAIK, llvm jit is incredibly slow, to the point that certain emulators that have used it have had to drop it for performance's sake.
- fasquoika 8y ago
  IIRC LLVM relatively recently did a major overhaul of the JIT API, including better support for lazy compilation. That might improve compilation speed drastically
avdicius 8y ago
Thanks, Gopal. I didn't do much, just maintained it on life support. Recently some good contributors arrived: Jakob and Tom. Let's hope this really revives the project.

dangirsh 8y ago

Don't miss the later comments by Richard Stallman (creator of Emacs), and Tom's reply: Richard> I don't think a 3% speedup is worth those drawbacks. Richard> Or even a 10% speedup. Richard> A really big speedup would justify the costs. Tom> It is 3x, not 3%. Mic drop.

chubot 8y ago
More context: In some simple benchmarks, it is about 3x faster than the bytecode interpreter. I'm always skeptical of statements of these, because workloads vary so much. JITs seem to do well for numerical benchmarks, e.g. summing a list of numbers or the mandelbrot fractal. They seem to do worse with string-based workloads, because the bottleneck is in memory allocations, and I have yet to see a JIT that does anything about that (i.e. analyzing code to reduce allocations). I imagine that ELisp is used mostly for string workloads and not numeric workloads. So I won't be surprised if the 3x number doesn't hold up. I'm interested in hearing more details and happy be to be corrected.
- chrisseaton 8y ago
  > I have yet to see a JIT that does anything about that (i.e. analyzing code to reduce allocations) You mean you've never seen a JIT that does anything about memory allocations for ELISP? Or do you mean you've never seen a JIT do anything at all about memory allocations? Because removing memory allocations through escape analysis and scalar replacement is a key feature of any sophisticated JIT, and there are definitely many JITs which do this. The JIT for Ruby I work on will effectively remove the allocation of string objects.
  - chubot 8y ago
    Hm yes I was wrong about that. It looks like v8 and PyPy do it (which is apparent from some Googling and grepping of the code). I have been reading some papers on JITs and I don't see escape analysis mentioned that often. In the PyPy paper (which is over a decade old) they mention it as future work. Still, I actually tried PyPy on a string-based workload and it was slower than CPython and used more memory. I don't know why but that contributes to my feeling that JITs are bad for string-based workloads. I'm interested in seeing any pointers to benchmarks that show the improvements resulting from escape analysis in JITs. I haven't seen anything like that and I've done a decent amount of research. A cursory look at this blog post makes me think it's not super straightforward: https://v8project.blogspot.com/2017/09/disabling-escape-analysis.html https://v8project.blogspot.com/2017/09/disabling-escape-anal... That post is less than a year old! i.e. the fact that v8 has been around for 10+ years and they're still updating escape analysis makes me wonder what the issue with it is. Is it hard to implement or does it not produce that much speedup? I appreciate any pointers.

sillysaurus3 8y ago

The reason it's difficult to make elisp fast is because the language defaults to dynamic scoping rather than lexical scoping. That means any function that uses `let` is no longer making a tailcall. Meaning you have to restore the value of each variable after you call whatever function would have been the tailcall. Stuff like that adds up. It's why Lua is so fast compared to JS, too.

rurban 8y ago
But this could be solved with escape analysis. Rarely do such let locals shadow outer locals, and as such don't need to be restored.
- dan-robertson 8y ago
  You can’t know whether you are shadowing an outer variable. With dynamic scope: (defun get-x () x) (defun foo (let ((x 7)) (get-x))) (foo) ;; => 7 (not shadowing x) (let ((x 4)) (foo)) ;; => 7 (is shadowing x) (setq old-get-x #'get-x) (defun get-x () (let ((x 10)) (funcall old-get-x))) (foo) ;; => 10
  - rurban 8y ago
    Ouch, you are right. So the only strategy would be to switch to lexical compilation blockwise, while fixing the remaining dynamic extent vars.
    - pzone 8y ago
      Which is more or less the strategy in the Emacs codebase - the intention is to incrementally port to lexical binding on a per-file basis. (This is seen through the annotation -- lexical-binding:t -- at the top of many .el files.)
  - kazinator 8y ago
    Emacs Lisp has supported lexical scope in its byte code compiler as an option for eons, and is starting to make more use of that. No reason why a brand new JIT couldn't follow suit.

fusiongyro 8y ago

I hate to be "that guy," but what happened to the Guile-Emacs project? Guile performs way better than Elisp too, but (perhaps more importantly) raises the possibility of writing configuration and extensions in Lua and Scheme.

rurban 8y ago
Too slow strings still
avar 8y ago
I wrote a comment here over a year ago summarizing some of the state of it, it seems like there's been no changes since then: https://news.ycombinator.com/item?id=13885682 https://news.ycombinator.com/item?id=13885682
rauhl 8y ago
IMHO switching to Guile from C is moving in the wrong direction: rather than going from a poor-but-performant systems programming language like C to a pedagogic language like Scheme, it’d be a better idea to move to an rich-and-performant language meant for industrial systems programming like Common Lisp. Scheme’s a pretty little language, but it’s not meant for building large applications. Indeed, elisp’s weaknesses are IMHO mainly where it’s most akin to Scheme (e.g. its flat namespace). I also think it’d be a bad idea to make it too easy to write emacs extensions in anything but a Lisp, because that would fragment the ecosystem. If we’re going to replace elisp, it should be with a better Lisp, not with a lesser language (e.g. Lua, Python, Ruby … whatever). The great thing about a JIT is that we get a speedup without changing the actual implementation language or fragmenting the ecosystem.
- fusiongyro 8y ago
  The idea was to replace Emacs's embedded Elisp interpreter with Guile's, not to rewrite Emacs in Guile Scheme.
  - eadmund 8y ago

vemv 8y ago

Tangentially related: applying some commonly-recommended optimizations, you can shave Emacs init time to about 2 seconds, from whatever your previous init time was (7 in my case, but I also read about 60s -> 2s improvements). Init time alone greatly affects how a given system is perceived.

fusiongyro 8y ago
What are these recommended optimizations?
- kovrik 8y ago
  As far as I know, things like: changing gc-cons-threshold, setting file-name-handler-alist to nil, adjusting some jit-lock-* values. And using deferred package loading.
- yankcrime 8y ago
  This section from the Doom Emacs distribution's FAQ covers some of the recommendations: https://github.com/hlissner/doom-emacs/wiki/FAQ#how-is-dooms-startup-so-fast https://github.com/hlissner/doom-emacs/wiki/FAQ#how-is-dooms...
- dan-robertson 8y ago
  The extreme version is to reproduce a step from building Emacs where you produce a binary with the libraries you want already loaded into memory. This is like making a lisp-image except the image also includes a big C text editor
  - vemv 8y ago
    This sounds pretty neat (and something a CI server could routinely do) - has it been done in practice?
    - dan-robertson 8y ago
      Doing this requires various security features that operating systems have gained since the 1980s to be turned off so it’s not super advisable if you can avoid it

davidw 8y ago

> From: Tom Tromey Of course it is. That guy's an amazing wizard.

deleted 8y ago
[deleted]

microcolonel 8y ago

Cool to see that libjit can actually help with elisp, I've not really seen many other uses of libjit.

zokier 8y ago

The authors blog has some extra details (see also the comments) https://tromey.com/blog/?p=982 https://tromey.com/blog/?p=982

didibus 8y ago

Make my emacs faster, yes please!

mijoharas 8y ago

Absolutely love the message from stallman and the response (paraphrased below, full quote [0]): Richard> I don't think a 3% speedup is worth those drawbacks [ed: added complexity]. Or even a 10% speedup. A really big speedup would justify the costs. Tom> It is 3x, not 3%. [0] https://lists.gnu.org/archive/html/emacs-devel/2018-08/msg00439.html https://lists.gnu.org/archive/html/emacs-devel/2018-08/msg00...

vmsp 8y ago

The code is in branch feature/libjit of emacs' repo, if anyone wants to take a look. http://git.savannah.gnu.org/cgit/emacs.git/tree/?h=feature/libjit http://git.savannah.gnu.org/cgit/emacs.git/tree/?h=feature/l...

avdicius 8y ago

The development of libjit almost halted a while back. But thanks to the last GSoC it is set to receive a major update soon: https://github.com/ademakov/libjit/pull/14 https://github.com/ademakov/libjit/pull/14

vkazanov 8y ago
Is it about the new register allocator? How good is it?
- avdicius 8y ago
  Yes, initial benchmark results: https://github.com/M4GNV5/GSoC2018 https://github.com/M4GNV5/GSoC2018

ezoe 8y ago

https://lists.gnu.org/archive/html/emacs-devel/2018-08/msg00437.html https://lists.gnu.org/archive/html/emacs-devel/2018-08/msg00... >To replace an interpreter by a JIT compiler means more oomplexity and > also more possible problems. (For example, if there are platforms > someday that libjit does not support.) Reading a Lisp interpreter is > very useful for learning. > If the plan is to add a jit and keep the Lisp interpreter as well, > we don't lose its advantages for study, and we can still support all > the platforms -- but we add complexity even more. > I don't think a 3% speedup is worth those drawbacks. Or even a 10% > speedup. A really big speedup would justify the costs. Why do they still let RMS allows to say something about technical today? No wonder Emacs is losing the users.

vkazanov 8y ago
While sometimes overruling he's completely reasonable here. There had been multiple approaches to jit-ing emacs bytecode, and all of them offerred a very limited speed-up. Jitting engines bring extra dependencies and one more layer of complexity to the C core of the editor. And not too many editors support a full language of their own. Notice that Emacs is mostly a volunteer effort and the resources are very limited. For example this particular engine seems to offer a good speed boost on numeric operations but one of the replies mentions that realistic complex code is sometimes slower than a pure byte-code VM. Probably a bug but still... After all those failed attempts certain level of scepticism is understandable.
fantispug 8y ago
A 3% increase in speed is a miniscule improvement. Emacs has a small dedicated group of maintainers, increasing the maintenance burden will stop other developements (such as the recent addition of async which is a much bigger performance issue). If emacs is losing users I would guess it is because to make it an IDE takes more configuration, and has more issues than competing products.
- Latteland 8y ago
  it's 3x, 3 times, not 3%

tmalsburg2 8y ago

With JIT compilation, wouldn't byte compilation become redundant? If yes, a JIT could significantly reduce the complexity of Emacs (contrary to RMS's comment).

gopalv 8y ago
> wouldn't byte compilation become redundant? Bitcode is still useful, even with a JIT (just like in Java) - the JIT needs to do the heavy parsing, semantic analysis etc everytime it starts. However, the JIT means it can execute the bitcode faster than an interpreter can, because it can keep more intermediate steps in registers directly (like spill to an xmm register, instead of heap) & can do a tiny bit of cpu guided optimization (like is AES-NI available etc).

jkabrg 8y ago

Wouldn't it be nice if ELisp were implemented in RPython? What sort of difficulties would this face over the JIT compiler in the link? It would have the advantage that such an ELisp implementation would still be an interpreter (annotated and auto-JITted). Presumably, that would keep it "simple".

Myrmornis 8y ago

Cool! I just compiled it. How do I know whether the jit is enabled / I wonder how to convince myself what the speedup is for different usages. git clone git@github.com:emacs-mirror/emacs.git cd emacs git checkout feature/libjit # Instructions for macos from # https://stuff-things.net/2018/01/30/building-emacs-25-on-macos-high-sierra/ brew install autoconf automake texinfo export PATH="/usr/local/opt/texinfo/bin:$PATH" make configure ./configure --with-ns make install open nextstep/Emacs.app