5 ms·

Building an Emacs Lisp VM in Rust

133 points by lukastyrychtr 5y ago

Y_Y 5y ago
I was expecting to read language evangelism from someone who hadn't heard of Remacs. This turned out to be a really interesting read and I'm looking forward to seeing what comes of it. I'd love to see a future where there is a choice between different cores for Emacs like Guile or a jitter or one of these rustbuckets.
- disgruntledphd2 5y ago
  Yeah, a multi threaded implementation of the Emacs VM would be amazing, I definitely wish the author well.
rightbyte 5y ago
"I ended up needing to make some tweaks to the ordering and structure of the lisp files to support a bytecode-only bootstrap." Why not make it two pass? We are not reading source from cards anymore.
- pdpi 5y ago
  > About a year ago I was bitten by the PL bug. It started with reading Crafting Interpreters and discovering the wonders hidden under the hood of a compiler. Doesn't sound like the author is super-experienced with this whole interpreter thing, thought might not have crossed their mind.
- celeritascelery 5y ago
  I honestly had not thought of that before. But that would make a lot of sense!
sizediterable 5y ago
I would love to see some benchmarks that demonstrate the impact of the more optimized representation. I found that despite some possibly sub-optimal performance trade-offs in favor of productivity in my own hobby interpreter, it can still do much better than Python and Wren in some micro-benchmarks that I performed (naive sum in a loop and naive fibonacci implementation). Python was no surprise since PyObjects are relatively "fat" [0], but Wren was created by the author of Crafting Interpreters (mentioned in the article) so I expected a bit better. My representation ended up looking something like: enum Value { Unit, False, True, Integer(isize), Float(FloatSize), String(Rc<str>), Closure(Rc<Closure>), Tuple(Rc<[Value]>), } Unfortunately since I'm depending on reference-counting, it can't implement `Copy` which I suspect is where I'm currently losing significant performance. For values embedded in bytecode instructions I can cheat and copy anyway since only the non-refcounted ones are allowed in them. [0] https://github.com/python/cpython/blob/main/Include/object.h#L105-L109 https://github.com/python/cpython/blob/main/Include/object.h...
- mamcx 5y ago
  Yeah, I wonder the same. I struggle a lot early on my lang with this (also: Wanna have 2d vectors) and eventually get a enum alike yours (but the Rc<str>, Rc<Closure>, Rc<[Value]> intrigue me!).
- emteycz 5y ago
  How about turning Value into MemoryCell and handling references outside of it?
  - sizediterable 5y ago
    It's something I'm willing to explore once I have more comprehensive benchmarks. Right now the ones I have are significantly weighted on integer operations that I think will be hurt by the additional pointer indirection. Once my benchmark suite is less biased I can make a more holistic decision about the trade-offs.
- celeritascelery 5y ago
  I plan at some point to implement some benchmarks to test exactly that. I have the feeling that the gain is not going to be as significant as I initially expected. Rust is pretty good at optimizing fat pointers. Though another advantages to word-width objects is that they can generally be used in atomics, whereas sizes greater then a word usually can't. Do you have a link to your hobby interpreter? I would love to take a look at it.

133 points by lukastyrychtr 5y ago

Y_Y 5y ago

I was expecting to read language evangelism from someone who hadn't heard of Remacs. This turned out to be a really interesting read and I'm looking forward to seeing what comes of it. I'd love to see a future where there is a choice between different cores for Emacs like Guile or a jitter or one of these rustbuckets.

disgruntledphd2 5y ago
Yeah, a multi threaded implementation of the Emacs VM would be amazing, I definitely wish the author well.

rightbyte 5y ago

"I ended up needing to make some tweaks to the ordering and structure of the lisp files to support a bytecode-only bootstrap." Why not make it two pass? We are not reading source from cards anymore.

pdpi 5y ago
> About a year ago I was bitten by the PL bug. It started with reading Crafting Interpreters and discovering the wonders hidden under the hood of a compiler. Doesn't sound like the author is super-experienced with this whole interpreter thing, thought might not have crossed their mind.
celeritascelery 5y ago
I honestly had not thought of that before. But that would make a lot of sense!

sizediterable 5y ago

I would love to see some benchmarks that demonstrate the impact of the more optimized representation. I found that despite some possibly sub-optimal performance trade-offs in favor of productivity in my own hobby interpreter, it can still do much better than Python and Wren in some micro-benchmarks that I performed (naive sum in a loop and naive fibonacci implementation). Python was no surprise since PyObjects are relatively "fat" [0], but Wren was created by the author of Crafting Interpreters (mentioned in the article) so I expected a bit better. My representation ended up looking something like: enum Value { Unit, False, True, Integer(isize), Float(FloatSize), String(Rc<str>), Closure(Rc<Closure>), Tuple(Rc<[Value]>), } Unfortunately since I'm depending on reference-counting, it can't implement `Copy` which I suspect is where I'm currently losing significant performance. For values embedded in bytecode instructions I can cheat and copy anyway since only the non-refcounted ones are allowed in them. [0] https://github.com/python/cpython/blob/main/Include/object.h#L105-L109 https://github.com/python/cpython/blob/main/Include/object.h...

mamcx 5y ago
Yeah, I wonder the same. I struggle a lot early on my lang with this (also: Wanna have 2d vectors) and eventually get a enum alike yours (but the Rc<str>, Rc<Closure>, Rc<[Value]> intrigue me!).
emteycz 5y ago
How about turning Value into MemoryCell and handling references outside of it?
- sizediterable 5y ago
  It's something I'm willing to explore once I have more comprehensive benchmarks. Right now the ones I have are significantly weighted on integer operations that I think will be hurt by the additional pointer indirection. Once my benchmark suite is less biased I can make a more holistic decision about the trade-offs.
celeritascelery 5y ago
I plan at some point to implement some benchmarks to test exactly that. I have the feeling that the gain is not going to be as significant as I initially expected. Rust is pretty good at optimizing fat pointers. Though another advantages to word-width objects is that they can generally be used in atomics, whereas sizes greater then a word usually can't. Do you have a link to your hobby interpreter? I would love to take a look at it.