3 ms·

I've been designing my own small language runtime in Rust (VM + JIT + AOT backends) mostly as a way to actually understand tradeoffs compiler authors make inste

I've been designing my own small language runtime in Rust (VM + JIT + AOT backends) mostly as a way to actually understand tradeoffs compiler authors make instead of just reading about them. Racket's approach to macros and language-oriented programming is one of the things I keep coming back to as a reference curious how much of that flexibility comes at a real runtime cost vs. being mostly a compile-time abstraction.

brabel 1mo ago
I can answer that for Common Lisp, specifically the SBCL implementation (there's several others). It compiles every function to native code (you can even inspect the assembly with `disassemble`). Macros are executed before code is compiled. Hence, once you've compiled a function, the macro disappears since its only role is to generate the expressions that are going to actually be compiled and then executed. For example, here's a simple CL macro: (defmacro defer (cleanup &body action) `(unwind-protect (progn ,@action) ,cleanup)) `unwind-protect` is like a `try/finally` in other languages, and I used that above to create something similar to `defer` in Go/Zig which is related to it, but with the operands inverted. The ` symbol is a quasiquote. Unlike quote `'` it lets you unquote symbols inside with the , operator. That's why you'll always see a bunch of '`' and ',' in macros. The @, thing is a "spread" (looks different in Racket from what I saw in the post, which used `...`). It just spreads whatever was on the list in the place you put that on, so if `action` is `(p 1) (p 2)`, then `(progn ,@action)` becomes `(progn (p 1) (p 2))`. You can inspect what the actual code that will be compiled looks like with `macroexpand`: CL-USER> (macroexpand '(defer (cleanup) (do-something))) (UNWIND-PROTECT (PROGN (DO-SOMETHING)) (CLEANUP)) `progn` is a "special operator" that's needed when you want more than one expression to be evaluated in order. Example calling the macro: CL-USER> (defer (print "done") (print "hello") (print "again")) "hello" "again" "done" As you can see, it executed the deferred expression last. We can prove that macros disappear after compile-time with an example: CL-USER> (disassemble (lambda (x) (+ x x))) ; disassembly for (LAMBDA (X)) ; Size: 36 bytes. Origin: #x8005F70664 ; (LAMBDA (X)) ; 64: AA0A40F9 LDR R0, [THREAD, #16] ; binding-stack-pointer ; 68: AA0B00F9 STR R0, [CFP, #16] ; 6C: EA030CAA MOV R0, R2 ; 70: EB030CAA MOV R1, R2 ; 74: 297E80D2 MOVZ TMP, #1009 ; 78: 5E6B69F8 LDR LR, [NULL, TMP] ; SB-KERNEL:TWO-ARG-+ ; 7C: DE130091 ADD LR, LR, #4 ; 80: C0031FD6 BR LR ; 84: E00120D4 BRK #15 ; Invalid argument count trap You can see the Assembly is very simple for `(+ x x)` (the ADD instruction plus a bunch of stack/error maintenance). If we instead had a macro that did this: (defmacro my-macro (x) `(+ ,x ,x)) And a function used that: (defun f (x) (my-macro x)) Now, disassembling the function: CL-USER> (disassemble #'f) ; disassembly for F ; Size: 36 bytes. Origin: #x8005C00374 ; F ; 74: AA0A40F9 LDR R0, [THREAD, #16] ; binding-stack-pointer ; 78: AA0B00F9 STR R0, [CFP, #16] ; 7C: EA030CAA MOV R0, R2 ; 80: EB030CAA MOV R1, R2 ; 84: 297E80D2 MOVZ TMP, #1009 ; 88: 5E6B69F8 LDR LR, [NULL, TMP] ; SB-KERNEL:TWO-ARG-+ ; 8C: DE130091 ADD LR, LR, #4 ; 90: C0031FD6 BR LR ; 94: E00120D4 BRK #15 ; Invalid argument count trap Same thing exactly. I don't know Racket, but knowing it can compile to binary, I expect macros in Racket would work exactly the same.
- gus_massa 1mo ago
  Yes, Racket is quite similar, perhaps with more steps (but I guess you also simplified the steps in SBCL). It's something like 20 steps, but let's keep it short: Racket ---expand the macros---> Simplified Racket "Kernel" Simplified Racket "Kernel"---schemify---> Rumble (that is Chez Scheme + a few libraries and macros) Rumble ---expand the macros---> Simplified Chez Scheme Simplified Chez Scheme ---constants propagation/folding/other---> Simplified Chez Scheme Simplified Chez Scheme ---compiler---> native code You can see the result with https://pkgs.racket-lang.org/package/disassemble https://pkgs.racket-lang.org/package/disassemble
deleted 1mo ago
[deleted]
so-cal-schemer 1mo ago
Some resources on implementing Scheme and variations: Essentials of Programming Languages 3e https://news.ycombinator.com/item?id=24324653 https://news.ycombinator.com/item?id=24324653 Essentials of Compilation: An Incremental Approach in Racket https://github.com/IUCompilerCourse/Essentials-of-Compilation/ https://github.com/IUCompilerCourse/Essentials-of-Compilatio... Lisp in Small Pieces https://news.ycombinator.com/item?id=8600048 https://news.ycombinator.com/item?id=8600048

brabel 1mo ago

I can answer that for Common Lisp, specifically the SBCL implementation (there's several others). It compiles every function to native code (you can even inspect the assembly with `disassemble`). Macros are executed before code is compiled. Hence, once you've compiled a function, the macro disappears since its only role is to generate the expressions that are going to actually be compiled and then executed. For example, here's a simple CL macro: (defmacro defer (cleanup &body action) `(unwind-protect (progn ,@action) ,cleanup)) `unwind-protect` is like a `try/finally` in other languages, and I used that above to create something similar to `defer` in Go/Zig which is related to it, but with the operands inverted. The ` symbol is a quasiquote. Unlike quote `'` it lets you unquote symbols inside with the , operator. That's why you'll always see a bunch of '`' and ',' in macros. The @, thing is a "spread" (looks different in Racket from what I saw in the post, which used `...`). It just spreads whatever was on the list in the place you put that on, so if `action` is `(p 1) (p 2)`, then `(progn ,@action)` becomes `(progn (p 1) (p 2))`. You can inspect what the actual code that will be compiled looks like with `macroexpand`: CL-USER> (macroexpand '(defer (cleanup) (do-something))) (UNWIND-PROTECT (PROGN (DO-SOMETHING)) (CLEANUP)) `progn` is a "special operator" that's needed when you want more than one expression to be evaluated in order. Example calling the macro: CL-USER> (defer (print "done") (print "hello") (print "again")) "hello" "again" "done" As you can see, it executed the deferred expression last. We can prove that macros disappear after compile-time with an example: CL-USER> (disassemble (lambda (x) (+ x x))) ; disassembly for (LAMBDA (X)) ; Size: 36 bytes. Origin: #x8005F70664 ; (LAMBDA (X)) ; 64: AA0A40F9 LDR R0, [THREAD, #16] ; binding-stack-pointer ; 68: AA0B00F9 STR R0, [CFP, #16] ; 6C: EA030CAA MOV R0, R2 ; 70: EB030CAA MOV R1, R2 ; 74: 297E80D2 MOVZ TMP, #1009 ; 78: 5E6B69F8 LDR LR, [NULL, TMP] ; SB-KERNEL:TWO-ARG-+ ; 7C: DE130091 ADD LR, LR, #4 ; 80: C0031FD6 BR LR ; 84: E00120D4 BRK #15 ; Invalid argument count trap You can see the Assembly is very simple for `(+ x x)` (the ADD instruction plus a bunch of stack/error maintenance). If we instead had a macro that did this: (defmacro my-macro (x) `(+ ,x ,x)) And a function used that: (defun f (x) (my-macro x)) Now, disassembling the function: CL-USER> (disassemble #'f) ; disassembly for F ; Size: 36 bytes. Origin: #x8005C00374 ; F ; 74: AA0A40F9 LDR R0, [THREAD, #16] ; binding-stack-pointer ; 78: AA0B00F9 STR R0, [CFP, #16] ; 7C: EA030CAA MOV R0, R2 ; 80: EB030CAA MOV R1, R2 ; 84: 297E80D2 MOVZ TMP, #1009 ; 88: 5E6B69F8 LDR LR, [NULL, TMP] ; SB-KERNEL:TWO-ARG-+ ; 8C: DE130091 ADD LR, LR, #4 ; 90: C0031FD6 BR LR ; 94: E00120D4 BRK #15 ; Invalid argument count trap Same thing exactly. I don't know Racket, but knowing it can compile to binary, I expect macros in Racket would work exactly the same.

gus_massa 1mo ago
Yes, Racket is quite similar, perhaps with more steps (but I guess you also simplified the steps in SBCL). It's something like 20 steps, but let's keep it short: Racket ---expand the macros---> Simplified Racket "Kernel" Simplified Racket "Kernel"---schemify---> Rumble (that is Chez Scheme + a few libraries and macros) Rumble ---expand the macros---> Simplified Chez Scheme Simplified Chez Scheme ---constants propagation/folding/other---> Simplified Chez Scheme Simplified Chez Scheme ---compiler---> native code You can see the result with https://pkgs.racket-lang.org/package/disassemble https://pkgs.racket-lang.org/package/disassemble

deleted 1mo ago

[deleted]

so-cal-schemer 1mo ago

Some resources on implementing Scheme and variations: Essentials of Programming Languages 3e https://news.ycombinator.com/item?id=24324653 https://news.ycombinator.com/item?id=24324653 Essentials of Compilation: An Incremental Approach in Racket https://github.com/IUCompilerCourse/Essentials-of-Compilation/ https://github.com/IUCompilerCourse/Essentials-of-Compilatio... Lisp in Small Pieces https://news.ycombinator.com/item?id=8600048 https://news.ycombinator.com/item?id=8600048