4 ms·

Reader Macros in Common Lisp (2014)

93 points by susam 4y ago

timonoko 4y ago
Apropos. Is there reader macro definitions for "(" and ")" ? Or is Gödel rolling in his grave for such insolent heresy?
- sph 4y ago
  It would be a noop. A reader macro that takes a s-expr bounded by parentheses, and returns a s-expr, literally is just the identity function^Hmacro.
  - stassats 4y ago
    It's not. Reader macros don't take s-expressions, they take characters.
- stassats 4y ago
  Sure, what's so special about it? (get-macro-character #\() => SB-IMPL::READ-LIST (get-macro-character #\)) => SB-IMPL::READ-RIGHT-PAREN (that one just signals an error, the read-list one picks up the closing #\))
  - timonoko 4y ago
    In other words : You cannot redefine ")" because it cannot be undefined while reading the new definition.
    - aidenn0 4y ago
      No, it can totally be redefined. Reading happens before evaluation
    - creepycrawler 4y ago
      Sure you can. CL-USER> (set-macro-character #\) (get-macro-character #\()) T CL-USER> )format t "Hello") Hello NIL
    - thelopa 4y ago
      That’s completely incorrect. The parent comment has it right. In normal usage, the meaning of #\) is entirely determined by the implementation for the #\( reader macro. This happens because the reader macro for #\( keeps consuming characters until it has located AND consumed the matching #\). The reader would only run the #\) reader macro if it encountered a #\) that didn’t have a corresponding #\(. That’s what they meant when they said “it signals an error”. Specifically, it signals that there are unbalanced parentheses. If you redefine the #\) reader macro, you could use it in a top-level context. That’s probably not a good idea since it’s easy to accidentally have too many #\) when closing a deeply nested expression and thus accidentally invoke the macro. Aside from signaling an error, the #\) reader macro is also useful because it changes the rules when reading symbols. Basically, if you write (+ foo bar), the existence of the #\) macro helps the reader know that you’re referencing the “bar” symbol rather than the “bar)” symbol. Generally, when people define new balanced-pair syntax for Common Lisp, (such as a #\{ macro for hash tables) they will follow the same pattern and define a corresponding reader macro for the closing side that always signals an error for all the same reasons. Edit: also, as others have pointed out, you seem to be mistakenly assuming that the redefinition takes affect mid-way through reading the expression. That’s not how CL works. CL cleanly separates the process of executing code into a few distinct phases. First, the reader reads an entire expression (“form” in lisp terms). Then, that form is macroexpanded (traditional macros, not reader macros!) as needed before (optionally) being compiled and then executed. The change to the read table would happen during the execution phase — well ordered after the original characters for that form are out of the picture. You COULD force a change to the readtable mid-way through reading a form using the #. reader macro, but that definitely gets into chainsaw-juggling territory.
      - timonoko 4y ago
        Ok. I understood my error in about 5.3 second but let it stay, because it was funny, and would trigger somebody to generate a wall-chart of explanations. Anyways already in 1982 I was diabolically opposed to this kind of shit. Read and Print should be as simple as possible and always one-to-one. When you print something to disk, that is what you get when reading, no additional adjustment needed. If you want reading macros, for example, you make your own Read. Better Read could even be in standard package "Additional-macros-for-common-lisp".
sph 4y ago
Great timing. It's my turn to write a Lisp, and I was just thinking about implementing reader macros this morning :-) My goal is to create the barest-metal Lisp OS (basically Lisp REPL with full ring-0 access, the OS) with an asm reader macro to write low level assembly opcodes you can jmp into. (defn add1 (x) (declare x int64) #asm( mov rax, %x add rax, 1 )) (Just thought of this syntax on the spot, I'm a few dozen hours away from that still.)
- timonoko 4y ago
  1970's calling. Been there. Done that: https://news.ycombinator.com/item?id=33517092#33517797 https://news.ycombinator.com/item?id=33517092#33517797
  - sph 4y ago
    Thanks for the inspiration. The fact that you've already done it won't stop me. Got more details/links to study?
    - timonoko 4y ago
      https://github.com/timonoko/nokolisp_addons/blob/master/assembly.lsp https://github.com/timonoko/nokolisp_addons/blob/master/asse... Keyword is the line (dos-eval '(debug < TO.DEB > NUL)) ("mapc" and "map" had their parameters in wrong order, which annoys me even today)
- macoovacany 4y ago
  https://movitz.common-lisp.dev/ https://movitz.common-lisp.dev/
- moonchild 4y ago
  Why a reader macro? Just use s-expressions, like sbcl and ccl.
  - guenthert 4y ago
dgan 4y ago
so I started learning common lisp lately. It's insane how one can combine very high level language, with down-to-metal compilation. I think that's the only language allowing it? Absolutely impressive. Feels like writing python but compiling to native
- krastanov 4y ago
  Julia has similar capabilities, probably because it was heavily inspired by lisps. You can even modify Julia's compiler from within Julia. I often write code at a python level of abstraction and then use julia introspection to check the machine code that was generated.
  - celeritascelery 4y ago
    So if I understand correctly, in Julia you programmatically look at generated machine code? Is there a way to modify it, or is just for making sure some optimizations were applied?
    - adgjlsfhk1 4y ago
      @code_native just lets you look at generated code, but Julia also uses macros frequently to give the compiler hints about how to compile your code. Some examples are @inbounds which disables bounds checks, @fastmath which is the local version of C/Fortran's --math-mode=fast, @simd which lets the compiler assume it can re-order loops (it will do so anyway if it can prove you won't notice). If you need more fine grained control (which is very rare) you can also emit LLVM bytecode (or direct assembly) directly.
    - eigenspace 4y ago
      We can modify the code at a few different levels. The easiest level is our untyped intermediate representation. The next easiest level is to modify things at the level of the LLVM code which is basically one step above assembler, and almost always better to work on than direct machine code (also machine code can be embedded in LLVM code if you need to). You can also use https://github.com/YingboMa/AsmMacro.jl https://github.com/YingboMa/AsmMacro.jl if you like. We are also working out interfaces to make it easier to programatically work on our typed IR through a technique and set of interfaces known as "abstract interpretation".
vitiral 4y ago
Okay let me get this straight... Are you saying that I don't need to use parenthesis as much in lisp? Instead I could define a few "reader macros" and then immediately have a different syntax? Say if I wanted to change (if (test-clause) (action1) (action2)) To if (test-clause) do (action1) else (action2) Is this actually possible inline in lisp, meaning I just need to import a library that implements the if-reader-macro?
- sph 4y ago
  For this concept of user-defined languages/DSLs, look no further than Racket.
- fiddlerwoaroof 4y ago
  This works in many Common Lisp implementations: https://readable.sourceforge.io https://readable.sourceforge.io
  - vitiral 4y ago
    Cool stuff, thanks.
- kazinator 4y ago
  Yes; for isntance see the CMU infix.cl file from the 1990's: https://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/lisp/code/syntax/infix/infix.cl https://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/... It has test cases at the bottom. like checking that f(a)*=g(b) expands to (setf (f a) (* (f a) (g b)))) The module installs the syntax under the #i dispatch characters; you wrap the infix expression with #I( ... )

93 points by susam 4y ago

timonoko 4y ago

Apropos. Is there reader macro definitions for "(" and ")" ? Or is Gödel rolling in his grave for such insolent heresy?

sph 4y ago
It would be a noop. A reader macro that takes a s-expr bounded by parentheses, and returns a s-expr, literally is just the identity function^Hmacro.
- stassats 4y ago
  It's not. Reader macros don't take s-expressions, they take characters.
stassats 4y ago
Sure, what's so special about it? (get-macro-character #\() => SB-IMPL::READ-LIST (get-macro-character #\)) => SB-IMPL::READ-RIGHT-PAREN (that one just signals an error, the read-list one picks up the closing #\))
- timonoko 4y ago
  In other words : You cannot redefine ")" because it cannot be undefined while reading the new definition.
  - aidenn0 4y ago
    No, it can totally be redefined. Reading happens before evaluation
  - creepycrawler 4y ago
    Sure you can. CL-USER> (set-macro-character #\) (get-macro-character #\()) T CL-USER> )format t "Hello") Hello NIL
  - thelopa 4y ago
    That’s completely incorrect. The parent comment has it right. In normal usage, the meaning of #\) is entirely determined by the implementation for the #\( reader macro. This happens because the reader macro for #\( keeps consuming characters until it has located AND consumed the matching #\). The reader would only run the #\) reader macro if it encountered a #\) that didn’t have a corresponding #\(. That’s what they meant when they said “it signals an error”. Specifically, it signals that there are unbalanced parentheses. If you redefine the #\) reader macro, you could use it in a top-level context. That’s probably not a good idea since it’s easy to accidentally have too many #\) when closing a deeply nested expression and thus accidentally invoke the macro. Aside from signaling an error, the #\) reader macro is also useful because it changes the rules when reading symbols. Basically, if you write (+ foo bar), the existence of the #\) macro helps the reader know that you’re referencing the “bar” symbol rather than the “bar)” symbol. Generally, when people define new balanced-pair syntax for Common Lisp, (such as a #\{ macro for hash tables) they will follow the same pattern and define a corresponding reader macro for the closing side that always signals an error for all the same reasons. Edit: also, as others have pointed out, you seem to be mistakenly assuming that the redefinition takes affect mid-way through reading the expression. That’s not how CL works. CL cleanly separates the process of executing code into a few distinct phases. First, the reader reads an entire expression (“form” in lisp terms). Then, that form is macroexpanded (traditional macros, not reader macros!) as needed before (optionally) being compiled and then executed. The change to the read table would happen during the execution phase — well ordered after the original characters for that form are out of the picture. You COULD force a change to the readtable mid-way through reading a form using the #. reader macro, but that definitely gets into chainsaw-juggling territory.
    - timonoko 4y ago
      Ok. I understood my error in about 5.3 second but let it stay, because it was funny, and would trigger somebody to generate a wall-chart of explanations. Anyways already in 1982 I was diabolically opposed to this kind of shit. Read and Print should be as simple as possible and always one-to-one. When you print something to disk, that is what you get when reading, no additional adjustment needed. If you want reading macros, for example, you make your own Read. Better Read could even be in standard package "Additional-macros-for-common-lisp".

sph 4y ago

Great timing. It's my turn to write a Lisp, and I was just thinking about implementing reader macros this morning :-) My goal is to create the barest-metal Lisp OS (basically Lisp REPL with full ring-0 access, the OS) with an asm reader macro to write low level assembly opcodes you can jmp into. (defn add1 (x) (declare x int64) #asm( mov rax, %x add rax, 1 )) (Just thought of this syntax on the spot, I'm a few dozen hours away from that still.)

timonoko 4y ago
1970's calling. Been there. Done that: https://news.ycombinator.com/item?id=33517092#33517797 https://news.ycombinator.com/item?id=33517092#33517797
- sph 4y ago
  Thanks for the inspiration. The fact that you've already done it won't stop me. Got more details/links to study?
  - timonoko 4y ago
    https://github.com/timonoko/nokolisp_addons/blob/master/assembly.lsp https://github.com/timonoko/nokolisp_addons/blob/master/asse... Keyword is the line (dos-eval '(debug < TO.DEB > NUL)) ("mapc" and "map" had their parameters in wrong order, which annoys me even today)
macoovacany 4y ago
https://movitz.common-lisp.dev/ https://movitz.common-lisp.dev/
moonchild 4y ago
Why a reader macro? Just use s-expressions, like sbcl and ccl.
- guenthert 4y ago

dgan 4y ago

so I started learning common lisp lately. It's insane how one can combine very high level language, with down-to-metal compilation. I think that's the only language allowing it? Absolutely impressive. Feels like writing python but compiling to native

krastanov 4y ago
Julia has similar capabilities, probably because it was heavily inspired by lisps. You can even modify Julia's compiler from within Julia. I often write code at a python level of abstraction and then use julia introspection to check the machine code that was generated.
- celeritascelery 4y ago
  So if I understand correctly, in Julia you programmatically look at generated machine code? Is there a way to modify it, or is just for making sure some optimizations were applied?
  - adgjlsfhk1 4y ago
    @code_native just lets you look at generated code, but Julia also uses macros frequently to give the compiler hints about how to compile your code. Some examples are @inbounds which disables bounds checks, @fastmath which is the local version of C/Fortran's --math-mode=fast, @simd which lets the compiler assume it can re-order loops (it will do so anyway if it can prove you won't notice). If you need more fine grained control (which is very rare) you can also emit LLVM bytecode (or direct assembly) directly.
  - eigenspace 4y ago
    We can modify the code at a few different levels. The easiest level is our untyped intermediate representation. The next easiest level is to modify things at the level of the LLVM code which is basically one step above assembler, and almost always better to work on than direct machine code (also machine code can be embedded in LLVM code if you need to). You can also use https://github.com/YingboMa/AsmMacro.jl https://github.com/YingboMa/AsmMacro.jl if you like. We are also working out interfaces to make it easier to programatically work on our typed IR through a technique and set of interfaces known as "abstract interpretation".

vitiral 4y ago

Okay let me get this straight... Are you saying that I don't need to use parenthesis as much in lisp? Instead I could define a few "reader macros" and then immediately have a different syntax? Say if I wanted to change (if (test-clause) (action1) (action2)) To if (test-clause) do (action1) else (action2) Is this actually possible inline in lisp, meaning I just need to import a library that implements the if-reader-macro?

sph 4y ago
For this concept of user-defined languages/DSLs, look no further than Racket.
fiddlerwoaroof 4y ago
This works in many Common Lisp implementations: https://readable.sourceforge.io https://readable.sourceforge.io
- vitiral 4y ago
  Cool stuff, thanks.
kazinator 4y ago
Yes; for isntance see the CMU infix.cl file from the 1990's: https://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/lisp/code/syntax/infix/infix.cl https://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/... It has test cases at the bottom. like checking that f(a)*=g(b) expands to (setf (f a) (* (f a) (g b)))) The module installs the syntax under the #i dispatch characters; you wrap the infix expression with #I( ... )