5 ms·

I do want to add that Lisp metaprogramming does one big trick for DSL design in that it works at the syntactic level and ignores binding. Thus all the tricks ar

by tel 7y ago

I do want to add that Lisp metaprogramming does one big trick for DSL design in that it works at the syntactic level and ignores binding. Thus all the tricks around hygenics and anaphoria etc. Since ignoring binding means that you only deal with the call graph implicitly through working with the AS tree, you get an opportunity to pareto your way through tougher DSL nuggets.

TeMPOraL 7y ago
What do you mean by "ignoring binding"? That doesn't sound right. Binding is one of the things I think about the most when building DSLs in Common Lisp.
- tel 7y ago
  Yes, the user has to since the system ignores it. A lisp macro operates, generally, over the abstract syntax tree as opposed to the "abstract binding tree" where alpha equivalences have already been factored out. Or, concretely, a lisp macro distinguishes between syntax `(my-let (x 3) x)` and `(my-let (y 3) y)` even though they are, under implied semantics, identical. In that way, lisp macros have little to say about binding.
  - TeMPOraL 7y ago
    I still don't understand; probably because I'm not proficient in the mathematical underpinning of this. The way I see it, a (Common) Lisp macro can, but does not have to, distinguish between the two forms you've shown. Assuming `my-let' is the macro, from it's POV, `x' is a value. A reasonable implementation of "let" won't care about that value beyond it being a symbol comparable to other symbols by `eq'. This way, `(my-let (x 3) x)' and `(my-let (y 3) y)' will expand to effectively the same code - and after the compiler gets rid of symbol names altogether, the resulting code will be actually the same.
    - tel 7y ago
      The syntactic approach subsumes the more semantic approach where the graph structure is made explicit. So, yeah, you can "compile" list structures to the binding graph and indeed you must. Having more granular distinctions always helps in that it's easy to "forget" how things are different. It's also more primitive in that you have to actually do that step. It also leads to opportunities to do "wrong" things like anaphoric macros. This is essentially a tradeoff where more power is offered to the author of the macro and less to the reader. To make another comparison, we can take it one step further: a macro preprocessor which had access to the actual character stream can elect to do things not possible in lisp macros—e.g., distinguish on indentation or something to that effect—but also can recover all of the lisp macro techniques by merely parsing the character stream. --- The heart of all of this, popping the stack up to my original comment, is that for lots of macros you don't really need to care at all about binding structure—or at least only minimally—and thus you can get lots of practical work done doing "just" syntactic transforms. Once you are really working over a complex call graph—which is common in more complex DSLs like probabilistic programming languages—then CPS becomes really important as you want a way to split call graphs into pieces and a practical way of doing that is (value + continuation). Worth noting, CPS transformations are a popular technique in more sophisticated lispy macro transforms, too.
      - neilv 7y ago
        BTW, Racket syntax transformations work with Racket syntax objects [1], which is one way to do the indentation sensitivity you mention. Racket also has something called `#lang` for conveniently using arbitrary parsers, and a way to embed use of arbitrary readers for chunks of code within normal Racket code. [1] https://docs.racket-lang.org/reference/stxops.html https://docs.racket-lang.org/reference/stxops.html
      - TeMPOraL 7y ago
        > a macro preprocessor which had access to the actual character stream can elect to do things not possible in lisp macros—e.g., distinguish on indentation or something to that effect—but also can recover all of the lisp macro techniques by merely parsing the character stream. Common Lisp has the mechanism called "reader macros" that can do that. It's a separate mechanism, because it works at "read time", which is much earlier than the "macroexpansion time" when the regular macros are processed.

by tel 7y ago

TeMPOraL 7y ago

What do you mean by "ignoring binding"? That doesn't sound right. Binding is one of the things I think about the most when building DSLs in Common Lisp.

tel 7y ago
Yes, the user has to since the system ignores it. A lisp macro operates, generally, over the abstract syntax tree as opposed to the "abstract binding tree" where alpha equivalences have already been factored out. Or, concretely, a lisp macro distinguishes between syntax `(my-let (x 3) x)` and `(my-let (y 3) y)` even though they are, under implied semantics, identical. In that way, lisp macros have little to say about binding.
- TeMPOraL 7y ago
  I still don't understand; probably because I'm not proficient in the mathematical underpinning of this. The way I see it, a (Common) Lisp macro can, but does not have to, distinguish between the two forms you've shown. Assuming `my-let' is the macro, from it's POV, `x' is a value. A reasonable implementation of "let" won't care about that value beyond it being a symbol comparable to other symbols by `eq'. This way, `(my-let (x 3) x)' and `(my-let (y 3) y)' will expand to effectively the same code - and after the compiler gets rid of symbol names altogether, the resulting code will be actually the same.
  - tel 7y ago
    The syntactic approach subsumes the more semantic approach where the graph structure is made explicit. So, yeah, you can "compile" list structures to the binding graph and indeed you must. Having more granular distinctions always helps in that it's easy to "forget" how things are different. It's also more primitive in that you have to actually do that step. It also leads to opportunities to do "wrong" things like anaphoric macros. This is essentially a tradeoff where more power is offered to the author of the macro and less to the reader. To make another comparison, we can take it one step further: a macro preprocessor which had access to the actual character stream can elect to do things not possible in lisp macros—e.g., distinguish on indentation or something to that effect—but also can recover all of the lisp macro techniques by merely parsing the character stream. --- The heart of all of this, popping the stack up to my original comment, is that for lots of macros you don't really need to care at all about binding structure—or at least only minimally—and thus you can get lots of practical work done doing "just" syntactic transforms. Once you are really working over a complex call graph—which is common in more complex DSLs like probabilistic programming languages—then CPS becomes really important as you want a way to split call graphs into pieces and a practical way of doing that is (value + continuation). Worth noting, CPS transformations are a popular technique in more sophisticated lispy macro transforms, too.
    - neilv 7y ago
      BTW, Racket syntax transformations work with Racket syntax objects [1], which is one way to do the indentation sensitivity you mention. Racket also has something called `#lang` for conveniently using arbitrary parsers, and a way to embed use of arbitrary readers for chunks of code within normal Racket code. [1] https://docs.racket-lang.org/reference/stxops.html https://docs.racket-lang.org/reference/stxops.html
    - TeMPOraL 7y ago
      > a macro preprocessor which had access to the actual character stream can elect to do things not possible in lisp macros—e.g., distinguish on indentation or something to that effect—but also can recover all of the lisp macro techniques by merely parsing the character stream. Common Lisp has the mechanism called "reader macros" that can do that. It's a separate mechanism, because it works at "read time", which is much earlier than the "macroexpansion time" when the regular macros are processed.