3 ms·

You cannot. Because Lisp is homoiconic, you don't code in (what I personally believe to be) reasonable syntax, you code in abstract syntax trees. Lisp is so eas

You cannot. Because Lisp is homoiconic, you don't code in (what I personally believe to be) reasonable syntax, you code in abstract syntax trees. Lisp is so easy to metaprogram because everything is parentheses and the burden is on the user, not the parser, to determine the program's abstract syntactic structure. So, Lisp can only metaprogram its own syntax, you can't introduce a C-like syntax.

lisper 10y ago
> You cannot. You can. Reader macros let you put whatever syntax you want on top of Sexprs. For example: Welcome to Clozure Common Lisp Version 1.10-r16479M (DarwinX8664)! ? (spark-init) #P"/Users/ron/devel/spark/ergolib/init.lisp" ? (require :parcil) ... ? (in-readtable infix) |NIL| ? infix(x=1.23+4.56) 5.79 ? sin(x*x+1) 0.03341171 ? The code for this is here: https://github.com/rongarret/ergolib https://github.com/rongarret/ergolib
- wcrichton 10y ago
  For sure, I was just considering normal macros.
  - lisper 10y ago
    No, you were just flat-out wrong. > Lisp can only metaprogram its own syntax, you can't introduce a C-like syntax. This is a flat-out false statement, and it reflects a deep but sadly common misunderstanding of how Lisp works and why it's cool. Even if Lisp did not have reader macros as a standard feature, you could still write a C compiler in Lisp more easily than you could write one in any other language. The whole concept of "metaprogramming a syntax" (so that you can talk about whether or not a language can "only metaprogram its own syntax") is non-sensical, a category error. Metaprogramming is simply writing programs that write programs. What makes Lisp cool is that it separates the syntax from the program. A Lisp program, unlike all other languages, is not text. A Lisp program is a data structure. Lisp happens to define a textual surface syntax (S-expressions) that allows you to easily convert text into the particular data structures that are Lisp programs (and also incidentally data structures that are not Lisp programs) but you can also produce these data structures in other ways, like, for example, writing programs that produce them. The textual surface syntax is a detail. Writing an infix parser in Lisp is an elementary exercise, and you could use that parser to parse C-like code whether or not you had reader macros. All reader macros let you do is seamlessly integrate that C-like syntax into the Lisp REPL rather than having to embed your new language in strings or read it from files.
    - wcrichton 10y ago
      Yes I agree with you, I am not trying to say it is impossible to write a C interpreter/compiler in Lisp. Any Turing-complete language can do that. I'm more so interested in the built-in metaprogramming capabilities, and it's evident that reader macros go far in that direction. On a different note, would you say that Lisp's syntax is better because it makes an easy compile target (i.e. it's easier to compile C into Lisp instead of x86) or because it makes writing macros/AST transformations easier? I ask because my biggest beef with Lisp has always been that I thought its S-expressions to be less readable than other languages, but if you're arguing for Lisp's value as a compile target, then that makes a lot of sense.
      - lisper 10y ago
        > Lisp's syntax is better because it makes an easy compile target Not quite. Lisp's syntax is (mostly) irrelevant. Lisp's design is better because you don't have to worry about syntax to use Lisp as a compile target. There's a huge conceptual difference between (eval '(some code)) and (using Javascript or Python as an example) eval("Some code") In JS/Python case you are passing a string to EVAL. In the Lisp case you are passing a data structure to EVAL. That data structure has been generated by the reader (by virtue of using QUOTE) but that is not the important part. You could as easily have written: (eval (cons (quote some) (cons (quote code) nil))) or (eval (some-function-that-generates-code)) with the point being that some-function-that-generates-code does not return a string, it returns a data structure, so there is no syntax. The whole concept has evaporated because you're not dealing with strings any more. Syntax is for humans. When you have programs writing programs, syntax just gets in the way. But when your EVAL function takes a textual representation of a program as a string you have no choice but to muck with syntax. This is the reason that the misconception that syntax is essential is so widespread. It seems essential only because most programming languages don't distinguish between READ and EVAL.
        qwertyuiop924 10y ago
        point being, you can trivially parse code of any syntax, just like any other language. What makes lisp different is that once you have a parse tree of sexprs, you can then traverse that tree very easily: Lisp has tools built right into the language. You can then very easily transform that tree: In fact, you can write a hook whereby the code being written can provide arbitrary transformers, which are called by the compiler when seen in code: This is all a macro is: it's an API for code to hook into its own compiler: much like Rust's compiler addons. Unlike compiler addons, however, they don't depend on compiler internals: In fact, if a Lisp implementation didn't have macros and readtables, you could write a preprocessor to add them in (but gensym/hygene would be a pain to do). However, this would all be very confusing if the internal tree didn't look like the external representation. This is why Lisp is written in sexprs. The reason you see so many DSLs in lisp being written in a lispy syntax is that lisp hackers, like all other hackers, are fundamentally lazy: Since there's already a lisp syntax parser built into the language, they don't have to write their own, and macros often suffice for the most common reasons to write a DSL, which means that they didn't even have to write a compiler, they just hooked into the existing one. Besides, sexprs provide advantages even if you are writing your own language, and you can overload lisp's syntactic shortcuts for your own purposes.
dragonwriter 10y ago
> Because Lisp is homoiconic, you don't code in (what I personally believe to be) reasonable syntax, you code in abstract syntax trees. There's no reason a regular macro couldn't implement a basically C-like syntax (though it would have Lisp, not C, tokenization rules, to the extent that those are different) on what is passed as its argument. The actual call to the macro would have typical Lisp syntax, but what the macro consumes would be interpreted with whatever syntax was implemented by the macro. (And, a reader macro could implement more deeply C-like syntax.)
- wcrichton 10y ago
  I've never seen something like this before--do you know of any examples of this kind of macro usage?
  - dragonwriter 10y ago
    This article provides a simpler (but conceptually similar, where it comes to implementing syntax different than standard Lisp S-expression syntax) example of using reader macros to implement JSON literals in Lisp. https://gist.github.com/chaitanyagupta/9324402 https://gist.github.com/chaitanyagupta/9324402
- qwertyuiop924 10y ago
  Well, for regular macros, only if your C syntax was in a string. Otherwise, lisp would try to tokenize it, and choke. Heck, if your lisp of choice offers an eqivalent of TCL's uplevel, you wouldn't even need a macro, just a function.
  - dragonwriter 10y ago
    > Well, for regular macros, only if your C syntax was in a string. Otherwise, lisp would try to tokenize it, and choke. Which is why I said normal (rather than reader) macros could do a "basically C-like syntax" but with Lisp tokenization rules.
    - 10y ago
chriswarbo 10y ago
If you want to handle a C-like language, you just need to make a parser; once you've got an AST, that is your "Lisp syntax", so you can do all of your fancy metaprogramming, interpretation, compilation, etc. Using a parsing framework (e.g. parser combinators, parser generators, ometa, etc.) to parse a language which has a well-specified syntax (e.g. a BNF grammar) is pretty routine and mechanical; usually it just requires a one-to-one translation of the BNF form into the parser framework's syntax, then fiddling with ordering and precedence rules until your tests pass. Starting from scratch, I could knock out a C-like parser in maybe half an hour, e.g. using parsec or ometa. I'm sure Lisps have equivalents (I know Racket has built in support for defining new concrete synax)

lisper 10y ago

> You cannot. You can. Reader macros let you put whatever syntax you want on top of Sexprs. For example: Welcome to Clozure Common Lisp Version 1.10-r16479M (DarwinX8664)! ? (spark-init) #P"/Users/ron/devel/spark/ergolib/init.lisp" ? (require :parcil) ... ? (in-readtable infix) |NIL| ? infix(x=1.23+4.56) 5.79 ? sin(x*x+1) 0.03341171 ? The code for this is here: https://github.com/rongarret/ergolib https://github.com/rongarret/ergolib

wcrichton 10y ago
For sure, I was just considering normal macros.
- lisper 10y ago
  No, you were just flat-out wrong. > Lisp can only metaprogram its own syntax, you can't introduce a C-like syntax. This is a flat-out false statement, and it reflects a deep but sadly common misunderstanding of how Lisp works and why it's cool. Even if Lisp did not have reader macros as a standard feature, you could still write a C compiler in Lisp more easily than you could write one in any other language. The whole concept of "metaprogramming a syntax" (so that you can talk about whether or not a language can "only metaprogram its own syntax") is non-sensical, a category error. Metaprogramming is simply writing programs that write programs. What makes Lisp cool is that it separates the syntax from the program. A Lisp program, unlike all other languages, is not text. A Lisp program is a data structure. Lisp happens to define a textual surface syntax (S-expressions) that allows you to easily convert text into the particular data structures that are Lisp programs (and also incidentally data structures that are not Lisp programs) but you can also produce these data structures in other ways, like, for example, writing programs that produce them. The textual surface syntax is a detail. Writing an infix parser in Lisp is an elementary exercise, and you could use that parser to parse C-like code whether or not you had reader macros. All reader macros let you do is seamlessly integrate that C-like syntax into the Lisp REPL rather than having to embed your new language in strings or read it from files.
  - wcrichton 10y ago
    Yes I agree with you, I am not trying to say it is impossible to write a C interpreter/compiler in Lisp. Any Turing-complete language can do that. I'm more so interested in the built-in metaprogramming capabilities, and it's evident that reader macros go far in that direction. On a different note, would you say that Lisp's syntax is better because it makes an easy compile target (i.e. it's easier to compile C into Lisp instead of x86) or because it makes writing macros/AST transformations easier? I ask because my biggest beef with Lisp has always been that I thought its S-expressions to be less readable than other languages, but if you're arguing for Lisp's value as a compile target, then that makes a lot of sense.
    - lisper 10y ago
      > Lisp's syntax is better because it makes an easy compile target Not quite. Lisp's syntax is (mostly) irrelevant. Lisp's design is better because you don't have to worry about syntax to use Lisp as a compile target. There's a huge conceptual difference between (eval '(some code)) and (using Javascript or Python as an example) eval("Some code") In JS/Python case you are passing a string to EVAL. In the Lisp case you are passing a data structure to EVAL. That data structure has been generated by the reader (by virtue of using QUOTE) but that is not the important part. You could as easily have written: (eval (cons (quote some) (cons (quote code) nil))) or (eval (some-function-that-generates-code)) with the point being that some-function-that-generates-code does not return a string, it returns a data structure, so there is no syntax. The whole concept has evaporated because you're not dealing with strings any more. Syntax is for humans. When you have programs writing programs, syntax just gets in the way. But when your EVAL function takes a textual representation of a program as a string you have no choice but to muck with syntax. This is the reason that the misconception that syntax is essential is so widespread. It seems essential only because most programming languages don't distinguish between READ and EVAL.
      - qwertyuiop924 10y ago
        point being, you can trivially parse code of any syntax, just like any other language. What makes lisp different is that once you have a parse tree of sexprs, you can then traverse that tree very easily: Lisp has tools built right into the language. You can then very easily transform that tree: In fact, you can write a hook whereby the code being written can provide arbitrary transformers, which are called by the compiler when seen in code: This is all a macro is: it's an API for code to hook into its own compiler: much like Rust's compiler addons. Unlike compiler addons, however, they don't depend on compiler internals: In fact, if a Lisp implementation didn't have macros and readtables, you could write a preprocessor to add them in (but gensym/hygene would be a pain to do). However, this would all be very confusing if the internal tree didn't look like the external representation. This is why Lisp is written in sexprs. The reason you see so many DSLs in lisp being written in a lispy syntax is that lisp hackers, like all other hackers, are fundamentally lazy: Since there's already a lisp syntax parser built into the language, they don't have to write their own, and macros often suffice for the most common reasons to write a DSL, which means that they didn't even have to write a compiler, they just hooked into the existing one. Besides, sexprs provide advantages even if you are writing your own language, and you can overload lisp's syntactic shortcuts for your own purposes.

dragonwriter 10y ago

> Because Lisp is homoiconic, you don't code in (what I personally believe to be) reasonable syntax, you code in abstract syntax trees. There's no reason a regular macro couldn't implement a basically C-like syntax (though it would have Lisp, not C, tokenization rules, to the extent that those are different) on what is passed as its argument. The actual call to the macro would have typical Lisp syntax, but what the macro consumes would be interpreted with whatever syntax was implemented by the macro. (And, a reader macro could implement more deeply C-like syntax.)

wcrichton 10y ago
I've never seen something like this before--do you know of any examples of this kind of macro usage?
- dragonwriter 10y ago
  This article provides a simpler (but conceptually similar, where it comes to implementing syntax different than standard Lisp S-expression syntax) example of using reader macros to implement JSON literals in Lisp. https://gist.github.com/chaitanyagupta/9324402 https://gist.github.com/chaitanyagupta/9324402
qwertyuiop924 10y ago
Well, for regular macros, only if your C syntax was in a string. Otherwise, lisp would try to tokenize it, and choke. Heck, if your lisp of choice offers an eqivalent of TCL's uplevel, you wouldn't even need a macro, just a function.
- dragonwriter 10y ago
  > Well, for regular macros, only if your C syntax was in a string. Otherwise, lisp would try to tokenize it, and choke. Which is why I said normal (rather than reader) macros could do a "basically C-like syntax" but with Lisp tokenization rules.
  - 10y ago

chriswarbo 10y ago

If you want to handle a C-like language, you just need to make a parser; once you've got an AST, that is your "Lisp syntax", so you can do all of your fancy metaprogramming, interpretation, compilation, etc. Using a parsing framework (e.g. parser combinators, parser generators, ometa, etc.) to parse a language which has a well-specified syntax (e.g. a BNF grammar) is pretty routine and mechanical; usually it just requires a one-to-one translation of the BNF form into the parser framework's syntax, then fiddling with ordering and precedence rules until your tests pass. Starting from scratch, I could knock out a C-like parser in maybe half an hour, e.g. using parsec or ometa. I'm sure Lisps have equivalents (I know Racket has built in support for defining new concrete synax)