3 ms·

CL has huge mounds of syntax. Every macro or special form adds to it.

kazinator 4y ago
Macros don't add syntax; they just add semantics to syntax that already exists. You can write down the syntax of a call to a macro which has not yet been written, and Lisp will read it for you, giving you the tree. It just won't be able to expand it since the semantic definition through the macro name isn't there. A conventional language will not do that. C++11 will not correctly read the shape of a C++17 phrase structure, and just complain about it not being defined. Lisp macro can treat their arguments as a flat stream of tokens and parse them, even with LALR(1) parser generator or whatever. It's not not the typical way most macros are written. Macros leverage the ability of the invocation syntax to express pre-parsed nested structures. Even if a macro goes crazy like that, it's still self-contained; the churning toilet bowl of parsing it has whipped up is cleanly confined within the boundaries of the macro call, which can be cleanly substituted anywhere an expression can be used.
- munch117 4y ago
  If you don't know the syntax of let, then you might find yourself writing (let (x 5) (* x x)) where you should have written (let ((x 5)) (* x x)) That is syntax. Let me ask you a question: Are these two expressions equivalent? (f (g x)) and (let ((y (g x))) (f y)) In the absence of macros, the answer is yes. In the presence of macros, there's no way of knowing until you've seen the definition of f: If f is a macro, they might not be. You need to know the syntax of the f macro to answer the question. CL has a core language with a simple syntax. It also has a lot of macros in its standard library.
  - kazinator 4y ago
    Even though (let (x 5) (* x x)) is wrong, it parses. We know that it has an operator, followed by two arguments, and the first argument is made up of two things. Because it parses, I don't have to correct the author's punctuation, just their understanding of the necessary shape . The wrong shape they wrote has a coherent syntax, and we can discuss the problem in terms of differences in syntax, rather than differences between gibberish and valid syntax. This is why your question is able to be, "are these expressions equivalent" and not "are these two token sequences equivalent". I can write a TDD test case for a new macro that doesn't exist yet, which won't bring down the program. E.g. (test (my:let (x 5) (+ x x)) 10) This will catch the error that foo is not defined and report that the test case failed instead of returning the expected 42 value. The next test case in the same file can execute. You couldn't have a test for new C++ or Java syntax, in a C++ or Java file being processed by a compiler that wasn't maintained in order to recognize the syntax. The file will blow up at parse time, not allowing any of the test cases in it to proceed. > You need to know the syntax of the f macro to answer the question. But I can see what that syntax is: it's (f (g x)) right there: i.e. /\ f /\ /\ () g /\ x () g and x are clumped together, so they have a closer relationship of some kind than either one has with f, at least syntactically. From looking at the use, I don't know whether it is valid shape for f, or if so, what other shapes are valid. And I don't know the semantics at all. I'm not struggling with syntax, though, just connecting it with a meaning.
    - munch117 4y ago
      > Even though (let (x 5) (* x x)) is wrong, it parses. That's not the definition of syntax. If you're implementing a compiler, then you might use "syntax" as a shorthand for those parts of the syntax that are processed in a particular "syntax checking" stage of the compiler. But that's not all it is. > From looking at the use, I don't know whether it is valid shape for f That, on the other hand, is close to a definition of syntax: The rules for what has a valid shape and what doesn't.
      - deleted 4y ago
        [deleted]
- lispm 4y ago
  > Lisp will read it for you, giving you the tree The Lisp reader just reads s-expressions. It knows nothing about Lisp syntax and thus does not check it.
  - kazinator 4y ago
    In fact, in some cases there is nothing to do, syntactically, after the reader is done. An expression like (+ (* x y) (sqrt z)), after being read, is ready for evaluation by performing the function calls. (In principle; of course since we have macros, it has to be walked.) It was the reader which settled all the syntactic facts about what operation is applied to what operands. At this point we could move the goalposts: even eval and compile don't know anything about Lisp syntax. Because, look: (- 10) and (- 10 1) behave differently: the 10 operand changes semantic role between minuend and subtrahend based on syntax. All that eval knows is that it's passing arguments. A function can treat its argument values as the tokens of a syntax which are shored up according to arbitrary phrase structure rules, similarly to Unix programs like find and tcpdump.
    - lispm 4y ago
      > It was the reader which settled all the syntactic facts about what operation is applied to what operands. The reader does not know anything about operands or operators. It does not even know if the thing read is data or code. It also does not know if it is valid code. Enter (3 4 *) and the reader happily takes that expression, not caring about operators and operands. The reader will print zero warning about such expressions. To really know what the symbols in an expression actually are, one would need eval, compile or a code walker. Are they used as literal symbols? variables? functions? macros? block names? type names? ... > Because, look: (- 10) and (- 10 1) behave differently: the 10 operand changes semantic role between minuend and subtrahend based on syntax. All that eval knows is that it's passing arguments. That's semantics, not syntax. EVAL and COMPILE check that it is a function call, what arglist it expects (zero or more args, &optional args, &rest args, keyword args, and process them recursively. But Lisp has also special operators, macro operators and a lambda expression. There the syntax requirements are different or even context depended. Also if Lisp reads the expression (foo * +), the reader does not know if FOO is a function or a macro, it thus does not know what * and + are. Are they variables? Are they operators? Something else? They don't even need to have a definition at read time.

Macros don't add syntax; they just add semantics to syntax that already exists. You can write down the syntax of a call to a macro which has not yet been written, and Lisp will read it for you, giving you the tree. It just won't be able to expand it since the semantic definition through the macro name isn't there. A conventional language will not do that. C++11 will not correctly read the shape of a C++17 phrase structure, and just complain about it not being defined. Lisp macro can treat their arguments as a flat stream of tokens and parse them, even with LALR(1) parser generator or whatever. It's not not the typical way most macros are written. Macros leverage the ability of the invocation syntax to express pre-parsed nested structures. Even if a macro goes crazy like that, it's still self-contained; the churning toilet bowl of parsing it has whipped up is cleanly confined within the boundaries of the macro call, which can be cleanly substituted anywhere an expression can be used.

munch117 4y ago
If you don't know the syntax of let, then you might find yourself writing (let (x 5) (* x x)) where you should have written (let ((x 5)) (* x x)) That is syntax. Let me ask you a question: Are these two expressions equivalent? (f (g x)) and (let ((y (g x))) (f y)) In the absence of macros, the answer is yes. In the presence of macros, there's no way of knowing until you've seen the definition of f: If f is a macro, they might not be. You need to know the syntax of the f macro to answer the question. CL has a core language with a simple syntax. It also has a lot of macros in its standard library.
- kazinator 4y ago
  Even though (let (x 5) (* x x)) is wrong, it parses. We know that it has an operator, followed by two arguments, and the first argument is made up of two things. Because it parses, I don't have to correct the author's punctuation, just their understanding of the necessary shape . The wrong shape they wrote has a coherent syntax, and we can discuss the problem in terms of differences in syntax, rather than differences between gibberish and valid syntax. This is why your question is able to be, "are these expressions equivalent" and not "are these two token sequences equivalent". I can write a TDD test case for a new macro that doesn't exist yet, which won't bring down the program. E.g. (test (my:let (x 5) (+ x x)) 10) This will catch the error that foo is not defined and report that the test case failed instead of returning the expected 42 value. The next test case in the same file can execute. You couldn't have a test for new C++ or Java syntax, in a C++ or Java file being processed by a compiler that wasn't maintained in order to recognize the syntax. The file will blow up at parse time, not allowing any of the test cases in it to proceed. > You need to know the syntax of the f macro to answer the question. But I can see what that syntax is: it's (f (g x)) right there: i.e. /\ f /\ /\ () g /\ x () g and x are clumped together, so they have a closer relationship of some kind than either one has with f, at least syntactically. From looking at the use, I don't know whether it is valid shape for f, or if so, what other shapes are valid. And I don't know the semantics at all. I'm not struggling with syntax, though, just connecting it with a meaning.
  - munch117 4y ago
    > Even though (let (x 5) (* x x)) is wrong, it parses. That's not the definition of syntax. If you're implementing a compiler, then you might use "syntax" as a shorthand for those parts of the syntax that are processed in a particular "syntax checking" stage of the compiler. But that's not all it is. > From looking at the use, I don't know whether it is valid shape for f That, on the other hand, is close to a definition of syntax: The rules for what has a valid shape and what doesn't.
    - deleted 4y ago
      [deleted]
lispm 4y ago
> Lisp will read it for you, giving you the tree The Lisp reader just reads s-expressions. It knows nothing about Lisp syntax and thus does not check it.
- kazinator 4y ago
  In fact, in some cases there is nothing to do, syntactically, after the reader is done. An expression like (+ (* x y) (sqrt z)), after being read, is ready for evaluation by performing the function calls. (In principle; of course since we have macros, it has to be walked.) It was the reader which settled all the syntactic facts about what operation is applied to what operands. At this point we could move the goalposts: even eval and compile don't know anything about Lisp syntax. Because, look: (- 10) and (- 10 1) behave differently: the 10 operand changes semantic role between minuend and subtrahend based on syntax. All that eval knows is that it's passing arguments. A function can treat its argument values as the tokens of a syntax which are shored up according to arbitrary phrase structure rules, similarly to Unix programs like find and tcpdump.
  - lispm 4y ago
    > It was the reader which settled all the syntactic facts about what operation is applied to what operands. The reader does not know anything about operands or operators. It does not even know if the thing read is data or code. It also does not know if it is valid code. Enter (3 4 *) and the reader happily takes that expression, not caring about operators and operands. The reader will print zero warning about such expressions. To really know what the symbols in an expression actually are, one would need eval, compile or a code walker. Are they used as literal symbols? variables? functions? macros? block names? type names? ... > Because, look: (- 10) and (- 10 1) behave differently: the 10 operand changes semantic role between minuend and subtrahend based on syntax. All that eval knows is that it's passing arguments. That's semantics, not syntax. EVAL and COMPILE check that it is a function call, what arglist it expects (zero or more args, &optional args, &rest args, keyword args, and process them recursively. But Lisp has also special operators, macro operators and a lambda expression. There the syntax requirements are different or even context depended. Also if Lisp reads the expression (foo * +), the reader does not know if FOO is a function or a macro, it thus does not know what * and + are. Are they variables? Are they operators? Something else? They don't even need to have a definition at read time.