5 ms·

> The reader doesn't know about opperands or operators, but it certainly knows a symbol from an integer or string. That's the s-expression syntax. Beyond that,

• kazinator 4y ago
  You used the argument that the Lisp reader isn't parsing Lisp because it doesn't actually know Lisp; it has no idea what the operators mean. But, above, the message "error while parsing arguments to DEFMACRO DO" is just a destructuring error on "macro lambda list" pattern match on a piece of datum: the unevaluated macro call. That pattern matcher has no idea what DO means, in much same way that the reader doesn't know what DO means. The reader has done so much work, that the remaining parsing is just a very easy job which enjoys easy positional access to everything in the tree, at every level, at worst having to loop over a sequence of similar items. The syntax notation like: do ({var | (var [init-form [step-form]])}*) (end-test-form result-form*) declaration* {tag | statement}* is relying on read syntax; it has parentheses in it! Moreover, it is a greatly condensed specification, due to its reliance on the properties of the read syntax. The init-form can be anything: a struct literal, vector, string, character. Yet, I don't see the rules for that in this grammar; they given elsewhere, and just assumed here. If everything were to be made explicit here, the way a handful of parentheses have been made explicit, it would be pretty large specification, of which the do-specific parts above would be small. The do-specific material is just a handful of self-contained patterns applied to individual objects that have already been parsed out by the reader. For instance, the pattern pattern {var | (var [init-form [step-form]])}* is understood to be applied to that part of the form which can be conveniently retrieved as (cadr form). The amount of work left is very small, of low complexity. The pattern is not a recursive language; it is a regular language (if we pretend that var, init-form and step-form are terminal symbols). When the (var init-form step-form) is matched against the corresponding object, the step-form is easily retrieved by position, regardless of the complexity of init-form. This is not true of the unparsed expression in which scanning across an init-form to get to step-form is a complex step. The reader has done the work which allows this stage of processing to treat "init-form" as if it were a terminal symbol in a simple regular grammar that is just concerned with a positionally-determined portion of the do syntax. It looks like Lisp has a lot of syntax beyond the read syntax if we look at all of the forms together. They all share the read syntax, but have their unique bits that drive their shape. The more of them there are, the larger is the proportion of the language syntax that is contributed by those unique bits. Each time we write a macro and document it, the read syntax doesn't grow, but the amount of deep syntax does. From the perspective of any one S-expression, though, the bulk of the specification of its syntax is in the domain of the reader. The pattern match for a specific shape like do is a small amount of easily digestible additional information.
  • lispm 4y ago
    > That pattern matcher has no idea what DO means, in much same way that the reader doesn't know what DO means. 'means' is in the domain of semantics. We talk about syntax: structure. But what we know at this stage: DO is the CL:DO macro. So we know the actual operator, not just the symbol. We also know that it is a macro. We also know that the macro has a macro lambda list (CLHS 3.4.4 describes the syntax of those). We can now check the macro form against the actual CL:DO macro lambda list structure. And so on... Thus we are now talking about the Lisp language, its operators and its structure requirements. That's the purpose of a language parser: it has to know which symbols/identifiers are actual elements of the language, which operators these actually are and retrieves the syntactic constraints/rules for those. In a language like Common Lisp (or similar dialects) this is 'difficult', since there are lexical bindings, compile-time side-effects and even lexical bindings to establish new syntax (macrolet). > The syntax notation like: ... is relying on read syntax; it has parentheses in it! It describes constraints on the structure (!) of valid Lisp forms. -> syntax! (do () () "foo") -> Ill-formed DO end test list (do () (t) "foo") -> no legal go tag (do ((a b c d)) (t)) -> illegal form for a DO varlist See that the warnings talk about the actual Lisp operators and their requirements? We've left the domain of s-expression syntax here and talk about Lisp syntax requirements. None of these things are caught by the READER. All of these are actual warnings from SBCL. > The init-form can be anything: a struct literal, vector, string, character. Yet, I don't see the rules for that in this grammar; they given elsewhere, and just assumed here. The rules are in the language standard. The non-terminal INIT-FORM is specified to be a FORM. Which then is defined elsewhere. > The init-form can be anything Actually it can't. It must be a form (see the CLHS syntax description for DO). Thus ((foo)) for example is not valid, since it is not a valid Common Lisp form. > From the perspective of any one S-expression, though, the bulk of the specification of its syntax is in the domain of the reader. This is tautologic. I was claiming that Lisp has structural constraints/rules on top (!) of s-expressions (and especially s-expressions as data) and gave the example of the Common Lisp standard, which describes these in an eBNF syntax.