6 ms·

Lisp, Scheme, and Racket and related languages have many distinct syntax forms, just like other languages. The syntax forms appear visually similar due to the u

Lisp, Scheme, and Racket and related languages have many distinct syntax forms, just like other languages. The syntax forms appear visually similar due to the use of parentheses, but the forms themselves are distinct. For example, here are some of the distinct syntax forms in Racket (Scheme): (+ 3 4) # Procedure call (lambda (x) (+ x x)) # Lambda expression # See also case-lambda (let ((x 23) (y 42)) # Variable binding (+ x y)) # And also: # let*, letrec, letrec*, let-values, # let*-values, let-syntax, letrec-syntax, local (set! x 4) # Assignment (mutating) (define x 23) # Defined value (define (f x) # Defined procedure (+ x 42)) (quote a) # Quotation (#%datum . 10) # Quotation (keywords prohibited) # Conditionals (if (> 2 3) 'yes 'no) (cond ((> 3 2) 'greater) ((< 3 2) 'less)) (and a b) (or a b) (cond ('(1 2 3) => cadr) # Test clause (else #f)) # Else clause # Guarded evaluation (when (positive? -5) (display "hi")) (unless (positive? 5) (display "hi")) # Dispatch (case (+ 7 5) [(1 2 3) 'small] [(10 11 12) 'big]) # Sequencing (define x 0) (begin (set! x 5) (+ x 1)) # See also: begin0, begin-for-syntax # Iterations and Comprehensions # for, for/and, for/or, for/vector, for/hash, for/hasheq ... # Modules and imports (module id module-path form ...) (require ...) (provide ...) As you can see, although the language consistently uses parentheses, it has similar syntax structure to other languages: assignments, definitions, functions, switch/case, etc. Some of the syntax forms above have multiple variations, as well. See: https://docs.racket-lang.org/reference/syntax.html https://docs.racket-lang.org/reference/syntax.html

merlincorey 9y ago
I think your examples underscore the GP's point. Every single one of your examples is an S-Expression where the first element of the list is an operator/function followed by N arguments, from an abstract context-less view, anyway. In lisp there are atoms, lists, and expressions. They have uniform expression but not uniform meaning.
- jcrites 9y ago
  The notation uses S-expressions, that’s true, but the point that I was trying to make is that you can’t understand anything about the meaning of those S-expressions without parsing them in a context-aware way. It’s not as if there is just one form `(<op> <arg1> ... <argN>)` where the expression means invoking `op` with some arguments. A human or machine interpreting the code needs to be aware of these forms and the meaning/semantics of each one. Consider the following S-exps: (if a b) (set! a b) (define a b) They’re all three-term S-expressions, but you can’t understand their meaning without contextually interpreting the first term. There is a second layer of contextual syntax. Similarly: ((lambda (x y) (list y x)) 1 2) Consider the term `(x y)` in that expression. Without context, it looks like a call to the procedure `x` with `y` as input. With context, it’s a list of argument names for the lambda function. In languages like C# and Java there is just one meaning of: f(a, b); You know this is actually a procedure invocation with some arguments. There are trade offs to each approach. I’m not saying that one is better, just that there’s more to Scheme syntax than S-exps. Concatenative languages like Forth and Factor have even less syntax than Scheme.
  - taeric 9y ago
    Strictly, this isn't true. In f(a, b), if f is while, it is not a procedure call. Or if it is a declaration. Then, of course, there is the specifics of the call. Is it single dispatch? Where is it dispatched? Etc. Which isn't really to argue against the point. Yes, you need context. Pretty much always. Having gotten used to lisp, I find it much easier to understand. I realize familiarity helps, though.
- deleted 9y ago
  [deleted]
emidln 9y ago
Actually, almost of those are just function calls. Some of them happen during the compile phase, some during runtime. Things that happen at compile-time, say a let function call[0] e.g. (let ((x 23) (y 42)) (+ x y)) will return data as a return and compilation will continue (or stop because some sort of error/exception occurs). The important bit is at compile-time, a function invocation passes the rest of the form to the function as arguments without first evaluating them. Thus there is no syntax for quoting the list of binding forms. At runtime, a function invocation passes the rest of the form to the function as arguments but will evaluate all arguments before doing so. Knowing what is and isn't a macro is a bit tricky in some lisps (and in others there are sigils, reified tables listing the macro symbols, or metadata to let you know what you're dealing with). The only distinct syntax elements you provided are: quote - 'small strings - "hi" function call - (function arg0 arg1 argN) dotted pair - (datum . 10) sharp dispatch - #%datum Lisps do have syntax. Some provide notations for alternate data types like keywords or fixnums; data structures other than lists like hashes, sets, vectors, etc; or ways of dispatching at read-time. It is worth pointing out that all of the things you mentioned as syntax (which certainly are syntax in most other languages) are merely function calls in lisp. [0] There isn't a significant difference between a macro and a special form here. One is looked up in the application or prelude and is written in lisp, the other is provided by the compiler and possibly not written in lisp.
- jcrites 9y ago
  Most of the examples I gave above are core syntax forms. They are defined in the language grammar and are not function calls or macros. See the Racket language reference [0]: > A core syntactic form[] is parsed as described for each form in Syntactic Forms. Parsing a core syntactic form typically involves recursive parsing of sub-forms, and may introduce bindings that determine the parsing of sub-forms. top-level-form = general-top-level-form | (#%expression expr) | (module id module-path (#%plain-module-begin module-level-form ...)) | (begin top-level-form ...) | (begin-for-syntax top-level-form ...) general-top-level-form = expr | (define-values (id ...) expr) | (define-syntaxes (id ...) expr) | (#%require raw-require-spec ...) expr = id | (#%plain-lambda formals expr ...+) | (case-lambda (formals expr ...+) ...) | (if expr expr expr) | (begin expr ...+) | (begin0 expr expr ...) | (let-values ([(id ...) expr] ...) expr ...+) | (letrec-values ([(id ...) expr] ...) expr ...+) | (set! id expr) | (quote datum) | (quote-syntax datum) | (quote-syntax datum #:local) | (with-continuation-mark expr expr expr) | (#%plain-app expr ...+) | (#%top . id) | (#%variable-reference id) | (#%variable-reference (#%top . id)) | (#%variable-reference) See section 1.2.3.2 which specifies the expansion steps. Note that the expansion model differentiates between handling transformers aka macros and core syntax forms. To be fair, it seems that Scheme (R5RS) considers some of these forms to be "derived expression types" and specifies their definition using `define-syntax` [1]. However, its core syntax forms still include the following [2]: expression = variable | literal | procedure call | lambda expression | conditional | assignment | derived expression | macro use | macro block ... and there's a whole sub-grammar for specifying transformers (macros): <transformer spec> = (syntax-rules (<identifier>*) <syntax rule>*) <syntax rule> = (<pattern> <template>) <pattern> = <pattern identifier> | (<pattern>*) | (<pattern>+ . <pattern>) | (<pattern>* <pattern> <ellipsis>) | #(<pattern>*) | #(<pattern>* <pattern> <ellipsis>) | <pattern datum> <pattern datum> = <string> | <character> | <boolean> | <number> <template> = <pattern identifier> | (<template element>*) | (<template element>+ . <template>) | #(<template element>*) | <template datum> <template element> = <template> | <template> <ellipsis> Being able to write or evaluate Scheme fully requires an understanding of all of these distinct syntax forms. [0] https://docs.racket-lang.org/reference/syntax-model.html#%28part._fully-expanded%29 https://docs.racket-lang.org/reference/syntax-model.html#%28... [1] http://www.schemers.org/Documents/Standards/R5RS/HTML/ http://www.schemers.org/Documents/Standards/R5RS/HTML/ [2] http://www.schemers.org/Documents/Standards/R5RS/HTML/ http://www.schemers.org/Documents/Standards/R5RS/HTML/
ScottBurson 9y ago
Right. There's still syntax, but instead of that syntax being defined directly in terms of characters and tokens, it's defined in terms of s-expressions. Thus s-expressions serve as an intermediate syntactic level between tokens and forms. (Forms are what other languages call expressions and/or statements.)
sparkie 9y ago
This is why I love John Shutt's Kernel[0]. All of these "distinct forms" are rolled into one form of "(combiner combiniend)", where combiner is runtime evaluated and can either be a regular applicative (like a lisp procedure call), or an operative (reminiscent of older lisps fexpr). Compound operatives can be created using the primitive operative $vau. Applicatives are created with the primitive wrap. (which $lambda uses in the standard library). Of the above examples you posted: lambda, let, set!, cond, and, or, when, unless, etc, may all be implemented as a library in the Kernel Report. Some of the others aren't in the report but can be trivially implemented as compound operatives. This simplifies the compiler implementation to requiring only a handful of primitives. A distinguishing feature of Kernel is the lack of quote in the report. Although it has a trivial implementation ($define! $quote ($vau (x) #ignore x)), quote is considered harmful by Shutt due to the way it interacts with operatives. Also omitted from the language is macros, as their capability can be provided by first class operatives at the library level. The big downside to Kernel is performance. Compilation is generally not possible because the behavior of operatives can depend on runtime defined environments. This would be one of the reasons macros were chosen over fexprs in the 80s. It means Kernel is not a good language for number crunching, so it'd be nice to have an FFI for the cases where we need to invoke optimised code. (Currently no FFI spec exists). [0]:http://web.cs.wpi.edu/%7Ejshutt/kernel.html http://web.cs.wpi.edu/%7Ejshutt/kernel.html
- jcrites 9y ago
  Thank you for sharing this. I hadn't heard of Kernel, and it seems to be highly relevant to a line of inquiry that I've been pondering for some time (you could say light research) in the area of language design. There seem to be a lot of interesting topics in the related dissertation [1]. Thanks again! [1] https://web.wpi.edu/Pubs/ETD/Available/etd-090110-124904/unrestricted/jshutt.pdf https://web.wpi.edu/Pubs/ETD/Available/etd-090110-124904/unr...
lispm 9y ago
Good explanation. Many people get that wrong and think the syntax for s-expressions is the syntax for Lisp.

merlincorey 9y ago

I think your examples underscore the GP's point. Every single one of your examples is an S-Expression where the first element of the list is an operator/function followed by N arguments, from an abstract context-less view, anyway. In lisp there are atoms, lists, and expressions. They have uniform expression but not uniform meaning.

jcrites 9y ago
The notation uses S-expressions, that’s true, but the point that I was trying to make is that you can’t understand anything about the meaning of those S-expressions without parsing them in a context-aware way. It’s not as if there is just one form `(<op> <arg1> ... <argN>)` where the expression means invoking `op` with some arguments. A human or machine interpreting the code needs to be aware of these forms and the meaning/semantics of each one. Consider the following S-exps: (if a b) (set! a b) (define a b) They’re all three-term S-expressions, but you can’t understand their meaning without contextually interpreting the first term. There is a second layer of contextual syntax. Similarly: ((lambda (x y) (list y x)) 1 2) Consider the term `(x y)` in that expression. Without context, it looks like a call to the procedure `x` with `y` as input. With context, it’s a list of argument names for the lambda function. In languages like C# and Java there is just one meaning of: f(a, b); You know this is actually a procedure invocation with some arguments. There are trade offs to each approach. I’m not saying that one is better, just that there’s more to Scheme syntax than S-exps. Concatenative languages like Forth and Factor have even less syntax than Scheme.
- taeric 9y ago
  Strictly, this isn't true. In f(a, b), if f is while, it is not a procedure call. Or if it is a declaration. Then, of course, there is the specifics of the call. Is it single dispatch? Where is it dispatched? Etc. Which isn't really to argue against the point. Yes, you need context. Pretty much always. Having gotten used to lisp, I find it much easier to understand. I realize familiarity helps, though.
deleted 9y ago
[deleted]

emidln 9y ago

Actually, almost of those are just function calls. Some of them happen during the compile phase, some during runtime. Things that happen at compile-time, say a let function call[0] e.g. (let ((x 23) (y 42)) (+ x y)) will return data as a return and compilation will continue (or stop because some sort of error/exception occurs). The important bit is at compile-time, a function invocation passes the rest of the form to the function as arguments without first evaluating them. Thus there is no syntax for quoting the list of binding forms. At runtime, a function invocation passes the rest of the form to the function as arguments but will evaluate all arguments before doing so. Knowing what is and isn't a macro is a bit tricky in some lisps (and in others there are sigils, reified tables listing the macro symbols, or metadata to let you know what you're dealing with). The only distinct syntax elements you provided are: quote - 'small strings - "hi" function call - (function arg0 arg1 argN) dotted pair - (datum . 10) sharp dispatch - #%datum Lisps do have syntax. Some provide notations for alternate data types like keywords or fixnums; data structures other than lists like hashes, sets, vectors, etc; or ways of dispatching at read-time. It is worth pointing out that all of the things you mentioned as syntax (which certainly are syntax in most other languages) are merely function calls in lisp. [0] There isn't a significant difference between a macro and a special form here. One is looked up in the application or prelude and is written in lisp, the other is provided by the compiler and possibly not written in lisp.

jcrites 9y ago
Most of the examples I gave above are core syntax forms. They are defined in the language grammar and are not function calls or macros. See the Racket language reference [0]: > A core syntactic form[] is parsed as described for each form in Syntactic Forms. Parsing a core syntactic form typically involves recursive parsing of sub-forms, and may introduce bindings that determine the parsing of sub-forms. top-level-form = general-top-level-form | (#%expression expr) | (module id module-path (#%plain-module-begin module-level-form ...)) | (begin top-level-form ...) | (begin-for-syntax top-level-form ...) general-top-level-form = expr | (define-values (id ...) expr) | (define-syntaxes (id ...) expr) | (#%require raw-require-spec ...) expr = id | (#%plain-lambda formals expr ...+) | (case-lambda (formals expr ...+) ...) | (if expr expr expr) | (begin expr ...+) | (begin0 expr expr ...) | (let-values ([(id ...) expr] ...) expr ...+) | (letrec-values ([(id ...) expr] ...) expr ...+) | (set! id expr) | (quote datum) | (quote-syntax datum) | (quote-syntax datum #:local) | (with-continuation-mark expr expr expr) | (#%plain-app expr ...+) | (#%top . id) | (#%variable-reference id) | (#%variable-reference (#%top . id)) | (#%variable-reference) See section 1.2.3.2 which specifies the expansion steps. Note that the expansion model differentiates between handling transformers aka macros and core syntax forms. To be fair, it seems that Scheme (R5RS) considers some of these forms to be "derived expression types" and specifies their definition using `define-syntax` [1]. However, its core syntax forms still include the following [2]: expression = variable | literal | procedure call | lambda expression | conditional | assignment | derived expression | macro use | macro block ... and there's a whole sub-grammar for specifying transformers (macros): <transformer spec> = (syntax-rules (<identifier>*) <syntax rule>*) <syntax rule> = (<pattern> <template>) <pattern> = <pattern identifier> | (<pattern>*) | (<pattern>+ . <pattern>) | (<pattern>* <pattern> <ellipsis>) | #(<pattern>*) | #(<pattern>* <pattern> <ellipsis>) | <pattern datum> <pattern datum> = <string> | <character> | <boolean> | <number> <template> = <pattern identifier> | (<template element>*) | (<template element>+ . <template>) | #(<template element>*) | <template datum> <template element> = <template> | <template> <ellipsis> Being able to write or evaluate Scheme fully requires an understanding of all of these distinct syntax forms. [0] https://docs.racket-lang.org/reference/syntax-model.html#%28part._fully-expanded%29 https://docs.racket-lang.org/reference/syntax-model.html#%28... [1] http://www.schemers.org/Documents/Standards/R5RS/HTML/ http://www.schemers.org/Documents/Standards/R5RS/HTML/ [2] http://www.schemers.org/Documents/Standards/R5RS/HTML/ http://www.schemers.org/Documents/Standards/R5RS/HTML/

ScottBurson 9y ago

Right. There's still syntax, but instead of that syntax being defined directly in terms of characters and tokens, it's defined in terms of s-expressions. Thus s-expressions serve as an intermediate syntactic level between tokens and forms. (Forms are what other languages call expressions and/or statements.)

sparkie 9y ago

This is why I love John Shutt's Kernel[0]. All of these "distinct forms" are rolled into one form of "(combiner combiniend)", where combiner is runtime evaluated and can either be a regular applicative (like a lisp procedure call), or an operative (reminiscent of older lisps fexpr). Compound operatives can be created using the primitive operative $vau. Applicatives are created with the primitive wrap. (which $lambda uses in the standard library). Of the above examples you posted: lambda, let, set!, cond, and, or, when, unless, etc, may all be implemented as a library in the Kernel Report. Some of the others aren't in the report but can be trivially implemented as compound operatives. This simplifies the compiler implementation to requiring only a handful of primitives. A distinguishing feature of Kernel is the lack of quote in the report. Although it has a trivial implementation ($define! $quote ($vau (x) #ignore x)), quote is considered harmful by Shutt due to the way it interacts with operatives. Also omitted from the language is macros, as their capability can be provided by first class operatives at the library level. The big downside to Kernel is performance. Compilation is generally not possible because the behavior of operatives can depend on runtime defined environments. This would be one of the reasons macros were chosen over fexprs in the 80s. It means Kernel is not a good language for number crunching, so it'd be nice to have an FFI for the cases where we need to invoke optimised code. (Currently no FFI spec exists). [0]:http://web.cs.wpi.edu/%7Ejshutt/kernel.html http://web.cs.wpi.edu/%7Ejshutt/kernel.html

jcrites 9y ago
Thank you for sharing this. I hadn't heard of Kernel, and it seems to be highly relevant to a line of inquiry that I've been pondering for some time (you could say light research) in the area of language design. There seem to be a lot of interesting topics in the related dissertation [1]. Thanks again! [1] https://web.wpi.edu/Pubs/ETD/Available/etd-090110-124904/unrestricted/jshutt.pdf https://web.wpi.edu/Pubs/ETD/Available/etd-090110-124904/unr...

lispm 9y ago

Good explanation. Many people get that wrong and think the syntax for s-expressions is the syntax for Lisp.