4 ms·

> There's something magically expressive about S-expressions and processing them. A downside I see with S-expressions – you often end up with something like (F

> There's something magically expressive about S-expressions and processing them. A downside I see with S-expressions – you often end up with something like (FOO BAR BAZ), where FOO identifies the type of thing, and BAR and BAZ are parameters/slots/etc – but you have to remember the (sometimes quite arbitrary) order that BAR and BAZ go in. Some Lisp family languages (such as Common Lisp, Clojure, Scheme with SRFI 88/89) support keywords (aka named parameters), so you can do something like (FOO :bar BAR :baz BAZ), which is considered equivalent to (FOO :baz BAZ :bar BAR). However, this is somewhat of a late development in the Lisp tradition, the status of its adoption is mixed, and (arguably) it is moving away (even if only by a little bit) from pure S-expressions. When people say "S-expressions", they normally aren't thinking of CLOS. I think JSON's model, in which lists and objects are independent first-class entities, with distinct syntax, has something to say for it. The downside of JSON objects, is their keys are strings, not symbols, and they don't have any kind of "class slot" (some people make up conventions here, for example a reserved property name such as "$class", but those conventions are not uniform or standardised). Oh, and we don't need all those commas. I think, if I was designing a LISP-ish language, I'd have a first-class syntax for objects which used different opening and closing characters – maybe (FOO bar baz) for a list, but [FOO :BAR bar :BAZ baz] for an object.

srcreigh 4y ago
This is equivalent to an association list (aka alist) which is a technique which goes back to JAM. (define xs `(foo (bar ,1) (baz ,0))) (second (assoc 'bar (rest xs))) ;; 1 (define (prop xs k) (second (assoc k (rest xs)))) (prop xs 'bar) ;; 1
- skissane 4y ago
  Association lists have some obvious downsides – if they didn't, Common Lisp-style keywords would have never been invented. One is that keywords are simpler syntax – `(foo :bar 1 :baz 0) is easier on the eyes than its association list equivalent. Another is that their native implementation (linear search) can be a performance drain if you use them too heavily in an application (especially if you have too many keys). It is a data structure which optimises for insertion over retrieval, but most applications retrieve much more than they insert. The ability to "shadow" (have multiple bindings for the same key, with the first being used and the subsequent being ignored, but potentially becoming active again when the earlier binding is removed) is useful in some cases, but more often than not an unnecessary feature, more likely to induce bugs than be genuinely beneficial – "shadowing" by accident rather than intention. A sufficiently smart runtime could optimise around some of these issues (this looks an association list, I'll use some optimised data structure to store it, but fall back on a more traditional cons-cell based storage if you try to do with it things the optimised data structure doesn't support.) But, having an actually separate data type enables equivalent performance optimisations, without anywhere near as much "smartness" in the runtime (which can be difficult to understand and maintain.) Plus, that kind of "smartness" can lead to difficult to understand performance regressions (why does adding this new code suddenly make completely unrelated parts of the application a lot slower?)
  - samatman 4y ago
    Lua's tables with metatables is a particularly eloquent solution to this, because as you say, it's always clear when a pair is being shadowed vs. replaced.
adamddev1 4y ago
Yes definitely a valid point. For my case I used TypeScript and made a typed array of objects (with recursive definitions to arrays of the same) that cold be arranged and processed like an S-Expression. This typing and having named elements in the objects really hit the sweet spot for me while still being able to process the list/array in a functional/recursive way.
kazinator 4y ago
Keyword parameters may be a late development, but they are based on property lists. Property lists are so ancient, that Lisp 1 and 1.5 used them for storing bindings. For instance, a symbol having a SUBR (machine language routine) or EXPR (interpreted routine) binding was stored in that symbol's property list. Property lists and keyword arguments aren't tied to CLOS, though of course CLOS uses them for the flexibla constructor arguments passed to make-instance.
eadmund 4y ago
A flaw with the JSON model is that there is no inherent sorting to the order of pairs. This has problems for canonicalisation, but also for shadowing. A Lisp can use an alist to represent a set of key-value pairs, and is easy enough to shadow values there: (defvar original '((a . 1) (b . 2))) → ((A . 1) (B . 2)) (assoc 'b foo) → (B . 2) (defvar bar `((b . 3) ,@foo)) → ((B . 3) (A . 1) (B . 2)) (assoc 'b bar) → (B . 3) This makes it easy to implement things like inheritance or trees of lexical environments. I sometimes wish that Lisp had better ergonomics for hash tables, but normally I am pretty happy that folks don't reach for them first. Frankly, I think positionality is normally good. Can you image how terrible it would be to program a language in which addition is represented as: add(operand: 1, operand: 2)
- renox 4y ago
  Positionality is good only when arguments are interchangeable/commutative like addition or when the types are different which ensure that any mistakes in the order will be detected by the compiler. In all the other case positionality is bad: see memset for example.
rileyphone 4y ago
What you're describing with parameter ordering is Lisp's Mysterious Tuple Problem [0]. It's also part of why Lisp works so well for one or only a few developers, but breaks down for large teams. Javascript (and Self before it) has really started a structural revolution with the JSON object as a syntactic form. I'm currently working on a language very similar to your last paragraph, where all objects do have such a field indicating their class, so an object in the textual representation might look like: { Point x 2 y 3 } Instead of n-ary tuple operation like Lisp, my design limits all functional application/message sends to a form of a receiver object, message, optional it object, and additional map of args in the preceding syntactic way (still not sure if they would have a class or be classless). Multiple dispatch would be limited to just the receiver and the it object. You can see in Smalltalk where this syntax would be helpful in the awkward selector syntax it adopts. bool ifTrue: [block] ifFalse: [block] Looks better as bool if { If $then block $else block } Where here the $ in slot names means the corresponding value isn't evaluated, a reference to the late John Shutt's Kernel [1] that explored replacing the lambda calculus at the base of Lisp with a lazy vau calculus in order to restore the historically maligned fexpr [2]. Originally they were removed from the language because they made compiling difficult at a time when performance really mattered (1980), and replaced with special forms that weren't first class subjects of the language (and neither really were the syntactic macros that accompanied them). Going back to the original topic, Alan Kay listed Lisp as one of the primary inspirations for Smalltalk, and especially the fexpr, which he saw as underappreciated: > There were not just EXPRs (which evaluated their arguments), but FEXPRs (which did not). My next question was, why on earth call it a functional language? Why not just base everything on FEXPRs and force evaluation on the receiving side when needed? I could never get a good answer, but the question was very helpful when it came time to invent Smalltalk, because this started a line of thought that said "take the hardest and most profound thing you need to do, make it great, and then build every easier thing out of it". That was the promise of LISP and the lure of lambda—needed was a better "hardest and most profound" thing. Objects should be it. [3] I'm still in search of the "hardest and most profound thing" - here's to hoping it's real. [0] https://www.codeproject.com/Articles/1186940/Lisps-Mysterious-Tuple-Problem https://www.codeproject.com/Articles/1186940/Lisps-Mysteriou... [1] https://web.cs.wpi.edu/%7Ejshutt/kernel.html https://web.cs.wpi.edu/%7Ejshutt/kernel.html [2] https://fexpr.blogspot.com/2011/04/fexpr.html https://fexpr.blogspot.com/2011/04/fexpr.html [3] http://worrydream.com/EarlyHistoryOfSmalltalk/ http://worrydream.com/EarlyHistoryOfSmalltalk/
nlitened 4y ago
> I think JSON's model, in which lists and objects are independent first-class entities, with distinct syntax, has something to say for it. > Oh, and we don't need all those commas. I think, if I was designing a LISP-ish language, I'd have a first-class syntax for objects which used different opening and closing characters – maybe (FOO bar baz) for a list, but [FOO :BAR bar :BAZ baz] for an object. I think Clojure has what you're looking for, and the syntax is great, much more light-weight than JSON (the syntax even has its own name: EDN). 'Objects' are first-class hash maps, your example could look like this `{:class FOO :bar BAR :baz BAZ}`.
dreamcompiler 4y ago
Another flaw in JSON is that there's no built-in way to connote referential integrity. Which is another way of saying that JSON can represent trees but not arbitrary graphs. Here's a circular list for example in Common Lisp: #1='(foo bar baz #1#) [Printing that is a good way to overflow your stack] More usefully, here is an employee who is an acting supervisor of himself: '(:employee #1="Frank Smith" :supervisor #1#) There are ways around this problem in JSON but it requires a bespoke parser. In Common Lisp it's built in to the reader.
- skissane 4y ago
  Self-reference causes a lot of issues. Sometimes I wonder if it should just be prohibited, forcing everything to be a directed acyclic graph. If a language has only immutable data structures, it is easy to make it so that self-reference becomes impossible, just by making sure your primitives for constructing values don't permit cycles to be constructed. Even if you allow mutable data structures, it is still possible. For mutable compound heap objects (i.e. heap objects which possibly container pointers to other heap objects), they can have a back-pointer pointing to their parent. If you think about it, you can flesh out some rules about how to manipulate these objects which will make cyclic references impossible. When the rules are violated, rather than failing the operation, you could just add a pointer to a deep copy instead. (Not completely deep, you can still share rather than copy non-compound heap objects, such as strings–sharing non-compound objects can't produce cycles.) Banning direct self-reference doesn't mean nothing can ever refer to itself, just not directly. So, you couldn't have #1='(employee :name "Frank Smith" :supervisor #1#). But you could have something like '(employee :id 1 :name "Frank Smith" :supervisor (ref :to employee :id 1)). At a physical level there is no self-reference, but at a logical level there still is. Similarly, one would want to allow cyclic references via a symbol, so a symbol can occur in its own binding. So not completely banning self-reference, just limiting it, in the hope that limiting it will limit the issues it causes. I think, using the strategies I've outlined above, reference counting will work as a garbage collection strategy, without the possibility of memory leaks from cyclic references. (Allowing a symbol's binding to refer to itself will not leak memory, since a bound symbol should still exist even if nobody refers to it; to deallocate it, you have to explicitly unbind it, which will remove the binding, breaking the self-reference and allowing the reference count to fall to zero.)
didibus 4y ago
That's exactly what Clojure did, here's the quote from clojure.org: > Extends the code-as-data paradigm to maps and vectors Basically think of it as a better JSON with non of the issues you brought up. See: https://github.com/edn-format/edn https://github.com/edn-format/edn And example: > #myapp/Person {:first "Fred" :last "Mertz"}
- JonChesterfield 4y ago
  The keyword vs symbol distinction still seems weird to me. :first vs 'first. {} delimited sequence of even length is a nice complement to () delimited sequence of arbitrary length though.
  - nlitened 4y ago
    I think of it this way: keywords are like symbols, but they survive serialization/deserialization and macroexpansion intact, so you don't need to be extra careful with quoting to prevent automatic evaluation of the symbol to some other value. And since keywords are used in code very often as keys and enums, it reduces cognitive load immensely.
mpweiher 4y ago
> and they don't have any kind of "class slot" Yeah, that really sucks. Objective-S's object literals do. Otherwise they really wouldn't be object literals, would they? It was really an accident, because Smalltalk's literal arrays use #(), and since I wanted to have dictionary literals use the basic same look, I chose #{}. And also I use {} for blocks and block structure, rather than []. Well, it turns out you can tuck a name between the # and the {}. Yay! Classes. And thus objects! EDIT: Just saw that edn uses a very similar mechanism. Neat!

srcreigh 4y ago

This is equivalent to an association list (aka alist) which is a technique which goes back to JAM. (define xs `(foo (bar ,1) (baz ,0))) (second (assoc 'bar (rest xs))) ;; 1 (define (prop xs k) (second (assoc k (rest xs)))) (prop xs 'bar) ;; 1

skissane 4y ago
Association lists have some obvious downsides – if they didn't, Common Lisp-style keywords would have never been invented. One is that keywords are simpler syntax – `(foo :bar 1 :baz 0) is easier on the eyes than its association list equivalent. Another is that their native implementation (linear search) can be a performance drain if you use them too heavily in an application (especially if you have too many keys). It is a data structure which optimises for insertion over retrieval, but most applications retrieve much more than they insert. The ability to "shadow" (have multiple bindings for the same key, with the first being used and the subsequent being ignored, but potentially becoming active again when the earlier binding is removed) is useful in some cases, but more often than not an unnecessary feature, more likely to induce bugs than be genuinely beneficial – "shadowing" by accident rather than intention. A sufficiently smart runtime could optimise around some of these issues (this looks an association list, I'll use some optimised data structure to store it, but fall back on a more traditional cons-cell based storage if you try to do with it things the optimised data structure doesn't support.) But, having an actually separate data type enables equivalent performance optimisations, without anywhere near as much "smartness" in the runtime (which can be difficult to understand and maintain.) Plus, that kind of "smartness" can lead to difficult to understand performance regressions (why does adding this new code suddenly make completely unrelated parts of the application a lot slower?)
- samatman 4y ago
  Lua's tables with metatables is a particularly eloquent solution to this, because as you say, it's always clear when a pair is being shadowed vs. replaced.

adamddev1 4y ago

Yes definitely a valid point. For my case I used TypeScript and made a typed array of objects (with recursive definitions to arrays of the same) that cold be arranged and processed like an S-Expression. This typing and having named elements in the objects really hit the sweet spot for me while still being able to process the list/array in a functional/recursive way.

kazinator 4y ago

Keyword parameters may be a late development, but they are based on property lists. Property lists are so ancient, that Lisp 1 and 1.5 used them for storing bindings. For instance, a symbol having a SUBR (machine language routine) or EXPR (interpreted routine) binding was stored in that symbol's property list. Property lists and keyword arguments aren't tied to CLOS, though of course CLOS uses them for the flexibla constructor arguments passed to make-instance.

eadmund 4y ago

A flaw with the JSON model is that there is no inherent sorting to the order of pairs. This has problems for canonicalisation, but also for shadowing. A Lisp can use an alist to represent a set of key-value pairs, and is easy enough to shadow values there: (defvar original '((a . 1) (b . 2))) → ((A . 1) (B . 2)) (assoc 'b foo) → (B . 2) (defvar bar `((b . 3) ,@foo)) → ((B . 3) (A . 1) (B . 2)) (assoc 'b bar) → (B . 3) This makes it easy to implement things like inheritance or trees of lexical environments. I sometimes wish that Lisp had better ergonomics for hash tables, but normally I am pretty happy that folks don't reach for them first. Frankly, I think positionality is normally good. Can you image how terrible it would be to program a language in which addition is represented as: add(operand: 1, operand: 2)

renox 4y ago
Positionality is good only when arguments are interchangeable/commutative like addition or when the types are different which ensure that any mistakes in the order will be detected by the compiler. In all the other case positionality is bad: see memset for example.

rileyphone 4y ago

What you're describing with parameter ordering is Lisp's Mysterious Tuple Problem [0]. It's also part of why Lisp works so well for one or only a few developers, but breaks down for large teams. Javascript (and Self before it) has really started a structural revolution with the JSON object as a syntactic form. I'm currently working on a language very similar to your last paragraph, where all objects do have such a field indicating their class, so an object in the textual representation might look like: { Point x 2 y 3 } Instead of n-ary tuple operation like Lisp, my design limits all functional application/message sends to a form of a receiver object, message, optional it object, and additional map of args in the preceding syntactic way (still not sure if they would have a class or be classless). Multiple dispatch would be limited to just the receiver and the it object. You can see in Smalltalk where this syntax would be helpful in the awkward selector syntax it adopts. bool ifTrue: [block] ifFalse: [block] Looks better as bool if { If $then block $else block } Where here the $ in slot names means the corresponding value isn't evaluated, a reference to the late John Shutt's Kernel [1] that explored replacing the lambda calculus at the base of Lisp with a lazy vau calculus in order to restore the historically maligned fexpr [2]. Originally they were removed from the language because they made compiling difficult at a time when performance really mattered (1980), and replaced with special forms that weren't first class subjects of the language (and neither really were the syntactic macros that accompanied them). Going back to the original topic, Alan Kay listed Lisp as one of the primary inspirations for Smalltalk, and especially the fexpr, which he saw as underappreciated: > There were not just EXPRs (which evaluated their arguments), but FEXPRs (which did not). My next question was, why on earth call it a functional language? Why not just base everything on FEXPRs and force evaluation on the receiving side when needed? I could never get a good answer, but the question was very helpful when it came time to invent Smalltalk, because this started a line of thought that said "take the hardest and most profound thing you need to do, make it great, and then build every easier thing out of it". That was the promise of LISP and the lure of lambda—needed was a better "hardest and most profound" thing. Objects should be it. [3] I'm still in search of the "hardest and most profound thing" - here's to hoping it's real. [0] https://www.codeproject.com/Articles/1186940/Lisps-Mysterious-Tuple-Problem https://www.codeproject.com/Articles/1186940/Lisps-Mysteriou... [1] https://web.cs.wpi.edu/%7Ejshutt/kernel.html https://web.cs.wpi.edu/%7Ejshutt/kernel.html [2] https://fexpr.blogspot.com/2011/04/fexpr.html https://fexpr.blogspot.com/2011/04/fexpr.html [3] http://worrydream.com/EarlyHistoryOfSmalltalk/ http://worrydream.com/EarlyHistoryOfSmalltalk/

nlitened 4y ago

> I think JSON's model, in which lists and objects are independent first-class entities, with distinct syntax, has something to say for it. > Oh, and we don't need all those commas. I think, if I was designing a LISP-ish language, I'd have a first-class syntax for objects which used different opening and closing characters – maybe (FOO bar baz) for a list, but [FOO :BAR bar :BAZ baz] for an object. I think Clojure has what you're looking for, and the syntax is great, much more light-weight than JSON (the syntax even has its own name: EDN). 'Objects' are first-class hash maps, your example could look like this `{:class FOO :bar BAR :baz BAZ}`.

dreamcompiler 4y ago

Another flaw in JSON is that there's no built-in way to connote referential integrity. Which is another way of saying that JSON can represent trees but not arbitrary graphs. Here's a circular list for example in Common Lisp: #1='(foo bar baz #1#) [Printing that is a good way to overflow your stack] More usefully, here is an employee who is an acting supervisor of himself: '(:employee #1="Frank Smith" :supervisor #1#) There are ways around this problem in JSON but it requires a bespoke parser. In Common Lisp it's built in to the reader.

skissane 4y ago
Self-reference causes a lot of issues. Sometimes I wonder if it should just be prohibited, forcing everything to be a directed acyclic graph. If a language has only immutable data structures, it is easy to make it so that self-reference becomes impossible, just by making sure your primitives for constructing values don't permit cycles to be constructed. Even if you allow mutable data structures, it is still possible. For mutable compound heap objects (i.e. heap objects which possibly container pointers to other heap objects), they can have a back-pointer pointing to their parent. If you think about it, you can flesh out some rules about how to manipulate these objects which will make cyclic references impossible. When the rules are violated, rather than failing the operation, you could just add a pointer to a deep copy instead. (Not completely deep, you can still share rather than copy non-compound heap objects, such as strings–sharing non-compound objects can't produce cycles.) Banning direct self-reference doesn't mean nothing can ever refer to itself, just not directly. So, you couldn't have #1='(employee :name "Frank Smith" :supervisor #1#). But you could have something like '(employee :id 1 :name "Frank Smith" :supervisor (ref :to employee :id 1)). At a physical level there is no self-reference, but at a logical level there still is. Similarly, one would want to allow cyclic references via a symbol, so a symbol can occur in its own binding. So not completely banning self-reference, just limiting it, in the hope that limiting it will limit the issues it causes. I think, using the strategies I've outlined above, reference counting will work as a garbage collection strategy, without the possibility of memory leaks from cyclic references. (Allowing a symbol's binding to refer to itself will not leak memory, since a bound symbol should still exist even if nobody refers to it; to deallocate it, you have to explicitly unbind it, which will remove the binding, breaking the self-reference and allowing the reference count to fall to zero.)

didibus 4y ago

That's exactly what Clojure did, here's the quote from clojure.org: > Extends the code-as-data paradigm to maps and vectors Basically think of it as a better JSON with non of the issues you brought up. See: https://github.com/edn-format/edn https://github.com/edn-format/edn And example: > #myapp/Person {:first "Fred" :last "Mertz"}

JonChesterfield 4y ago
The keyword vs symbol distinction still seems weird to me. :first vs 'first. {} delimited sequence of even length is a nice complement to () delimited sequence of arbitrary length though.
- nlitened 4y ago
  I think of it this way: keywords are like symbols, but they survive serialization/deserialization and macroexpansion intact, so you don't need to be extra careful with quoting to prevent automatic evaluation of the symbol to some other value. And since keywords are used in code very often as keys and enums, it reduces cognitive load immensely.

mpweiher 4y ago

> and they don't have any kind of "class slot" Yeah, that really sucks. Objective-S's object literals do. Otherwise they really wouldn't be object literals, would they? It was really an accident, because Smalltalk's literal arrays use #(), and since I wanted to have dictionary literals use the basic same look, I chose #{}. And also I use {} for blocks and block structure, rather than []. Well, it turns out you can tuck a name between the # and the {}. Yay! Classes. And thus objects! EDIT: Just saw that edn uses a very similar mechanism. Neat!