3 ms·

If that's not observable to the program, so what? At a math lecture I once went to, the speaker said "sure, define it that way, but show me the theorems!" Ever

by kmill 9y ago

If that's not observable to the program, so what? At a math lecture I once went to, the speaker said "sure, define it that way, but show me the theorems!" Everyone seems to have their own idea for what homoiconicity means, but I've yet to see anyone draw a compelling inference from such a distinction. There are vague suggestions that it's important for metaprogramming, but in my previous comment I was instead suggesting that "autoiconicity" is more definable and useful. So "s-expression" means two things to Lispers: 1) the textual language that the reader converts into 2) an inductively defined tree form. The original McCarthy paper calls the second s-expressions. It'd be possible to program entirely in the tree form with the appropriate editor, but this is unpopular to do. You would miss out on numerous features of the reader: 1) comments 2) quasiquotation (for Common Lisp; in Scheme this is better represented) 3) writing numeric literals with the syntax you want 4) writing symbols in the form you want (that is, symbol or package synonyms) Anyway, the point is that Lisp source code is a sequence of characters, not a tree. Sure the parsing routine is slight, but why does everyone want to confuse the issue and say that they are literally the same thing? (Or, at the least, insist that it is special that the AST reflects the structure of the source code. Is Java's AST really so different from Java source?) Lisp doesn't need to be homoiconic to be the powerful language it is. Alan Kay is quite a thinker and visionary, though, and he is on to something, but this particular idea needs some more refinement. (Thought experiment: it's not impossible to make an interpreter for most languages that evaluates code by directly manipulating a string containing the original source file. Doesn't that make every language homoiconic? Code is data!)

lispm 9y ago
The interface for EVAL and COMPILE is code as data, not text. The macro facility operates on code as data. The pretty-printer operates on code as data. Tools like code-walkers and macroexpanders operate on code as data. Interpreter operate on code as data. This enables sophisticated tracing and stepping (stuff which I would need to instrument in compiled code), self-modifying and inspecting code, easy access to source forms in the debugger, etc. There are many ways, Lisp implementations make use of code as data, editing for some reasons does not seem to be especially attractive. Though structure editors exist, like the later one for Medley/Interlisp-D. Comments then are a part of s-expressions - they are not textual comments outside. Quasiquotation is in Common Lisp not fully a feature of the reader, since the representation of quasiquote expressions is undefined. A Common Lisp implementation is free to preserve quasiquote expressions and some actually do that. The code as string idea is possible, but the general meaning of code as data (or the less useful idea of homoiconicity) means code as data, but not as a string. Strings are the trivial uninteresting case. No, s-expression-based Lisp code is not what I would consider to be an AST.
- kmill 9y ago
  I have to say, I do not understand what you are addressing in my comment: whether you are rebutting, agreeing, expanding, informing, thinking out loud, or simply expressing what you think is special about Lisp. > The interface for EVAL and COMPILE is code as data, not text. Is that essential? or is it just convenient? I'm not aware of anything in Common Lisp (other than the spec) that requires eval to accept s-expressions rather than strings. I'm assuming objects have string forms that when read give an object that is eql. An aside: one could say eval and compile are hacks to deal with the fact that in Lisp, code is merely a kind of data. In a language like Mathematica, data is also a kind of code (there is no difference between code and data). This has logical issues, however, so I wouldn't take code=data as a design goal. > The macro facility operates on code as data. ... code as data ... code as data Sure, Lisp has a data type for code and it has an expressive language to manipulate code (what I was calling "autoiconicity"). Lisp also has staged metaprogramming that seamlessly invokes such code manipulators before evaluation/compilation. Many languages, with less expressivity, can also load, manipulate, and write code -- with the code represented as structured data of course. > (or the less useful idea of homoiconicity) means code as data, but not as a string But if you read my comments, you'd know I think "homoiconicity" does not mean "code as data," but "essentially the same representation internally and externally." It's a property of an implementation of a language and not the language itself. Alan Kay defines the word offhandedly and asserts Lisp is homoiconic in his thesis, and, except for some implementations of Lisp, it doesn't apply in general. This is the point of my thought experiment: every language has some implementation that is homoiconic. (And so what? That's also my point.) > Interpreter operate on code as data. This enables sophisticated tracing and stepping (stuff which I would need to instrument in compiled code), self-modifying and inspecting code, easy access to source forms in the debugger, etc. SBCL compiles everything to machine code, even with eval. It is not essential for an interpreter to directly operate on s-expressions, and whether one does or not is not observable. These things can all be implemented by a transformation of the code after macro expansion and before compilation. Or perhaps JIT compilation like PyPy through RPython. On the other hand, we can take most languages and create interpreters that evaluate the AST, giving us such tracing, stepping, and access to source in a debugger. (By the way, do you really mean self-modifying and self-inspecting code? Can you give a useful example of this in Common Lisp? I wasn't aware that Common Lisp requires an implementation to be able to give the definition of a function. I thought it was a property of the implementation and not the language.) > No, s-expression-based Lisp code is not what I would consider to be an AST. Great. Now would you care to define AST in a way that excludes s-expressions (and in a way that isn't ad-hoc)? It seems to me that s-expressions are a tree representing syntax abstractly; I'm not sure how it's not, or what you gain (inference-wise or practically) by excluding s-expressions. Are you drawing the distinction that it is only when an s-expression is paired with an interpretation that you get an AST? (By the way, this is a powerful idea that's used for nanopass compilers: lots of stages each with a slightly different interpretation for the constituent s-expressions.)

by kmill 9y ago

lispm 9y ago

The interface for EVAL and COMPILE is code as data, not text. The macro facility operates on code as data. The pretty-printer operates on code as data. Tools like code-walkers and macroexpanders operate on code as data. Interpreter operate on code as data. This enables sophisticated tracing and stepping (stuff which I would need to instrument in compiled code), self-modifying and inspecting code, easy access to source forms in the debugger, etc. There are many ways, Lisp implementations make use of code as data, editing for some reasons does not seem to be especially attractive. Though structure editors exist, like the later one for Medley/Interlisp-D. Comments then are a part of s-expressions - they are not textual comments outside. Quasiquotation is in Common Lisp not fully a feature of the reader, since the representation of quasiquote expressions is undefined. A Common Lisp implementation is free to preserve quasiquote expressions and some actually do that. The code as string idea is possible, but the general meaning of code as data (or the less useful idea of homoiconicity) means code as data, but not as a string. Strings are the trivial uninteresting case. No, s-expression-based Lisp code is not what I would consider to be an AST.

kmill 9y ago
I have to say, I do not understand what you are addressing in my comment: whether you are rebutting, agreeing, expanding, informing, thinking out loud, or simply expressing what you think is special about Lisp. > The interface for EVAL and COMPILE is code as data, not text. Is that essential? or is it just convenient? I'm not aware of anything in Common Lisp (other than the spec) that requires eval to accept s-expressions rather than strings. I'm assuming objects have string forms that when read give an object that is eql. An aside: one could say eval and compile are hacks to deal with the fact that in Lisp, code is merely a kind of data. In a language like Mathematica, data is also a kind of code (there is no difference between code and data). This has logical issues, however, so I wouldn't take code=data as a design goal. > The macro facility operates on code as data. ... code as data ... code as data Sure, Lisp has a data type for code and it has an expressive language to manipulate code (what I was calling "autoiconicity"). Lisp also has staged metaprogramming that seamlessly invokes such code manipulators before evaluation/compilation. Many languages, with less expressivity, can also load, manipulate, and write code -- with the code represented as structured data of course. > (or the less useful idea of homoiconicity) means code as data, but not as a string But if you read my comments, you'd know I think "homoiconicity" does not mean "code as data," but "essentially the same representation internally and externally." It's a property of an implementation of a language and not the language itself. Alan Kay defines the word offhandedly and asserts Lisp is homoiconic in his thesis, and, except for some implementations of Lisp, it doesn't apply in general. This is the point of my thought experiment: every language has some implementation that is homoiconic. (And so what? That's also my point.) > Interpreter operate on code as data. This enables sophisticated tracing and stepping (stuff which I would need to instrument in compiled code), self-modifying and inspecting code, easy access to source forms in the debugger, etc. SBCL compiles everything to machine code, even with eval. It is not essential for an interpreter to directly operate on s-expressions, and whether one does or not is not observable. These things can all be implemented by a transformation of the code after macro expansion and before compilation. Or perhaps JIT compilation like PyPy through RPython. On the other hand, we can take most languages and create interpreters that evaluate the AST, giving us such tracing, stepping, and access to source in a debugger. (By the way, do you really mean self-modifying and self-inspecting code? Can you give a useful example of this in Common Lisp? I wasn't aware that Common Lisp requires an implementation to be able to give the definition of a function. I thought it was a property of the implementation and not the language.) > No, s-expression-based Lisp code is not what I would consider to be an AST. Great. Now would you care to define AST in a way that excludes s-expressions (and in a way that isn't ad-hoc)? It seems to me that s-expressions are a tree representing syntax abstractly; I'm not sure how it's not, or what you gain (inference-wise or practically) by excluding s-expressions. Are you drawing the distinction that it is only when an s-expression is paired with an interpretation that you get an AST? (By the way, this is a powerful idea that's used for nanopass compilers: lots of stages each with a slightly different interpretation for the constituent s-expressions.)