4 ms·

As an Emacs and Lisp lover, the description of text-based tools and the power of using text to me just screams why something like Lisp is so powerful: homoiconi

by podiki 6y ago

As an Emacs and Lisp lover, the description of text-based tools and the power of using text to me just screams why something like Lisp is so powerful: homoiconicity. (And at least one big difference for elisp compared to Bash.) To quote Wikipedia [0] for anyone unfamiliar with the term, as I was until learning some Lisp: "A language is homoiconic if a program written in it can be manipulated as data using the language, and thus the program's internal representation can be inferred just by reading the program itself. This property is often summarized by saying that the language treats "code as data"." [0] https://en.wikipedia.org/wiki/Homoiconicity https://en.wikipedia.org/wiki/Homoiconicity (Edit: and if you want a good terminal emulator, you have eshell (emacs-y), ansi-term, and my current fave vterm [1]) [1] https://github.com/akermu/emacs-libvterm https://github.com/akermu/emacs-libvterm

foobarian 6y ago
I've been hearing the "code is data" mantra often, but the reason I never thought this was a desirable property is that it seems trivial to define any number of languages as their own AST. But there is a good reason that we don't program by writing out ASTs - it's relatively hard to read and manipulate. Does the Lisp implementation just happen to be particularly readable?
- tadfisher 6y ago
  In a word, yes. S-expressions are a decent representation for data (lists and hierarchical structures) and the sort of code you write in a language derived from the lambda calculus, which is function calls. This is not by accident, though, since it is very desirable to manipulate code via macros.
- someguy101010 6y ago
  The ast of a lisp program is essentially the same as the code of lisp itself. There is no intermediary step. Practically I'm sure there is, but conceptually that is they way it behaves.
- konjin 6y ago
  The AST and the parse tree are basically the same. You only need to rename the bracket nodes to their first element.
- samus 6y ago
  LISP syntax is a data format. Unlike JSON though, its only structured data type is linked lists. `(a . b)` represents a linked list node (a pair or a `cons`) with the symbol `a` as first element and `b` as the second. The syntax `(a b c)` is a shorthand for `(a . (b . (c . nil)))`, with `nil` representing the empty list. It is commonplace to define what we nowadays call embedded DSLs using macros, which are just functions that manipulate LISP data structures before the interpreter evaluates them. It is the first dynamic programming language, and arguably the most dynamic one. The only other non-esoteric language that comes close is PL/I, which also had macros and where every keyword could be redefined. But its standard is ungodly long, while Scheme dialects can be descriped in a few tens of pages. Also, it is quite easy to write an interpreter for core LISP: 100 to 200 lines, depending on your desire for readability. Edit: Yes, this means there are lots of parentheses, which can be hard to read.
  - dragonwriter 6y ago
    > It is the first dynamic programming language, and arguably the most dynamic one. The only other non-esoteric language that comes close is PL/I, which also had macros and where every keyword could be redefined REBOL/Red are on a similar level.
- karlicoss 6y ago
  It is quite a bit more readable comparing (for example) to what you'd do in Python, basically a matter of few extra quotes. Here's an example https://github.com/magnars/dash.el/blob/a17b6b5409825891423b3867cd7bea84852d6ddd/dash.el#L52-L66 https://github.com/magnars/dash.el/blob/a17b6b5409825891423b... Another benefit apart from macros is that you can replace whole subexpressions to modify the function code: https://github.com/raxod502/el-patch#el-patch https://github.com/raxod502/el-patch#el-patch Although arguably you can achieve the same with source code and other languages: https://github.com/adamchainz/patchy#patchy https://github.com/adamchainz/patchy#patchy Although (as a keen Emacs user & someone who often hacks into existing librairies) user I'm not really convinced either that homoiconicity plays that huge of a role in Emacs malleability in particular -- to me it's more about hooks/advice system and being able to dynamically override things to tweak into the way I want them to behave -- and I don't see a good reason why this couldn't be possible in other languages.
- podiki 6y ago
  In addition to the other replies, I think it always takes time to learn a new language and its syntax, preferred formatting, etc. So I don't know what is more "natural," as that would depend on your background, but once I learned s-expressions it is my favorite way to read code. (Combine that with Emacs to handle all formatting, parens matching, and so on, and you can just think about code rather than decipher it too.)
- marcosdumay 6y ago
  It's kinda of a messy concept. Source code is never like an AST, because an AST is a tree on a computer's memory, with pointers and everything, while source code is text. I see no reason why a language can not have a more complex source -> AST transformation, while still keeping code as readable data that can be manipulated by macros. So, if you want homoiconicity with a slightly more complex syntax, I don't think it won't bring any problem. But if you keep making your language's syntax more and more complex, you will make macro creation harder and harder, so there is probably a practical limitation somewhere. Operators and statement sequences are very likely not complex enough to be a problem, but I don't think anybody will ever make native C++ macros work.

by podiki 6y ago

foobarian 6y ago

I've been hearing the "code is data" mantra often, but the reason I never thought this was a desirable property is that it seems trivial to define any number of languages as their own AST. But there is a good reason that we don't program by writing out ASTs - it's relatively hard to read and manipulate. Does the Lisp implementation just happen to be particularly readable?

tadfisher 6y ago
In a word, yes. S-expressions are a decent representation for data (lists and hierarchical structures) and the sort of code you write in a language derived from the lambda calculus, which is function calls. This is not by accident, though, since it is very desirable to manipulate code via macros.
someguy101010 6y ago
The ast of a lisp program is essentially the same as the code of lisp itself. There is no intermediary step. Practically I'm sure there is, but conceptually that is they way it behaves.
konjin 6y ago
The AST and the parse tree are basically the same. You only need to rename the bracket nodes to their first element.
samus 6y ago
LISP syntax is a data format. Unlike JSON though, its only structured data type is linked lists. `(a . b)` represents a linked list node (a pair or a `cons`) with the symbol `a` as first element and `b` as the second. The syntax `(a b c)` is a shorthand for `(a . (b . (c . nil)))`, with `nil` representing the empty list. It is commonplace to define what we nowadays call embedded DSLs using macros, which are just functions that manipulate LISP data structures before the interpreter evaluates them. It is the first dynamic programming language, and arguably the most dynamic one. The only other non-esoteric language that comes close is PL/I, which also had macros and where every keyword could be redefined. But its standard is ungodly long, while Scheme dialects can be descriped in a few tens of pages. Also, it is quite easy to write an interpreter for core LISP: 100 to 200 lines, depending on your desire for readability. Edit: Yes, this means there are lots of parentheses, which can be hard to read.
- dragonwriter 6y ago
  > It is the first dynamic programming language, and arguably the most dynamic one. The only other non-esoteric language that comes close is PL/I, which also had macros and where every keyword could be redefined REBOL/Red are on a similar level.
karlicoss 6y ago
It is quite a bit more readable comparing (for example) to what you'd do in Python, basically a matter of few extra quotes. Here's an example https://github.com/magnars/dash.el/blob/a17b6b5409825891423b3867cd7bea84852d6ddd/dash.el#L52-L66 https://github.com/magnars/dash.el/blob/a17b6b5409825891423b... Another benefit apart from macros is that you can replace whole subexpressions to modify the function code: https://github.com/raxod502/el-patch#el-patch https://github.com/raxod502/el-patch#el-patch Although arguably you can achieve the same with source code and other languages: https://github.com/adamchainz/patchy#patchy https://github.com/adamchainz/patchy#patchy Although (as a keen Emacs user & someone who often hacks into existing librairies) user I'm not really convinced either that homoiconicity plays that huge of a role in Emacs malleability in particular -- to me it's more about hooks/advice system and being able to dynamically override things to tweak into the way I want them to behave -- and I don't see a good reason why this couldn't be possible in other languages.
podiki 6y ago
In addition to the other replies, I think it always takes time to learn a new language and its syntax, preferred formatting, etc. So I don't know what is more "natural," as that would depend on your background, but once I learned s-expressions it is my favorite way to read code. (Combine that with Emacs to handle all formatting, parens matching, and so on, and you can just think about code rather than decipher it too.)
marcosdumay 6y ago
It's kinda of a messy concept. Source code is never like an AST, because an AST is a tree on a computer's memory, with pointers and everything, while source code is text. I see no reason why a language can not have a more complex source -> AST transformation, while still keeping code as readable data that can be manipulated by macros. So, if you want homoiconicity with a slightly more complex syntax, I don't think it won't bring any problem. But if you keep making your language's syntax more and more complex, you will make macro creation harder and harder, so there is probably a practical limitation somewhere. Operators and statement sequences are very likely not complex enough to be a problem, but I don't think anybody will ever make native C++ macros work.