6 ms·

I don’t have much cause to write Lua, but I can very much support lispy syntax. When we all started coding, we probably used a text editor and thought in terms

I don’t have much cause to write Lua, but I can very much support lispy syntax. When we all started coding, we probably used a text editor and thought in terms of editing characters: backspacing, typing, maybe mousing around for bigger changes. Then at some point we graduated to a programmers editor where we got comfortable cutting, pulling, yanking, killing larger units of text: words, lines, paragraphs. With tools like paredit, you can take another evolutionary step to edit the structures of your code. I find this very productive. It feels a little like those code block programming languages for kids. If for no other reason than to enable slurping and barfing code structures in my text editor, this is a win. May all languages someday have a lisp syntax.

drb91 9y ago
I'm with you almost all the way.... however, I think what we should learn here is that every language deserves tools as powerful as paredit--that is, ast-aware editing--not that all languages should convert their ast to s-expressions. Imagine not being able to save syntactically incorrect code, or being able to have semantic diffs between domain objects (e.g. functions, classes, strings, types, apis) rather than thinking in terms of characters and lines, or being able to query a code base, like "find me all places where I assign to a variable named 'status' with type under 'error' and list me the names of the functions whose frame the variable belongs to". Paredit is simply where text and ast manipulation converge to the closest point.
- thesmallestcat 9y ago
  As a fallback. I like LISPs as much as the next guy/gal, but editing the AST by default is too low-level. The importance of paredit is a language smell in my opinion, not something to aspire to.
  - jorams 9y ago
    Can you explain in which direction something better should go? The AST is very much at a higher level than how the vast majority of programming is done, which is by manipulating the text that later gets parsed into an AST. I'm having trouble envisioning what editing a higher level than the AST would look like, without jumping straight to visual programming, which I find very inefficient in every incarnation I've seen. (And we could debate how far above an AST such systems actually are.)
    - briankelly 9y ago
      That seems backwards to me - AST-aware editing is basically embedding more of the compiler into the editor. It seems more valuable to me to be writing in a language that can express higher level concepts with lesser boilerplate, which would eliminate the need (and make far more difficult) for AST aware editing.
      - drb91 9y ago
        Why not both?
      - cptn_brittish 9y ago
        In regards to lisp a large amount of lisp editors are written in lisp so they do already do exactly that'
        kazinator 9y ago
        You might be surprised how little a Lisp dialect brings to the table specifically for developing the text editor features geared toward editing specifically Lisp. The data structure representing Lisp data (i.e. code) is stripped of relevant text editing attributes, like how it is divided into lines and using what indentation, and, oh those semicolon-delimited comments. Text editors also have to let the user deal with bad syntax. So it's not just a matter of, "oh, we're written in Lisp, so just hand the buffer to the Lisp reader, do something and spit it out again".
      - flavio81 9y ago
        >that can express higher level concepts with lesser boilerplate, The whole premise around Lisp is precisely to express higher level concepts with lesser boilerplate. This is achieved through a syntax that is really close to the AST, and macros.
      - imtringued 9y ago
        I'm not sure what that is even supposed to mean. Having high quality tooling is completely independent from having a programming language with less boilerplate code. You can have both at the same time. So why are you advocating against it? Take Java for instance. In Eclipse the a lot of AST-aware editing features are hidden in the "quickfixes" which you can trigger with CTRL+1. For example you want to use an ArrayList but you haven't imported it yet. Use quickfixes: Bang the import was created with just two keypresses. In a raw text editor you have to remember the full package string or google it but when you want to look up what a "XYZ" it's rather easy to find it. The java import system is verbose but it makes it obvious what you're importing. one class = one file Want to know what X is? Just open X.java. Compare it to something like C/C++ where you include a header that contains multiple definitions. I honestly cannot use C++ without an IDE because of this. At the bare minimum my editor needs to be able to jump from a function call to the function definition or variable declaration to the struct definition. In this case a feature that reduces boilerplate actually made the language harder to use without AST aware navigation. Then there are quality of life things like extracting parts of a big function into seperate smaller functions. Just select the code you want and the IDE is going to create a function definition based on it and replace your selection with a call to that function. I don't see how you could avoid this by making a more expressive language. Does your fancy programming language automatically organise your code into functions or data types and all you need to do is crank out code without a care? No, you still to need think about that yourself, but the IDE can automatically move the code around for you. Also let's talk about my favourite feature that I would never use without the assistance of an IDE: typeinference. value blah = foo.bar([1,2,3]) What the hell is a blah? With an IDE I could just hover over blah and instantly know what the function returns. Without that feature I have to go to the definition of bar which is still easier with an IDE. After all if foo is also defined via type inference I have to lookup several function definitions and that's going to take some time. On top of that a tool like grep is imprecise and will give me hundreds of call sites but only one definition. You can't fix everything by making a programming language more expressive and in my experience the dumber a programming language is, the easier it is to use with a dumb text editor.
  - flavio81 9y ago
    >The importance of paredit is a language smell in my opinion, not something to aspire to. Paredit is just a nice-to-have. Many lispers don't use it.
- ZenoArrow 9y ago
  > "Paredit is simply where text and ast manipulation converge to the closest point." What about Parinfer? I don't use it but seems like it's even lower friction than Paredit. https://shaunlebron.github.io/parinfer/ https://shaunlebron.github.io/parinfer/
- wruza 9y ago
  >not being able to save syntactically incorrect code Going deeper, it would be nice to not being able to enter syntactically incorrect code.
  - kazinator 9y ago
    That's possible, but it requires the machine to enter some things for you. Since ( is syntactically incorrect, it has to be entered together with ). We do see small examples of this in GUI's with strict validation on inputs that is applied for each keystroke. One useful compromise is syntax highlighting that doesn't let you enter syntactically incorrect code without flagging the problem in a loud red color.
    - wruza 9y ago
      Red hints annoy me honestly, when I meet these in real editors. I was thinking of something like grammar walker that completes input to shortest finished constructs. While not semantically correct, but syntactically like yes. pr pr; printf( printf(<?expr?>); Or to not edit code as text at all, but as a tree. Like apple plist editor or windows registry editor, but with convenient shortcuts to add constructs. Xcode can sort of simulate this with templates and placeholders: if␣ if (<?condition?>) { <?body?> }
- pflanze 9y ago
  S-expressions are not directly an AST: they are a form easily matched to an AST (which could be the Lisp at hands, or some language for which there is some translator/parser/interpreter around, or some language for there is no code in the system to (fully) handle it). This makes the use of s-expressions simple, they can be handled by the development tools without them always having to know what language they represent. If your development tool knows how to handle a particular language, then it can handle the AST for that language, but not any other. You could instruct the tool about the other languages, but that requires ways to let the tool know; in the end this is similar to the static versus dynamic typing discussion: static typing doesn't allow any potentially unsafe data, and a tool working with an AST directly wouldn't allow any potentially invalid programs, which in principle (or theory) is nice but difficult to achieve practically, it presents overheads for the tool implementer and for the user probably alike. And it probably takes away a lot of interactivity: just like reloading modules in a statically typed language (like ML or Haskell) is not as direct as being able to modify values and functions in a running system and having them be called directly. So I think S-expressions are a very pragmatic concept that is going to be difficult to replace without the replacement costing more than it benefits.
  - lispm 9y ago
    That's absolutely right. S-expressions are a syntax for data. It's more related to what a tokenizer does, but with nested lists of tokens - not a flat token stream. It's only loosely related to an AST, since the s-expression represents some form of nesting - but not explicitly what kinds. S-expressions know nothing about the syntax of the program language Lisp (what is the syntax for the LET operator?). If we have a Lisp expression, an AST for it would need to represent specific language constructs - like knowing that LET is a built-in operator and has parts like a list of bindings (where each binding is a symbol or a list of symbol + value form, optionally declarations (with their specific syntax) and a list of body forms. The s-expression does not represent that: in (let ((let 'let)) (list let 'let)) the s-expression does not represent that LET is a built-in operator, a variable name, a symbol, a variable name and a symbol. All we see is five LET symbols and no additional information. Writing Lisp code using s-expressions allows us to use data manipulation functions both on text and on internal Lisp data: reverse, transpose, move, move in, move out, ... We can then also enrich the editor (or other tools) to understand the syntax of Lisp. Example: know what the structure of a LET form is and then allow for example to manipulate the binding list, knowing where it is and what a variable and what a value form is.
- conistonwater 9y ago
  One thing to consider is that s-expressions make implementing paredit easy. It takes an enormous amount of effort (e.g., libclang in C++) to get close to that without s-expressions. It's as if you think of programming languages as "fixed", and we only get to choose which kinds of tools we develop, but another way of doing it would be to design programming languages in ways that specifically make developing such tools easier. Then s-expressions could well be the go-to solution for that.
agumonkey 9y ago
now we need fnl-mode to enjoy lisp-lua repl and paredit in emacs :)

drb91 9y ago

I'm with you almost all the way.... however, I think what we should learn here is that every language deserves tools as powerful as paredit--that is, ast-aware editing--not that all languages should convert their ast to s-expressions. Imagine not being able to save syntactically incorrect code, or being able to have semantic diffs between domain objects (e.g. functions, classes, strings, types, apis) rather than thinking in terms of characters and lines, or being able to query a code base, like "find me all places where I assign to a variable named 'status' with type under 'error' and list me the names of the functions whose frame the variable belongs to". Paredit is simply where text and ast manipulation converge to the closest point.

thesmallestcat 9y ago
As a fallback. I like LISPs as much as the next guy/gal, but editing the AST by default is too low-level. The importance of paredit is a language smell in my opinion, not something to aspire to.
- jorams 9y ago
  Can you explain in which direction something better should go? The AST is very much at a higher level than how the vast majority of programming is done, which is by manipulating the text that later gets parsed into an AST. I'm having trouble envisioning what editing a higher level than the AST would look like, without jumping straight to visual programming, which I find very inefficient in every incarnation I've seen. (And we could debate how far above an AST such systems actually are.)
  - briankelly 9y ago
    That seems backwards to me - AST-aware editing is basically embedding more of the compiler into the editor. It seems more valuable to me to be writing in a language that can express higher level concepts with lesser boilerplate, which would eliminate the need (and make far more difficult) for AST aware editing.
    - drb91 9y ago
      Why not both?
    - cptn_brittish 9y ago
      In regards to lisp a large amount of lisp editors are written in lisp so they do already do exactly that'
      - kazinator 9y ago
        You might be surprised how little a Lisp dialect brings to the table specifically for developing the text editor features geared toward editing specifically Lisp. The data structure representing Lisp data (i.e. code) is stripped of relevant text editing attributes, like how it is divided into lines and using what indentation, and, oh those semicolon-delimited comments. Text editors also have to let the user deal with bad syntax. So it's not just a matter of, "oh, we're written in Lisp, so just hand the buffer to the Lisp reader, do something and spit it out again".
    - flavio81 9y ago
      >that can express higher level concepts with lesser boilerplate, The whole premise around Lisp is precisely to express higher level concepts with lesser boilerplate. This is achieved through a syntax that is really close to the AST, and macros.
    - imtringued 9y ago
      I'm not sure what that is even supposed to mean. Having high quality tooling is completely independent from having a programming language with less boilerplate code. You can have both at the same time. So why are you advocating against it? Take Java for instance. In Eclipse the a lot of AST-aware editing features are hidden in the "quickfixes" which you can trigger with CTRL+1. For example you want to use an ArrayList but you haven't imported it yet. Use quickfixes: Bang the import was created with just two keypresses. In a raw text editor you have to remember the full package string or google it but when you want to look up what a "XYZ" it's rather easy to find it. The java import system is verbose but it makes it obvious what you're importing. one class = one file Want to know what X is? Just open X.java. Compare it to something like C/C++ where you include a header that contains multiple definitions. I honestly cannot use C++ without an IDE because of this. At the bare minimum my editor needs to be able to jump from a function call to the function definition or variable declaration to the struct definition. In this case a feature that reduces boilerplate actually made the language harder to use without AST aware navigation. Then there are quality of life things like extracting parts of a big function into seperate smaller functions. Just select the code you want and the IDE is going to create a function definition based on it and replace your selection with a call to that function. I don't see how you could avoid this by making a more expressive language. Does your fancy programming language automatically organise your code into functions or data types and all you need to do is crank out code without a care? No, you still to need think about that yourself, but the IDE can automatically move the code around for you. Also let's talk about my favourite feature that I would never use without the assistance of an IDE: typeinference. value blah = foo.bar([1,2,3]) What the hell is a blah? With an IDE I could just hover over blah and instantly know what the function returns. Without that feature I have to go to the definition of bar which is still easier with an IDE. After all if foo is also defined via type inference I have to lookup several function definitions and that's going to take some time. On top of that a tool like grep is imprecise and will give me hundreds of call sites but only one definition. You can't fix everything by making a programming language more expressive and in my experience the dumber a programming language is, the easier it is to use with a dumb text editor.
- flavio81 9y ago
  >The importance of paredit is a language smell in my opinion, not something to aspire to. Paredit is just a nice-to-have. Many lispers don't use it.
ZenoArrow 9y ago
> "Paredit is simply where text and ast manipulation converge to the closest point." What about Parinfer? I don't use it but seems like it's even lower friction than Paredit. https://shaunlebron.github.io/parinfer/ https://shaunlebron.github.io/parinfer/
wruza 9y ago
>not being able to save syntactically incorrect code Going deeper, it would be nice to not being able to enter syntactically incorrect code.
- kazinator 9y ago
  That's possible, but it requires the machine to enter some things for you. Since ( is syntactically incorrect, it has to be entered together with ). We do see small examples of this in GUI's with strict validation on inputs that is applied for each keystroke. One useful compromise is syntax highlighting that doesn't let you enter syntactically incorrect code without flagging the problem in a loud red color.
  - wruza 9y ago
    Red hints annoy me honestly, when I meet these in real editors. I was thinking of something like grammar walker that completes input to shortest finished constructs. While not semantically correct, but syntactically like yes. pr pr; printf( printf(<?expr?>); Or to not edit code as text at all, but as a tree. Like apple plist editor or windows registry editor, but with convenient shortcuts to add constructs. Xcode can sort of simulate this with templates and placeholders: if␣ if (<?condition?>) { <?body?> }
pflanze 9y ago
S-expressions are not directly an AST: they are a form easily matched to an AST (which could be the Lisp at hands, or some language for which there is some translator/parser/interpreter around, or some language for there is no code in the system to (fully) handle it). This makes the use of s-expressions simple, they can be handled by the development tools without them always having to know what language they represent. If your development tool knows how to handle a particular language, then it can handle the AST for that language, but not any other. You could instruct the tool about the other languages, but that requires ways to let the tool know; in the end this is similar to the static versus dynamic typing discussion: static typing doesn't allow any potentially unsafe data, and a tool working with an AST directly wouldn't allow any potentially invalid programs, which in principle (or theory) is nice but difficult to achieve practically, it presents overheads for the tool implementer and for the user probably alike. And it probably takes away a lot of interactivity: just like reloading modules in a statically typed language (like ML or Haskell) is not as direct as being able to modify values and functions in a running system and having them be called directly. So I think S-expressions are a very pragmatic concept that is going to be difficult to replace without the replacement costing more than it benefits.
- lispm 9y ago
  That's absolutely right. S-expressions are a syntax for data. It's more related to what a tokenizer does, but with nested lists of tokens - not a flat token stream. It's only loosely related to an AST, since the s-expression represents some form of nesting - but not explicitly what kinds. S-expressions know nothing about the syntax of the program language Lisp (what is the syntax for the LET operator?). If we have a Lisp expression, an AST for it would need to represent specific language constructs - like knowing that LET is a built-in operator and has parts like a list of bindings (where each binding is a symbol or a list of symbol + value form, optionally declarations (with their specific syntax) and a list of body forms. The s-expression does not represent that: in (let ((let 'let)) (list let 'let)) the s-expression does not represent that LET is a built-in operator, a variable name, a symbol, a variable name and a symbol. All we see is five LET symbols and no additional information. Writing Lisp code using s-expressions allows us to use data manipulation functions both on text and on internal Lisp data: reverse, transpose, move, move in, move out, ... We can then also enrich the editor (or other tools) to understand the syntax of Lisp. Example: know what the structure of a LET form is and then allow for example to manipulate the binding list, knowing where it is and what a variable and what a value form is.
conistonwater 9y ago
One thing to consider is that s-expressions make implementing paredit easy. It takes an enormous amount of effort (e.g., libclang in C++) to get close to that without s-expressions. It's as if you think of programming languages as "fixed", and we only get to choose which kinds of tools we develop, but another way of doing it would be to design programming languages in ways that specifically make developing such tools easier. Then s-expressions could well be the go-to solution for that.

agumonkey 9y ago

now we need fnl-mode to enjoy lisp-lua repl and paredit in emacs :)