13 ms·

Show HN: Parinfer – a simpler way to write Lisp

674 points by undershirt 11y ago

davexunit 11y ago
Good to see more people working on editors for Lisp, but I don't really see this as an improvement over the (immaculate, IMO) Paredit for Emacs.
- ScottBurson 11y ago
  Yes, those of us who have been editing Lisp using Paredit-like commands for years [0] may not find that much value here. But I think we're not the target audience; the goal is to help get newcomers over the hump, and that will be a wonderful thing if it works. [0] 35 years, in my case. David Chapman and I implemented balanced-parenthesis editing commands for Zmacs (the Lisp Machine editor) in 1980. To my knowledge, it was the first such implementation for a text editor; the Interlisp structure editor already existed, but was considered hard to use.
  - mbrock 11y ago
    This Chapman, I assume? https://meaningness.wordpress.com/about-david-chapman/ https://meaningness.wordpress.com/about-david-chapman/ I'm something of a fan. This article is great, about his history at MIT's artificial intelligence lab, Ken Wilber, Heidegger, and Buddhism. http://meaningness.com/metablog/ken-wilber-boomeritis-artificial-intelligence http://meaningness.com/metablog/ken-wilber-boomeritis-artifi...
    - ScottBurson 11y ago
      Yep, that's him! Thanks for the links -- I had lost track of him and hadn't seen his blog.
      - mbrock 11y ago
        :) Here he is on Twitter: https://twitter.com/meaningness https://twitter.com/meaningness While sleuthing a bit on Google I realize he was also a contributor to the "UNIX-HATERS handbook." Weirdly, that was really fascinating literature to me as a ... 15 year old, maybe, in like 2004? In my Swedish teenage room full of messed up Linux and OpenBSD boxes, reading everything I could that seemed interesting and intelligent. That handbook, the C2 wiki, various mailing lists, Usenet groups, IRC channels... I attribute a lot of my personality and interests to this nerdy/literate heritage.
        ScottBurson 11y ago
        Yes. I'm in the UHH too :-) Bottom of p. 211.
- Blackthorn 11y ago
  This is a huge improvement imo, and something that should be seen as a great advancement in usability. We've gone from needing to use hotkeys (paredit) to not needing to use them and having the machine figure out what you want to do. Maybe others have different opinions, but I find this way of doing things a lot more natural and usable. It's like an old Oblivion mod for archery that asked the question: "Why do other archery mods have a hotkey for denocking your arrow? We just automatically denock if you look at the ground. It just makes sense."
  - Illotus 11y ago
    This. It may not be huge for someone who is accustomed to those hotkeys, but for everyone this is a big deal. Imagine not having to commit those hotkeys to (muscle) memory at all.
- danneu 11y ago
  If it lets people write lisp with the same ease of writing Javascript or Python (i.e. without investing time learning a tool specifically for editing s-expressions), then that's an advantage over Paredit kinda like Sublime's advantage over Vim/Emacs. Trying it out in Atom, it looks really promising. You just kinda start writing, like on the blank line at the end of a function body, and Parinfer fills in the parens depending on your indentation level in an intuitive fashion, merging your form into the tree above it. In fact, it seems to strictly complement Paredit. Parinfer puts the parens where you want them, and Paredit is there for explicitly editing the tree.
  - davexunit 11y ago
    > kinda like Sublime's advantage over Vim/Emacs. Sublime is trash compared to Emacs.
    - roryokane 11y ago
      danneu wasn’t necessarily saying that Sublime is better than Emacs overall. He was referring to Sublime’s specific comparative advantage over Emacs. The relevant advantage here is that Sublime is better at being easy to learn by new users, because it follows conventions used by the OS and by other programs.
  - sklogic 11y ago
    Did I miss something and Sublime somehow got some advantage over Emacs? When did it happen and what kind of advantage, exactly?
    - roryokane 11y ago
      Sublime Text’s advantage in this context is the advantage that all GUI text editors share, including Atom and GEdit. This advantage is a combination of being more familiar and having better defaults. Some ways that Sublime is more familiar are that it uses the OS-standard editing hotkeys, such as moving by words with Ctrl-arrows or Option-arrows. It also uses OS-standard terminology, like “close window” instead of “kill buffer” and “paste” instead of “yank”. The result is that new users don’t have to learn anything to be productive – they just edit as if they are in Microsoft Word, while enjoying the programming-specific features of a text editor such as syntax highlighting and block folding. The better defaults include easily creating more than one scratch buffer (just run “New File” multiple times), automatically balancing parens when you type, support for pixel-by-pixel smooth scrolling, and scrollbars that don’t confusingly shrink when you start scrolling past the end of the buffer. I’m sure Emacs supports all these things, but you have to manually configure it to do so first. Many users are scared off from Emacs after seeing that it requires up-front investment to get features that other editors already provide with no work.
sdegutis 11y ago
When I started working on a Clojure web app full time, I forced myself to spend a week learning and configuring Emacs, and learning Paredit. To this day I have not found a tool that works as well as Paredit for editing and navigating s-expressions. Parinfer looks like a cool subset of Paredit's abilities. I'd love to see it expanded to include all of what Paredit can do. EDIT: I was mistaken, Parinfer does things Paredit doesn't. They could totally work together, that'd be pretty awesome.
- rntz 11y ago
  I see no reason they couldn't coexist. Parinfer to guess paren insertion; paredit to cover the operations parinfer can't infer. I'm more concerned that Parinfer appears to be modal, based on whether you manage indentation or parens. This seems overly complicated; it would be nice if there's some elegant way to merge the features of both into one mode. But I don't know how to do this, and I haven't actually used Parinfer, so maybe there's nothing to worry about.
  - rcthompson 11y ago
    It looks like one of the modes is mainly intended for fixing unmatched parens before switching into the other mode.
- nanny 11y ago
  >To this day I have not found a tool that works as well as Paredit for editing and navigating s-expressions. https://github.com/Fuco1/smartparens https://github.com/Fuco1/smartparens Basically paredit on steroids.
  - sdegutis 11y ago
    That's a lot of paragraphs. Got a tl;dr on how it improves on Paredit?
    - robto 11y ago
      I was curious too, so I took a walk through the documentation. I found this: https://github.com/Fuco1/smartparens/wiki/Paredit-and-smartparens https://github.com/Fuco1/smartparens/wiki/Paredit-and-smartp... I think I might try it out. It seems by default it's less strict than paredit, though, so I'll have to tweak it a bit.
drcode 11y ago
Wow, I totally want to do my programming with this now... is there support for matching indents on 'let, as with align-cljlet?
chrisoakman 11y ago
Hi folks -- I've been working with Shaun on the tail end of this project. Have an early prototype of Parinfer working with the Atom editor here: https://github.com/oakmac/atom-parinfer https://github.com/oakmac/atom-parinfer It's pretty early software, but feedback appreciated :)
- cnp 11y ago
  Parinfer + Atom -- what a dream come true. Awesome work. (Edit: just took it for a spin, and it "just works", even with atom-vim -- doubly awesome. I really feel like having to navigate parens while learning the language was such a barrier to diving in to ClojureScript. Cant wait to get started.)
  - chrisoakman 11y ago
    This comment makes me so happy; thank you :) I'm glad it "just works"!
    - omginternets 11y ago
      Your project seriously makes me want to learn lisp.
      - tripzilch 11y ago
        Me too! And I already wanted to learn Lisp. Because I kind of half-assumed this tool to be there already :-P I always assumed, obviously Lisp didn't get this popular if people had to keep track of all their parentheses. Sure they probably had to in "the old days" (although I wonder even when?). It just seems like a tedious thing to have to do in order to work with such a consistently structured language. Of course you can get most of it with highlighting matching parens and auto-formatting shortcuts. Like most languages. But I kind of expected Lisp tooling to go the extra mile. Because it is that structured. From my limited knowledge of Lisp, I figured some vague ideas on how such a tool would work / should be designed. It's kind of cool to see that the author's initial steps match up with that idea: define a grammar, some invariants, and "have at it" (ok, so I didn't figure that far). And then he made it work! Maybe a silly question, but since the author is in this thread: Was it hard to make? :-) BTW the idea of switching between these two modes is a very clever approach to keeping certain invariants invariant while still being able to modify the code--because the trivial way to preserve the invariant would be the tool immediately reverting your edit :)
da4c30ff 11y ago
For emacs, there's the electric-indentation-mode and the fantastic aggressive-indent-mode. https://github.com/Malabarba/aggressive-indent-mode https://github.com/Malabarba/aggressive-indent-mode
- msbarnett 11y ago
  This reminds me more of Paredit, in that it handles s-expression aware source transformations, and not just indentation: http://emacsrocks.com/e14.html http://emacsrocks.com/e14.html I would deeply love to see this kind of inference engine ported to emacs as an alternative to Paredit's chords
- ruricolist 11y ago
  A more direct comparison for Emacs would be `adjust-parens`: https://elpa.gnu.org/packages/adjust-parens.html https://elpa.gnu.org/packages/adjust-parens.html
  - alphapapa 11y ago
    Wow! This is fantastic! This seems to do just what Parinfer does. Why isn't this mode more well-known? This makes it even better: (defun my/lisp-dedent-adjust-parens () (interactive) (save-excursion (x4-smarter-beginning-of-line) (call-interactively 'lisp-dedent-adjust-parens))) (defun my/lisp-indent-adjust-parens () (interactive) (save-excursion (x4-smarter-beginning-of-line) (call-interactively 'lisp-indent-adjust-parens))) (local-set-key (kbd "<M-left>") 'my/lisp-dedent-adjust-parens) (local-set-key (kbd "<M-right>") 'my/lisp-indent-adjust-parens) Now you can just M-left and M-right to adjust the indentation of any line, regardless of where the point is, and adjust-parens fixes the parens for you.
incepted 11y ago
This would make `git diff` an absolute nightmare.
- msellout 11y ago
  Not if you're using that style of bracing already. I'm guessing you have a different style convention?
- Estragon 11y ago
  "git diff -w"
- kazinator 11y ago
  How so? All it does is automate the maintenance of exactly the same formatting that the developer would otherwise produce by hand.
- chrisoakman 11y ago
  I can speak to this as I've been using Parinfer on existing code for the last week or so. There can potentially be a "big diff" when you run Paren Mode on a file for the first time, but that's typically a one-time event. The beauty of Parinfer is that most Lisp code already uses these conventions. Once you start using Parinfer's Indent Mode regularly it feels crazy to write or edit lisp without it. One of my plans for atom-parinfer[0] is to have a comment flag in a file that will automatically signal "use Parinfer on this file". Similar to JSHint inline configurations [1]. [0] https://github.com/oakmac/atom-parinfer https://github.com/oakmac/atom-parinfer [1] http://jshint.com/docs/#inline-configuration http://jshint.com/docs/#inline-configuration
munificent 11y ago
Wow, this is a fantastically documented and executed idea. I'm not enough of a Lisper to have an opinion on the content (though it certainly seems cool), but the presentation is top notch.
- aidenn0 11y ago
  As a lisper, the content is similarly well executed. It's a rarity to see both in one place.
jegutman 11y ago
I would even find this useful when writing python and defining nested dictionaries and lists.
lispm 11y ago
> Most programming languages have several syntax rules. Lisp has one: everything is a list. Oh, not again. The author is confusing s-expression syntax with Lisp syntax. Even s-expressions are more than lists (conses, cyclic conses, symbols, vectors, strings, various numbers, structures, ...) and the s-expression reader knows about more than pure s-expressions - for example comments. Not every s-expression is a valid Lisp expression. For example (let "let") is not a valid Common Lisp expression. The syntax for LET is: let ({var | (var [init-form])}*) declaration* form* => result* This is also using the syntax definition for 'declaration' and 'form'. Lisp has built-in syntax (provided by special operators in Common Lisp) and arbitrary complex syntax provided by macros. This is valid for other languages with a notation based on s-expressions, like Scheme or Clojure.
- chrisoakman 11y ago
  I can assure you the author of Parinfer is aware of this distinction ;) That sentence is an oversimplification to give context to the rest of the content in that section. Should probably have an asterisk to clarify.
  - lispm 11y ago
    It's one of the sentences which creates endless confusions and a false image of Lisp. 'Everything is a list'. Oh, Lisp has only one data structure? Lisp is difficult to read, because it has no other syntax? Lisp is a primitive language and not usable for real programmers... etc. etc. It also creates the impression that even proponents of Lisp-like languages don't actually understand the basics...
    - oconnore 11y ago
      Who is endlessly confused? Who suspects that the author of a neat lisp editor plugin has never written a non-trivial line of Common Lisp, Clojure, or Scheme? What are the chances that people in either category are also curious and excited to experiment with novel editing techniques?
ZenoArrow 11y ago
In the Parinfer introduction, 'C Style indentation' is mentioned. What I can't understand is why 'C Style indentation' isn't the default convention, at least in modern user-friendly Lisps. One of the biggest problems people have with Lisp is the readability of grouped parentheses, but the C Style indentation example shows you can write Lisp without them. I get that it's nice when programs fit in less lines of code, but isn't readability more important than line count?
- pnathan 11y ago
  C style is profoundly less readable for someone who has taken to time to familiarize themselves with Lisp.
  - ZenoArrow 11y ago
    Even when you can't use a Lisp-friendly editor?
    - pnathan 11y ago
      Really, all you need are highlighted matching parens, I can't think of a single programmer's editor that doesnt' do that.
      - ZenoArrow 11y ago
        Do you get that when you're reading through online tutorials?
        pavelludiq 11y ago
        https://github.com/orthecreedence/highlight-lisp https://github.com/orthecreedence/highlight-lisp provides paren matching on mouse hover, I find it annoying but I can see how it can be useful to some.
        oberstein 11y ago
        You don't really need it when reading tutorials either. Rainbow, simple matching, or manually counting parens is very much helpful when you're editing code, not so much when you're reading, at least if the code follows standard formatting idioms. You really get used to it: http://www.loper-os.org/wp-content/parphobia.png http://www.loper-os.org/wp-content/parphobia.png I would even argue you don't even absolutely need matching despite its helpfulness since Lisp was done and is done on many primitive editors without such features including pen and paper. As a first order approximation, you have just as many parens (braces, brackets) as a C-like language, the opening one is just on the other side, and C too can be done just fine with primitive editors.
abc_lisper 11y ago
Emacs people, I want this.
- lgas 11y ago
  Then write it.
- alphapapa 11y ago
  https://elpa.gnu.org/packages/adjust-parens.html https://elpa.gnu.org/packages/adjust-parens.html
cmrdporcupine 11y ago
Looks like what Jetbrains would implement if they implemented a proper Lispy language support. Very nice.
- porker 11y ago
  > Looks like what Jetbrains would implement if they implemented a proper Lispy language support. Let's hope they do, soon!
- sotojuan 11y ago
  Not by Jetbrains, but https://cursiveclojure.com/ https://cursiveclojure.com/
- nnq 11y ago
  Am I the only one who is incredibly annoyed by how Jetbrains IDEs indent code?! Really, the fact that they totally fail to indent even mildly broken code is infuriating. At least ten times a day I feel like yelling at webstorm "YES, I know that that code is broken, that there are two unpaired {s and one unpaired ', but I'll fix that F LATER! NOW I just want to type this code here and have it properly indented even if there's broken code 10 lines above it and 20 lines below it. I KNOW, and I DON'T care NOW!". I just want to download stuff from my head to the editor before I forget it, then later fix the mess, and have the IDE/Editor still help me in the meantime. If it needs to temporarily revert to a caveman-style indent-like-the-previous-line or event to temporarily disable auto-indent, I'm ok with that, but don't horribly mess up what I type in failed attempts to semantically auto-indent broken code that I don't wanna fix right now...
  - taternuts 11y ago
    No - you aren't the only one, this infuriates me sometimes. The worst is when you have a file that someone accidentally mixed indentations or something, or use 2 spaces instead of 4 (even 4 spaces seems to break for me) for continuation/promise stuff. I'll enter insert mode and it'll be in-explicably tabbed out 5-6 times further than it should and I have to exit insert mode and manually move over to the correct spot or finish typing the function in shit-mode then manually fix the indenting. There's been several times where I've given up in frustration and opened up VIM or sublime text because I don't want to deal with it anymore.
Blackthorn 11y ago
To copy a comment I made elsewhere: I'm seriously impressed. This is amazing. Paredit always ended up being a requirement for me, but this takes it to the next level. What other editing problems out there could have excessive hotkey usage replaced by something automatic with the same level of power, I wonder? This is a great example of an area where user interaction experts can really have a lot of impact, and I think shows why engineers ought to be more accepting of disciplines like that.
mfikes 11y ago
I'm wondering what Parinfer would feel like if used when editing the expression to submit in a REPL. (In other words, it could be nice outside of source editors.)
- danneu 11y ago
  This is a good point. I immediately dismissed Clojure when I played with the in-browser REPL http://www.tryclj.com/ http://www.tryclj.com/. Within moments, I imagined the daily plight of the professional Clojure developer tediously balancing parens like I had to do in that REPL. Had I known how polished my workflow would be a year later with Paredit + Emacs in-buffer line-by-line code evaluation, or that tool-supported s-expression editing would let me write/refactor/move/nest code much faster than in other dynamically-typed languages, I would've stopped what I was doing and started learning Clojure immediately. But you don't know any of that if you're someone going to http://www.tryclj.com http://www.tryclj.com for the first time. Seems like Parinfer could be used in environments where you don't have Paredit available or, perhaps more importantly, where the user hasn't already credentialized in a niche tool for editing s-expressions.
ChicagoBoy11 11y ago
I don't know what's more impressive: The work itself or the beautiful documentation which accompanies it. Very impressed!
sbochins 11y ago
Wow, this is really something. I never really thought about it and assumed that paredit would be as good as it gets. But, this seems to show that is not the case. I really need to check this out.
sagarjauhari 11y ago
Love the fact that it is written in Clojure
mark_l_watson 11y ago
A beautifully written web site. It made me happy just to look at it. I haven't been doing much Clojure or Common Lisp development this year but I am putting Parinfer on my look-at list.
cabalamat 11y ago
Is there a version of this site without the distracting animated text, so I can see what is going on?
- coldtea 11y ago
  The main way to see what's going is is the animated text -- because what's going on is live re-indenting of code.
Sevzinn 11y ago
Made just good enough to get a job Rejection
nickpsecurity 11y ago
Great improvement on dealing with the parentheses!
yarrel 11y ago
As an aside, I love the lego diagrams.
tmalsburg2 11y ago
Eh, not sure I like it. For this to work, they have to make the indentation part of the syntax. Otherwise stuff is way too ambiguous to infer the correct parenthesis. A result is that the programmer has to be intimately familiar with how the inference works and what the syntax of indentation is. I feel that this just shifts cognitive load from managing parenthesis to managing indentation and inference. From the documentation: The rules for what happens when inserting/deleting parens must be learned. Also, the case necessitating a "Paren Mode" comes at the cost of forcing the user to understand when and how to switch editing modes. Also, the preprocessor step performed when opening files will cause more formatting-related changes in your commit history when collaborating with others not using Parinfer.
- chrisoakman 11y ago
  they have to make the indentation part of the syntax The syntax remains unchanged. Parinfer only "infers" the structure of your code based on the same indentation that you typically would do anyway as part of formatting your code to be readable. shifts cognitive load from managing parenthesis to managing indentation Yes, exactly :)
namuol 11y ago
Taken a step further, a structural editor could prevent incorrect syntax outright, and even allow you to view/edit the structure of your code in ways that simple syntax-highlighted text cannot. Here's one such experimental editor: https://github.com/darwin/plastic https://github.com/darwin/plastic Edit: I see now that Parinfer refers to Plastic under its Acknowledgements section. :)
michaelmior 11y ago
This is great! I would love to see similar functionality integrated into Dr. Racket.
dTal 11y ago
In the example: (foo [1 2 3 4 5 6 7 8 9]) How does one change it to: (foo [1 2 3] [4 5 6] [7 8 9]) ? edit: Ah, it works if you put all the forms on one line. I suppose it would be considered non-standard to indent that particular syntax in such a way.
- chrisoakman 11y ago
  You could also just put a "[" in front of "4" and "7". Parinfer will infer the closing brackets.
  - dTal 11y ago
    That yeilds (foo [1 2 3 [4 5 6] [7 8 9]])
oberstein 11y ago
Very neat demo, I hope it gets integrated into the major Clojure IDEs soon as it could make the onboarding experience smoother. I'm happy with just vim though -- I never liked the fashion of editors inserting/deleting/replacing non-whitespace characters (even that took a while to get used to) when they think they can to 'help' me.
dopamean 11y ago
Man I wish the paren management stuff for vim didn't suck.
jonathaneunice 11y ago
Automagical.
daly 11y ago
You've done great work. But speaking as an old lisp programmer it doesn't seem useful. Code, for me, is a means of communication with both the machine and the human reader. The machine doesn't care about lisp indentation but code format matters when trying to communicate with human readers. Communication with future readers, including yourself, is what makes programs maintainable. Pretty-printing is fine since the code format follows a standard, which makes it easy to ignore and easy to read, generally a "good thing". On the other hand, there is information in the choice of code format, similar to the information found in a poetry format. Properly indented code is "good code" and, over time, is likely to be what flows from your fingers regardless of the editor. But excellent code is an art form that requires thought. Lisp is a language where indentation is meaningless but shape communicates. It is not something to delegate to an editor.
ksmithbaylor 11y ago
Wow! This is really impressive! I can't wait until someone ports this to vim. I might even take a stab at it myself. Great work!
zeckalpha 11y ago
If I go to lisp for day to day work, I plan on using the syntax from SRFI-110 to encourage other people to work with me: http://srfi.schemers.org/srfi-110/srfi-110.html http://srfi.schemers.org/srfi-110/srfi-110.html Even if the code at rest is in S-expression syntax, editing it in this syntax would be slick.
deleted 11y ago
[deleted]
lewisl9029 11y ago
I'm curious about this point from the article: > And alternative syntaxes (Lisps without parens) have faultered since they sacrifice some of Lisp's power that seasoned users aren't willing to part with. Could Elixer be an example of one of these Lisps without parens? [1] Can anyone give an example of what kinds of Lisp powers are sacrificed in a syntax where the parens are implicit rather than explicit? I can't think of anything off the top of my head. It seems to me that Parinfer, when working as intended, almost turns a regular Lisp into one of these parenless Lisps, by performing parens manipulation for us, which in turn allows us to treat whatever Lisp we're working with as if it was a parenless Lisp. [1]: https://blog.8thlight.com/patrick-gombert/2013/11/26/lispy-elixir.html https://blog.8thlight.com/patrick-gombert/2013/11/26/lispy-e...
- thenipper 11y ago
  I don't know it at all, but isn't Dylan a Lisp w/o parens?
- elcritch 11y ago
  Using parens just makes manipulating the AST easier, but doesn't necessarily remove the homoiconocity of a language. I've barely done any lisp programming, though I have done a fair amount of work with AST's and compilers. S-exprs lend to thinking of the structure of the code as data, useful for "lisp-ing". Personally I don't care to write in lisp since I loose track of parens way too readily. This project might be very handy! In the meantime I've come to view Julia as my goto lisp. Quite fun to be able to write hygienic macros, but still have a mostly familiar imperative syntax for common tasks. Actually, the disciplined (read limited) usage of macros in Julia is a big draw for me, since I've heard too many horror stories of macro readers and spaghetti code in large lisp projects. Interesting look at elixir, I'll have to play with it sometime.
- roryokane 11y ago
  I think the author is talking about minor variations on Lisp syntax that include indentation-sensitivity, not about different languages that are homoiconic. Such variant syntaxes include i-expressions (http://srfi.schemers.org/srfi-49/srfi-49.html http://srfi.schemers.org/srfi-49/srfi-49.html) and sweet-expressions (http://readable.sourceforge.net/ http://readable.sourceforge.net/), which the author mentions in the Acknowledgements. For example, here are some sweet-expressions: define factorial(n) if {n <= 1} 1 * n factorial(- n 1) This desugars to: (define (factorial n) (if (<= n 1) 1 (* n (factorial (- n 1))))) I think the Lisp power that the author is referring to is homoiconicity, and the corresponding ease of writing macros. For example, i-expressions sacrifice homoiconicity in some contexts. But the author seems to have overlooked that sweet-expressions, which were developed later, do not sacrifice homoiconicity. A human reader can still easily understand the AST from reading sweet-expression syntax. The real reasons that sweet-expressions have faltered relate more to the increased difficulty of collaborating with other developers, the lack of editor plugins that highlight sweet-expressions correctly, and incompatibility with Clojure syntax.
hulkaad 11y ago
I look at it and keep wondering why it hasn't always been this way.
titanomachy 11y ago
What tools did you use to make the documentation? It's excellent.

674 points by undershirt 11y ago

Good to see more people working on editors for Lisp, but I don't really see this as an improvement over the (immaculate, IMO) Paredit for Emacs.

ScottBurson 11y ago
Yes, those of us who have been editing Lisp using Paredit-like commands for years [0] may not find that much value here. But I think we're not the target audience; the goal is to help get newcomers over the hump, and that will be a wonderful thing if it works. [0] 35 years, in my case. David Chapman and I implemented balanced-parenthesis editing commands for Zmacs (the Lisp Machine editor) in 1980. To my knowledge, it was the first such implementation for a text editor; the Interlisp structure editor already existed, but was considered hard to use.
- mbrock 11y ago
  This Chapman, I assume? https://meaningness.wordpress.com/about-david-chapman/ https://meaningness.wordpress.com/about-david-chapman/ I'm something of a fan. This article is great, about his history at MIT's artificial intelligence lab, Ken Wilber, Heidegger, and Buddhism. http://meaningness.com/metablog/ken-wilber-boomeritis-artificial-intelligence http://meaningness.com/metablog/ken-wilber-boomeritis-artifi...
  - ScottBurson 11y ago
    Yep, that's him! Thanks for the links -- I had lost track of him and hadn't seen his blog.
    - mbrock 11y ago
      :) Here he is on Twitter: https://twitter.com/meaningness https://twitter.com/meaningness While sleuthing a bit on Google I realize he was also a contributor to the "UNIX-HATERS handbook." Weirdly, that was really fascinating literature to me as a ... 15 year old, maybe, in like 2004? In my Swedish teenage room full of messed up Linux and OpenBSD boxes, reading everything I could that seemed interesting and intelligent. That handbook, the C2 wiki, various mailing lists, Usenet groups, IRC channels... I attribute a lot of my personality and interests to this nerdy/literate heritage.
      - ScottBurson 11y ago
        Yes. I'm in the UHH too :-) Bottom of p. 211.
Blackthorn 11y ago
This is a huge improvement imo, and something that should be seen as a great advancement in usability. We've gone from needing to use hotkeys (paredit) to not needing to use them and having the machine figure out what you want to do. Maybe others have different opinions, but I find this way of doing things a lot more natural and usable. It's like an old Oblivion mod for archery that asked the question: "Why do other archery mods have a hotkey for denocking your arrow? We just automatically denock if you look at the ground. It just makes sense."
- Illotus 11y ago
  This. It may not be huge for someone who is accustomed to those hotkeys, but for everyone this is a big deal. Imagine not having to commit those hotkeys to (muscle) memory at all.
danneu 11y ago
If it lets people write lisp with the same ease of writing Javascript or Python (i.e. without investing time learning a tool specifically for editing s-expressions), then that's an advantage over Paredit kinda like Sublime's advantage over Vim/Emacs. Trying it out in Atom, it looks really promising. You just kinda start writing, like on the blank line at the end of a function body, and Parinfer fills in the parens depending on your indentation level in an intuitive fashion, merging your form into the tree above it. In fact, it seems to strictly complement Paredit. Parinfer puts the parens where you want them, and Paredit is there for explicitly editing the tree.
- davexunit 11y ago
  > kinda like Sublime's advantage over Vim/Emacs. Sublime is trash compared to Emacs.
  - roryokane 11y ago
    danneu wasn’t necessarily saying that Sublime is better than Emacs overall. He was referring to Sublime’s specific comparative advantage over Emacs. The relevant advantage here is that Sublime is better at being easy to learn by new users, because it follows conventions used by the OS and by other programs.
- sklogic 11y ago
  Did I miss something and Sublime somehow got some advantage over Emacs? When did it happen and what kind of advantage, exactly?
  - roryokane 11y ago
    Sublime Text’s advantage in this context is the advantage that all GUI text editors share, including Atom and GEdit. This advantage is a combination of being more familiar and having better defaults. Some ways that Sublime is more familiar are that it uses the OS-standard editing hotkeys, such as moving by words with Ctrl-arrows or Option-arrows. It also uses OS-standard terminology, like “close window” instead of “kill buffer” and “paste” instead of “yank”. The result is that new users don’t have to learn anything to be productive – they just edit as if they are in Microsoft Word, while enjoying the programming-specific features of a text editor such as syntax highlighting and block folding. The better defaults include easily creating more than one scratch buffer (just run “New File” multiple times), automatically balancing parens when you type, support for pixel-by-pixel smooth scrolling, and scrollbars that don’t confusingly shrink when you start scrolling past the end of the buffer. I’m sure Emacs supports all these things, but you have to manually configure it to do so first. Many users are scared off from Emacs after seeing that it requires up-front investment to get features that other editors already provide with no work.

sdegutis 11y ago

When I started working on a Clojure web app full time, I forced myself to spend a week learning and configuring Emacs, and learning Paredit. To this day I have not found a tool that works as well as Paredit for editing and navigating s-expressions. Parinfer looks like a cool subset of Paredit's abilities. I'd love to see it expanded to include all of what Paredit can do. EDIT: I was mistaken, Parinfer does things Paredit doesn't. They could totally work together, that'd be pretty awesome.

rntz 11y ago
I see no reason they couldn't coexist. Parinfer to guess paren insertion; paredit to cover the operations parinfer can't infer. I'm more concerned that Parinfer appears to be modal, based on whether you manage indentation or parens. This seems overly complicated; it would be nice if there's some elegant way to merge the features of both into one mode. But I don't know how to do this, and I haven't actually used Parinfer, so maybe there's nothing to worry about.
- rcthompson 11y ago
  It looks like one of the modes is mainly intended for fixing unmatched parens before switching into the other mode.
nanny 11y ago
>To this day I have not found a tool that works as well as Paredit for editing and navigating s-expressions. https://github.com/Fuco1/smartparens https://github.com/Fuco1/smartparens Basically paredit on steroids.
- sdegutis 11y ago
  That's a lot of paragraphs. Got a tl;dr on how it improves on Paredit?
  - robto 11y ago
    I was curious too, so I took a walk through the documentation. I found this: https://github.com/Fuco1/smartparens/wiki/Paredit-and-smartparens https://github.com/Fuco1/smartparens/wiki/Paredit-and-smartp... I think I might try it out. It seems by default it's less strict than paredit, though, so I'll have to tweak it a bit.

drcode 11y ago

Wow, I totally want to do my programming with this now... is there support for matching indents on 'let, as with align-cljlet?

chrisoakman 11y ago

Hi folks -- I've been working with Shaun on the tail end of this project. Have an early prototype of Parinfer working with the Atom editor here: https://github.com/oakmac/atom-parinfer https://github.com/oakmac/atom-parinfer It's pretty early software, but feedback appreciated :)

cnp 11y ago
Parinfer + Atom -- what a dream come true. Awesome work. (Edit: just took it for a spin, and it "just works", even with atom-vim -- doubly awesome. I really feel like having to navigate parens while learning the language was such a barrier to diving in to ClojureScript. Cant wait to get started.)
- chrisoakman 11y ago
  This comment makes me so happy; thank you :) I'm glad it "just works"!
  - omginternets 11y ago
    Your project seriously makes me want to learn lisp.
    - tripzilch 11y ago
      Me too! And I already wanted to learn Lisp. Because I kind of half-assumed this tool to be there already :-P I always assumed, obviously Lisp didn't get this popular if people had to keep track of all their parentheses. Sure they probably had to in "the old days" (although I wonder even when?). It just seems like a tedious thing to have to do in order to work with such a consistently structured language. Of course you can get most of it with highlighting matching parens and auto-formatting shortcuts. Like most languages. But I kind of expected Lisp tooling to go the extra mile. Because it is that structured. From my limited knowledge of Lisp, I figured some vague ideas on how such a tool would work / should be designed. It's kind of cool to see that the author's initial steps match up with that idea: define a grammar, some invariants, and "have at it" (ok, so I didn't figure that far). And then he made it work! Maybe a silly question, but since the author is in this thread: Was it hard to make? :-) BTW the idea of switching between these two modes is a very clever approach to keeping certain invariants invariant while still being able to modify the code--because the trivial way to preserve the invariant would be the tool immediately reverting your edit :)

da4c30ff 11y ago

For emacs, there's the electric-indentation-mode and the fantastic aggressive-indent-mode. https://github.com/Malabarba/aggressive-indent-mode https://github.com/Malabarba/aggressive-indent-mode

msbarnett 11y ago
This reminds me more of Paredit, in that it handles s-expression aware source transformations, and not just indentation: http://emacsrocks.com/e14.html http://emacsrocks.com/e14.html I would deeply love to see this kind of inference engine ported to emacs as an alternative to Paredit's chords
ruricolist 11y ago
A more direct comparison for Emacs would be `adjust-parens`: https://elpa.gnu.org/packages/adjust-parens.html https://elpa.gnu.org/packages/adjust-parens.html
- alphapapa 11y ago
  Wow! This is fantastic! This seems to do just what Parinfer does. Why isn't this mode more well-known? This makes it even better: (defun my/lisp-dedent-adjust-parens () (interactive) (save-excursion (x4-smarter-beginning-of-line) (call-interactively 'lisp-dedent-adjust-parens))) (defun my/lisp-indent-adjust-parens () (interactive) (save-excursion (x4-smarter-beginning-of-line) (call-interactively 'lisp-indent-adjust-parens))) (local-set-key (kbd "<M-left>") 'my/lisp-dedent-adjust-parens) (local-set-key (kbd "<M-right>") 'my/lisp-indent-adjust-parens) Now you can just M-left and M-right to adjust the indentation of any line, regardless of where the point is, and adjust-parens fixes the parens for you.

incepted 11y ago

This would make `git diff` an absolute nightmare.

msellout 11y ago
Not if you're using that style of bracing already. I'm guessing you have a different style convention?
Estragon 11y ago
"git diff -w"
kazinator 11y ago
How so? All it does is automate the maintenance of exactly the same formatting that the developer would otherwise produce by hand.
chrisoakman 11y ago
I can speak to this as I've been using Parinfer on existing code for the last week or so. There can potentially be a "big diff" when you run Paren Mode on a file for the first time, but that's typically a one-time event. The beauty of Parinfer is that most Lisp code already uses these conventions. Once you start using Parinfer's Indent Mode regularly it feels crazy to write or edit lisp without it. One of my plans for atom-parinfer[0] is to have a comment flag in a file that will automatically signal "use Parinfer on this file". Similar to JSHint inline configurations [1]. [0] https://github.com/oakmac/atom-parinfer https://github.com/oakmac/atom-parinfer [1] http://jshint.com/docs/#inline-configuration http://jshint.com/docs/#inline-configuration

munificent 11y ago

Wow, this is a fantastically documented and executed idea. I'm not enough of a Lisper to have an opinion on the content (though it certainly seems cool), but the presentation is top notch.

aidenn0 11y ago
As a lisper, the content is similarly well executed. It's a rarity to see both in one place.

jegutman 11y ago

I would even find this useful when writing python and defining nested dictionaries and lists.

lispm 11y ago

> Most programming languages have several syntax rules. Lisp has one: everything is a list. Oh, not again. The author is confusing s-expression syntax with Lisp syntax. Even s-expressions are more than lists (conses, cyclic conses, symbols, vectors, strings, various numbers, structures, ...) and the s-expression reader knows about more than pure s-expressions - for example comments. Not every s-expression is a valid Lisp expression. For example (let "let") is not a valid Common Lisp expression. The syntax for LET is: let ({var | (var [init-form])}*) declaration* form* => result* This is also using the syntax definition for 'declaration' and 'form'. Lisp has built-in syntax (provided by special operators in Common Lisp) and arbitrary complex syntax provided by macros. This is valid for other languages with a notation based on s-expressions, like Scheme or Clojure.

chrisoakman 11y ago
I can assure you the author of Parinfer is aware of this distinction ;) That sentence is an oversimplification to give context to the rest of the content in that section. Should probably have an asterisk to clarify.
- lispm 11y ago
  It's one of the sentences which creates endless confusions and a false image of Lisp. 'Everything is a list'. Oh, Lisp has only one data structure? Lisp is difficult to read, because it has no other syntax? Lisp is a primitive language and not usable for real programmers... etc. etc. It also creates the impression that even proponents of Lisp-like languages don't actually understand the basics...
  - oconnore 11y ago
    Who is endlessly confused? Who suspects that the author of a neat lisp editor plugin has never written a non-trivial line of Common Lisp, Clojure, or Scheme? What are the chances that people in either category are also curious and excited to experiment with novel editing techniques?

ZenoArrow 11y ago

In the Parinfer introduction, 'C Style indentation' is mentioned. What I can't understand is why 'C Style indentation' isn't the default convention, at least in modern user-friendly Lisps. One of the biggest problems people have with Lisp is the readability of grouped parentheses, but the C Style indentation example shows you can write Lisp without them. I get that it's nice when programs fit in less lines of code, but isn't readability more important than line count?

pnathan 11y ago
C style is profoundly less readable for someone who has taken to time to familiarize themselves with Lisp.
- ZenoArrow 11y ago
  Even when you can't use a Lisp-friendly editor?
  - pnathan 11y ago
    Really, all you need are highlighted matching parens, I can't think of a single programmer's editor that doesnt' do that.
    - ZenoArrow 11y ago
      Do you get that when you're reading through online tutorials?
      - pavelludiq 11y ago
        https://github.com/orthecreedence/highlight-lisp https://github.com/orthecreedence/highlight-lisp provides paren matching on mouse hover, I find it annoying but I can see how it can be useful to some.
      - oberstein 11y ago
        You don't really need it when reading tutorials either. Rainbow, simple matching, or manually counting parens is very much helpful when you're editing code, not so much when you're reading, at least if the code follows standard formatting idioms. You really get used to it: http://www.loper-os.org/wp-content/parphobia.png http://www.loper-os.org/wp-content/parphobia.png I would even argue you don't even absolutely need matching despite its helpfulness since Lisp was done and is done on many primitive editors without such features including pen and paper. As a first order approximation, you have just as many parens (braces, brackets) as a C-like language, the opening one is just on the other side, and C too can be done just fine with primitive editors.

abc_lisper 11y ago

Emacs people, I want this.

lgas 11y ago
Then write it.
alphapapa 11y ago
https://elpa.gnu.org/packages/adjust-parens.html https://elpa.gnu.org/packages/adjust-parens.html

cmrdporcupine 11y ago

Looks like what Jetbrains would implement if they implemented a proper Lispy language support. Very nice.

porker 11y ago
> Looks like what Jetbrains would implement if they implemented a proper Lispy language support. Let's hope they do, soon!
sotojuan 11y ago
Not by Jetbrains, but https://cursiveclojure.com/ https://cursiveclojure.com/
nnq 11y ago
Am I the only one who is incredibly annoyed by how Jetbrains IDEs indent code?! Really, the fact that they totally fail to indent even mildly broken code is infuriating. At least ten times a day I feel like yelling at webstorm "YES, I know that that code is broken, that there are two unpaired {s and one unpaired ', but I'll fix that F LATER! NOW I just want to type this code here and have it properly indented even if there's broken code 10 lines above it and 20 lines below it. I KNOW, and I DON'T care NOW!". I just want to download stuff from my head to the editor before I forget it, then later fix the mess, and have the IDE/Editor still help me in the meantime. If it needs to temporarily revert to a caveman-style indent-like-the-previous-line or event to temporarily disable auto-indent, I'm ok with that, but don't horribly mess up what I type in failed attempts to semantically auto-indent broken code that I don't wanna fix right now...
- taternuts 11y ago
  No - you aren't the only one, this infuriates me sometimes. The worst is when you have a file that someone accidentally mixed indentations or something, or use 2 spaces instead of 4 (even 4 spaces seems to break for me) for continuation/promise stuff. I'll enter insert mode and it'll be in-explicably tabbed out 5-6 times further than it should and I have to exit insert mode and manually move over to the correct spot or finish typing the function in shit-mode then manually fix the indenting. There's been several times where I've given up in frustration and opened up VIM or sublime text because I don't want to deal with it anymore.

Blackthorn 11y ago

To copy a comment I made elsewhere: I'm seriously impressed. This is amazing. Paredit always ended up being a requirement for me, but this takes it to the next level. What other editing problems out there could have excessive hotkey usage replaced by something automatic with the same level of power, I wonder? This is a great example of an area where user interaction experts can really have a lot of impact, and I think shows why engineers ought to be more accepting of disciplines like that.

mfikes 11y ago

I'm wondering what Parinfer would feel like if used when editing the expression to submit in a REPL. (In other words, it could be nice outside of source editors.)

danneu 11y ago
This is a good point. I immediately dismissed Clojure when I played with the in-browser REPL http://www.tryclj.com/ http://www.tryclj.com/. Within moments, I imagined the daily plight of the professional Clojure developer tediously balancing parens like I had to do in that REPL. Had I known how polished my workflow would be a year later with Paredit + Emacs in-buffer line-by-line code evaluation, or that tool-supported s-expression editing would let me write/refactor/move/nest code much faster than in other dynamically-typed languages, I would've stopped what I was doing and started learning Clojure immediately. But you don't know any of that if you're someone going to http://www.tryclj.com http://www.tryclj.com for the first time. Seems like Parinfer could be used in environments where you don't have Paredit available or, perhaps more importantly, where the user hasn't already credentialized in a niche tool for editing s-expressions.

ChicagoBoy11 11y ago

I don't know what's more impressive: The work itself or the beautiful documentation which accompanies it. Very impressed!

sbochins 11y ago

Wow, this is really something. I never really thought about it and assumed that paredit would be as good as it gets. But, this seems to show that is not the case. I really need to check this out.

sagarjauhari 11y ago

Love the fact that it is written in Clojure

mark_l_watson 11y ago

A beautifully written web site. It made me happy just to look at it. I haven't been doing much Clojure or Common Lisp development this year but I am putting Parinfer on my look-at list.

cabalamat 11y ago

Is there a version of this site without the distracting animated text, so I can see what is going on?

coldtea 11y ago
The main way to see what's going is is the animated text -- because what's going on is live re-indenting of code.

Sevzinn 11y ago

Made just good enough to get a job Rejection

nickpsecurity 11y ago

Great improvement on dealing with the parentheses!

yarrel 11y ago

As an aside, I love the lego diagrams.

tmalsburg2 11y ago

Eh, not sure I like it. For this to work, they have to make the indentation part of the syntax. Otherwise stuff is way too ambiguous to infer the correct parenthesis. A result is that the programmer has to be intimately familiar with how the inference works and what the syntax of indentation is. I feel that this just shifts cognitive load from managing parenthesis to managing indentation and inference. From the documentation: The rules for what happens when inserting/deleting parens must be learned. Also, the case necessitating a "Paren Mode" comes at the cost of forcing the user to understand when and how to switch editing modes. Also, the preprocessor step performed when opening files will cause more formatting-related changes in your commit history when collaborating with others not using Parinfer.

chrisoakman 11y ago
they have to make the indentation part of the syntax The syntax remains unchanged. Parinfer only "infers" the structure of your code based on the same indentation that you typically would do anyway as part of formatting your code to be readable. shifts cognitive load from managing parenthesis to managing indentation Yes, exactly :)

namuol 11y ago

Taken a step further, a structural editor could prevent incorrect syntax outright, and even allow you to view/edit the structure of your code in ways that simple syntax-highlighted text cannot. Here's one such experimental editor: https://github.com/darwin/plastic https://github.com/darwin/plastic Edit: I see now that Parinfer refers to Plastic under its Acknowledgements section. :)

michaelmior 11y ago

This is great! I would love to see similar functionality integrated into Dr. Racket.

dTal 11y ago

In the example: (foo [1 2 3 4 5 6 7 8 9]) How does one change it to: (foo [1 2 3] [4 5 6] [7 8 9]) ? edit: Ah, it works if you put all the forms on one line. I suppose it would be considered non-standard to indent that particular syntax in such a way.

chrisoakman 11y ago
You could also just put a "[" in front of "4" and "7". Parinfer will infer the closing brackets.
- dTal 11y ago
  That yeilds (foo [1 2 3 [4 5 6] [7 8 9]])

oberstein 11y ago

Very neat demo, I hope it gets integrated into the major Clojure IDEs soon as it could make the onboarding experience smoother. I'm happy with just vim though -- I never liked the fashion of editors inserting/deleting/replacing non-whitespace characters (even that took a while to get used to) when they think they can to 'help' me.

dopamean 11y ago

Man I wish the paren management stuff for vim didn't suck.

jonathaneunice 11y ago

Automagical.

daly 11y ago

You've done great work. But speaking as an old lisp programmer it doesn't seem useful. Code, for me, is a means of communication with both the machine and the human reader. The machine doesn't care about lisp indentation but code format matters when trying to communicate with human readers. Communication with future readers, including yourself, is what makes programs maintainable. Pretty-printing is fine since the code format follows a standard, which makes it easy to ignore and easy to read, generally a "good thing". On the other hand, there is information in the choice of code format, similar to the information found in a poetry format. Properly indented code is "good code" and, over time, is likely to be what flows from your fingers regardless of the editor. But excellent code is an art form that requires thought. Lisp is a language where indentation is meaningless but shape communicates. It is not something to delegate to an editor.

ksmithbaylor 11y ago

Wow! This is really impressive! I can't wait until someone ports this to vim. I might even take a stab at it myself. Great work!

zeckalpha 11y ago

If I go to lisp for day to day work, I plan on using the syntax from SRFI-110 to encourage other people to work with me: http://srfi.schemers.org/srfi-110/srfi-110.html http://srfi.schemers.org/srfi-110/srfi-110.html Even if the code at rest is in S-expression syntax, editing it in this syntax would be slick.

deleted 11y ago

[deleted]

lewisl9029 11y ago

I'm curious about this point from the article: > And alternative syntaxes (Lisps without parens) have faultered since they sacrifice some of Lisp's power that seasoned users aren't willing to part with. Could Elixer be an example of one of these Lisps without parens? [1] Can anyone give an example of what kinds of Lisp powers are sacrificed in a syntax where the parens are implicit rather than explicit? I can't think of anything off the top of my head. It seems to me that Parinfer, when working as intended, almost turns a regular Lisp into one of these parenless Lisps, by performing parens manipulation for us, which in turn allows us to treat whatever Lisp we're working with as if it was a parenless Lisp. [1]: https://blog.8thlight.com/patrick-gombert/2013/11/26/lispy-elixir.html https://blog.8thlight.com/patrick-gombert/2013/11/26/lispy-e...

thenipper 11y ago
I don't know it at all, but isn't Dylan a Lisp w/o parens?
elcritch 11y ago
Using parens just makes manipulating the AST easier, but doesn't necessarily remove the homoiconocity of a language. I've barely done any lisp programming, though I have done a fair amount of work with AST's and compilers. S-exprs lend to thinking of the structure of the code as data, useful for "lisp-ing". Personally I don't care to write in lisp since I loose track of parens way too readily. This project might be very handy! In the meantime I've come to view Julia as my goto lisp. Quite fun to be able to write hygienic macros, but still have a mostly familiar imperative syntax for common tasks. Actually, the disciplined (read limited) usage of macros in Julia is a big draw for me, since I've heard too many horror stories of macro readers and spaghetti code in large lisp projects. Interesting look at elixir, I'll have to play with it sometime.
roryokane 11y ago
I think the author is talking about minor variations on Lisp syntax that include indentation-sensitivity, not about different languages that are homoiconic. Such variant syntaxes include i-expressions (http://srfi.schemers.org/srfi-49/srfi-49.html http://srfi.schemers.org/srfi-49/srfi-49.html) and sweet-expressions (http://readable.sourceforge.net/ http://readable.sourceforge.net/), which the author mentions in the Acknowledgements. For example, here are some sweet-expressions: define factorial(n) if {n <= 1} 1 * n factorial(- n 1) This desugars to: (define (factorial n) (if (<= n 1) 1 (* n (factorial (- n 1))))) I think the Lisp power that the author is referring to is homoiconicity, and the corresponding ease of writing macros. For example, i-expressions sacrifice homoiconicity in some contexts. But the author seems to have overlooked that sweet-expressions, which were developed later, do not sacrifice homoiconicity. A human reader can still easily understand the AST from reading sweet-expression syntax. The real reasons that sweet-expressions have faltered relate more to the increased difficulty of collaborating with other developers, the lack of editor plugins that highlight sweet-expressions correctly, and incompatibility with Clojure syntax.

hulkaad 11y ago

I look at it and keep wondering why it hasn't always been this way.

titanomachy 11y ago

What tools did you use to make the documentation? It's excellent.