4 ms·
Sapling: Experimental vi-inspired editor where you edit code, not text
- drekipus 3y agoThis was something that I was thinking of for a while now, and now seeing what it looks like, I don't quite like it and I'm glad I don't have to try and implement it to see how it works! :D Great work though, this is a super cool project.
- gsuuon 3y agoI also had a similar thought, decided to implement it as a neovim plugin[1] to see what it feels like. Turned out okay! I use it daily now, but I think the important part is it's just a "mode" in an otherwise normal editor so I'm not forced to only navigate by AST nodes. [1] https://github.com/gsuuon/tshjkl.nvim https://github.com/gsuuon/tshjkl.nvim
- deleted 3y ago[deleted]
- Kab1r 3y agoI don't know why I haven't thought of this before, but I kinda love the idea of an AST Editor.
- linsomniac 3y agoA buddy of mine and I wanted to experiment with such a thing back in ~'85. I started by looking at the "elvis" editor code, trying to wrap my head around it, but I didn't really have a clear picture in my head about what a "modal AST editor" would do, just some vague ideas of an editor that operated on parsed syntax (AST wasn't on the horizon for us at that point). I was in HS at the time, and my buddy was working on a masters thesis, I think he was too busy and it was way too big a project for me to handle at that point. One interesting thing about GPT&cetera is that in order to "cheat" by using it, you need to clearly describe the problem you are trying to solve. If we can get more people to design before coding begins, I think we're going to have a win in any case.
- danielvaughn 3y agoIt’s a very interesting space. There are several experimental editors out there - they’re all rough as you might expect, but the possibilities are fun to think about. I’m building a semi-structured editor for UI - I’ve described it as “vim for designers.” Just as an example of how varied the general idea can be.
- ethanbond 3y agoI’m VERY interested in this vim for designers idea. Have any resources available to learn more?
- danielvaughn 3y agoGlad to hear there’s interest! Right now it’s still a small side project, but I could probably toss up a prototype to play around with. I was working on a programming language for designers, and have recently been thinking that it just works better if you build against an AST directly instead of text. The language is called Matry: https://matry.design/ https://matry.design/
- airstrike 3y agoI'm also very interested. I have similar thoughts about the future of "design" (for a very wide-ranging meaning of the word)
- danielvaughn 3y agoYeah the current tooling landscape is seriously underpowered.
- otikik 3y agoI'm worse, I have thought about this many times but never did anything with the idea. My respect for the authors for actually trying to materialize it.
- zokier 3y agoSince this project ended, we've have had quite a lot nice development in this front. Biggest highlight maybe is Combobulate for emacs, providing treesitter powered editing. Neovim also has treesitter and some plugins[2] to work with it. My editor of choice, Helix, also has treesitter based commands built-in[3]. All of these afaik have been released after sapling While it now seems like this mode of using TS/structured editing to extend traditional editors will be the dominant way, I think it still might be useful to keep thinking what an editor that embraces structured editing could look like. [1] https://github.com/mickeynp/combobulate https://github.com/mickeynp/combobulate [2] https://github.com/nvim-treesitter/nvim-treesitter/wiki/Extra-modules-and-plugins https://github.com/nvim-treesitter/nvim-treesitter/wiki/Extr... [3] https://docs.helix-editor.com/usage.html#navigating-using-tree-sitter-textobjects https://docs.helix-editor.com/usage.html#navigating-using-tr...
- paholg 3y agoI've been using and enjoying Helix for a while now, but I don't use any structured editing commands. I should definitely try some out!
- _a_a_a_ 3y agoI had reasonable experience with structure editors (worked on one long time ago, for beginner programmers) and at least in their naïve form they are a bloody pain to use, and far slower to get information in than a halfway decent touch typist. It's possible some improvements might make things better, but it's really hard to see. Happy to be proven wrong!
- conartist6 3y agoDon't forget the language intelligence core being built from the ground up as a universal engine for semantic editors! [4] https://github.com/bablr-lang/ https://github.com/bablr-lang/
- linsomniac 3y ago[flagged]
- neongreen 3y agoIt's 437 now so you can star
- neilv 3y agoFor Emacs users, there's Taylor Campbell's `paredit`: https://www.youtube.com/watch?v=D6h5dFyyUX0&t=51s https://www.youtube.com/watch?v=D6h5dFyyUX0&t=51s https://paredit.org/ https://paredit.org/ https://www.emacswiki.org/emacs/ParEdit https://www.emacswiki.org/emacs/ParEdit
- tomjakubowski 3y agoEmacs kids these days switched to using smartparens and now Parinfer
- theCodeStig 3y agoLispy https://github.com/abo-abo/lispy https://github.com/abo-abo/lispy
- ctrw 3y agoWhen you're too cool for temporal modal editing so you invent spacial modal editing. Been using that for a while and yes it is the best parts of vim with the best parts of Emacs. The main selling point is that I can now edit lisp code on my phone with the thumb key keyboard... Come to think of it I can probably use a physical corded keyboard now too...
- ctrw 3y agoNo, turns out physical chorded keyboards are a lot worse than I remember for code.
- neilv 3y agoCirca 2000, some people were using the Twiddler one-handed chorded keyboard with wearable computing. I'm not certain this was for coding, but Bradley Rhodes was using his wearable heavily, with Emacs as the primary UI in his head-up display.
- taeric 3y agoI think the core thought in "But Why" kind of gives away the problem this will face. The idea is that you will lose time as you "move between valid programs." A problem, though, is most "valid programs" are still not valid solutions to what you are working on. Consider, most of us work in rather phonetic languages as we type day to day. Surely it would be easier if we worked instead in languages where you did not have to do this. Let me type only valid sentences as I move from thought to thought. Surely that would be better? Why, then, do we not do that? Of course, I don't know why. I suspect there is a good deal to inertia style thoughts that explains this. But it is a good hint that the idea that you can break apart from symbols that need more symbols for context to build meaning is not one that really anything has fully converged to.
- silon42 3y agoI always think of this when I edit python and my code is indented wrong... not a fan by structuring by indentation, but it doesn't seem to be a problem for most people.
- taeric 3y agoCards on the table, I'm a lisp fan, so I'm probably the closest you can get to directly editing an AST for my dream program. I /like/ the idea of it. I just don't think it is a holy grail endpoint of program editing. Oddly, I do think there is something to making sure all of your data has a literal representation in the code, if you can. Such that I do think being able to directly interact with data is huge. I also love when you can symbolically work with programs. But, yeah, the overwhelming evidence is that this just isn't necessary for things. Both in communication, and in program specification.
- conartist6 3y agoI think you're thinking about it subtly wrong. A properly built semantic editor won't force you to move between valid programs in that sense. It will be comfortable decomposing programs into fragments. For example if I decompose the sentence "I press play" into words devoid of context, "I" still pretty much means what it did but "press" could now as easily mean pressing shirts or pressing an advantage, and "play" could be what children do, not the implied "play button". A good semantic editor for that sentence would allow me to detach the word "play" without losing the context that the meaning is "play button". Similarly a good semantic code editor would let me do something like pull two non-adjacent class methods out of a class and into a clipboard, then drop them into another class. If you did this this with text, the editor would not understand the clipboard content as class methods are (usually) not parseable outside a class.
- donpdonp 3y agoSapling is the first AST editor that works how Ive imagined it could. I'd love to leave all the whitespace wrangling behind and move only between AST nodes in an efficient way. This was a fun editor to try and I think there is something to learn here still to make treesitter editors more powerful/efficient.
- Jeaye 3y agoThis is one of the hills to climb when learning a lisp. Seasoned lispers don't edit text, or traverse text; they edit and traverse lisp forms (AST nodes). In my experience, vim/neovim work very well for this, due to their notion of "text objects". With a couple of plugins, text objects end up aligned directly with the lisp AST, but even better is that the AST is visual and tangible, since it's encoded in the text. It makes sense to benefit from this sort of thinking in other languages, but this is just another reminder, to me, of the beauty of lisps. A fun language idea would be to take the exact semantics of an existing language, like C++, and encode them within a lisp, losslessly. I suspect it would end up quite verbose. However, add in a macro system and then you can start building a DSL around the original language, keeping all the same underlying semantics.
- rakoo 3y agoIsn't that a bit what hy (https://hylang.org/ https://hylang.org/) tries to do ? AIUI it is a lisp interacting directly with the AST of Python, allowing seamless interop: Python modules can be used from hy and vice versa, everything is transparent.
- Jeaye 3y agoI'm not sure if it's doing exactly that, but it looks like it could be! Very neat. It'd be nice to see it for other langs, if Hy proves viability in the approach.
- kolme 3y agoOther commenter mentioned Hy, but what about Clojure? It is a lisp on top of Java, and all of its variants (Clojurescript -> Javascript, etc). Then there's Fennel (Lua) and others I can't remember right now.
- Jeaye 3y agoClojure sits on top of the JVM and allows for Java interop, but the language itself does not losslessly encode all of the semantics of Java. Same goes for ClojureScript and the other dialects. The difference here is that the lisp's AST would need to map 1:1 with its underlying language.
- rstat1 3y ago"where you edit code not text" gif shows the user editing text
- coolius 3y agosomebody tell them about antlr4
- LAC-Tech 3y agoOne of those things that I always thought was a great idea, and it's fantastic to see someone actually working on it. AST editing really is the next step for modal editors IMO.
- DonHopkins 3y agoReminds me of Steven Reiss's "PECAN" system he developed at Brown University in 1984, which he gave a talk about and gave us a copy of the Sun version to play around with at the UMD Human Computer Interaction Lab around 1985 or so. It seemed like an interesting idea, possibly useful for teaching programming to students who had never coded before (especially the data structure visualization and algorithm animation system BALSA), but essentially pretty frustrating and klunkey in practice. Of course we have a hell of a lot more computing power to piss away at the problem now, but it's still fundamentally at odds with how professional programmers code and how programming languages are designed (except for Lisp of course), and isn't a widely used approach for many practical reasons. https://cs.brown.edu/~spr/papers/pecanpaper.pdf https://cs.brown.edu/~spr/papers/pecanpaper.pdf >Graphical Program Development with PECAN Program Development Systems (1984) >Steven P. Reiss, Department of Computer Science, Brown University, Providence, RI 02912. >Abstract: >This paper describes the user's view of the PECAN family of program development systems. PECAN is a program development system generator for algebraic programming languages. The program development systems it produces support multiple views of the user's program, its semantics, and its execution. The program views include a syntax-directed editor, a declaration editor, and a structured flow graph editor. The semantic views include expression trees, data type diagrams, flow graphs, and the symbol table. Execution views show the program in action and the stack contents as the program executes. PECAN is designed to make effective use of powerful personal machines with high-resolution graphics displays and is currently implemented on APOLLO workstations. ChatGPT opines: >Subsequent research into syntax-directed editors has revealed that they are less effective than initially anticipated. These editors have been thoroughly examined over the past decade, leading to insights about their limitations and practical challenges. Studies have shown that while syntax-directed editors can be useful in certain contexts, such as language definitions and programming environment development, they haven't gained widespread popularity due to various factors. These include issues related to usability, integration with other development tools, and accommodating diverse programming styles. Additionally, the complexity of implementing efficient and robust syntax-directed editors that smoothly integrate into existing development environments has also contributed to their limited adoption. >Check out these search results to keep exploring. https://link.springer.com/chapter/10.1007/3-540-17189-4_87 https://link.springer.com/chapter/10.1007/3-540-17189-4_87 https://www.researchgate.net/publication/221330960_On_the_Usefulness_of_Syntax_Directed_Editors https://www.researchgate.net/publication/221330960_On_the_Us... >On the Usefulness of Syntax Directed Editors. (1986) >Bernard Lang, INRIA, B.P. 105, 78153, Le Chesnay, France. >Abstract >The intent of this position statement is to relate our experience with the actual use of a syntax directed editor (Mentor [5]) in different contexts: teaching, real software production, software maintenance and associated tasks, language design and development, development of programming tools and new programming structures. >1. Introduction >Originally the subject of this presentation was motivated by discussions to be found in the technical literature [1,2,3], and by the open questioning in less technical publications (as well as in private discussions) of the real usefulness of the syntax directed approach, its practicality and its high cost for allegedly low returns [4]. While the author does not dispute many of the points raised in the technical literature, he believes they emphasize too much the role of syntax directed tools for program creation, and thus lead to a biased assessment of their real usefulness. This is currently resulting in the marketing of sophisticated syntax directed editors, restricted to program creation, whose cost effectiveness may indeed be questionable. >The ideas expressed here reflect only the point of view of this author, and are intended as a basis for discussion on the role of syntax directed editors. The personal judgements on various aspects of syntax directed environments are not meant as an assessment of the usefulness of research in those areas, but only as a tentative evaluation of the relative importance of these aspects for users, given the current state of the art and the author's own experience. >This experience is based almost solely on the use of the Mentor system [5]. Though comparison of several systems would be desirable, the youth of this technology makes it rather difficult to acquire extensive practical experience with more than one of them. [...] I think Brett Victor's work on Learnable Programming, program visualization and live programming is a more modern and fruitful approach. http://worrydream.com/LearnableProgramming/ http://worrydream.com/LearnableProgramming/ >As you can see, live coding, on its own, is not especially useful. The programmer still must type at least a full line of code before seeing any effect. This means that she must already understand what line of code she needs to write. The programmer is still doing the creative work entirely in her head -- imagining the next addition to the program and then translating it into code. [...] >Here's an example. In many styles of programming today, when an application launches, it creates a large set of interconnected stateful objects. To see the effect of a code change, the application must be "relaunched" -- that is, its entire world is destroyed, and rebuilt again from scratch. How can we "create by reacting", continuously changing the code and seeing continuous effects in the flow and data, when there is no continuity between the application's state before and after the change? >We can't. That's the wrong question. A better question is: How do we design a new programming model that does allow for continuous change? We already have clear hints.* >Another example. Most programs today manipulate abstract data structures and opaque objects, not pictures. How can we visualize the state of these programs? >Again, wrong question. A better attitude is to assert that we have to be able to understand the state of our programs. We can then ask: How do we design data structures that can be visualized? Can we invent data structures that are intended to be visualized? How do we move towards a culture where only visually-understandable data is considered sound? Where opaque data is regarded in the same way that "goto" is today?* Forward reference: Some work that I've done in automatic visualization of ad-hoc data structures will be published later this year, in collaboration with Viewpoints Research. >In his influential essay No Silver Bullet, Fred Brooks makes the case that software is inherently "invisible and unvisualizable", and points out the universal failure of so-called "visual programming" environments. I don't fault Fred Brooks for his mistake -- the visual programming that he's thinking of indeed has little to offer. But that's because it visualizes the wrong thing. >Traditional visual environments visualize the code. They visualize static structure. But that's not what we need to understand. We need to understand what the code is doing. >Visualize data, not code. Dynamic behavior, not static structure. >Maybe we don't need a silver bullet. We just need to take off our blindfolds to see where we're firing.
- keefle 3y agoSuch works might be quite useful for natural language AI infused code editing as there would be a coherent interface for the AI to give commands to and a smaller room of error (given that the action space is smaller than when a standard text editor is used)
- conartist6 3y agoHere are some problems faced by semantic editors: 1. Bias. A lot of people think that if good ones haven't been built already, there must be some reason they can't be built 2. You really want the parser to report not just what types of nodes it found at particular locations, but what types of nodes it could have found at that location, because those are the types that will be valid for insertion there. A semantic editor needs to know because it should only be able to create programs that the parser would be able to parse. 3. Extensible parsers: To make a huge number of parsers that exactly match languages, you need to be able to quickly define parsers that are closely related to each other, and what's more the cost of using the derived parser must be closely related to the cost of using the original. Tree-sitter can derive languages at compile-time, but not at runtime leading to three major problems: - 3.1 Runaway storage costs: the Tree-sitter grammar for JS is ~4mb, and the grammar for TS is ~12 MB and the grammar for TSX is also ~12mb and these grammars have things they shouldn't have in them like Ember's glimmer syntax, because that syntax is too poorly supported to justify paying the costs of copying the JS, TS, and TSX grammars (causing 28MB of new code duplication, in git no less). - 3.2 Failed semantics: Because the JS runtime grammar is just code copied into the TS runtime grammar, tree-sitter can't distinguish which parts of TS programs are really just plain JS. - 3.3 Umbrella languages: Because the grammar defines languages that are bigger than what you can actually use, the JS tree-sitter grammar will happily parse JSX even when it's invalid, which is to say a semantic editor would suggest inserting JSX even when it would break your program, which is clearly bad.
- fulafel 3y agoPython's predecessor, ABC used a structured editor. See also: https://en.wikipedia.org/wiki/Structure_editor https://en.wikipedia.org/wiki/Structure_editor