9 ms·

The evolution of a structural code editor

242 points by crowdhailer 2y ago

naikrovek 2y ago
I have wanted something like this for so long that I can’t remember when I started. Strongly coupling source code to plain text is a lost opportunity to improve tooling.
- jtwaleson 2y ago
  Same! I think we need to free "code" from files, characters, tabs/spaces and variable names. Put millions of ASTs in a database. Imagine how much cheaper and better AI code assistants can be if they can only generate valid code with their "next token".
  - zokier 2y ago
    At some point the AST needs to be serialized as bytes on disk. In the big picture, the cost of having that serialization format be also human readable is relatively small as long as the syntax is designed to be sane (i.e. not c++/perl)
  - norman784 2y ago
    > Put millions of ASTs in a database. That's basically what unison does, they store the source code in binary format in an sqlite database.
- School-Cotton 2y ago
  The emacs mode paredit is basically structural editing for lisp code (and some of its more modern competitors like Smartparens support other languages too, but not as well as Lisp). It's great.
- norir 2y ago
  I disagree that plain text is an obstacle towards better tooling. You can do most of what the author describes with direct plain text edits pipelined into an analysis program. But I also think that the ux the author presents is undesirable. When writing new code, one must pass through a phase of broken code to reach the working code. I would not want an editor blocking me from typing a word that isn't yet defined but I do want it to notify me instantly that there is a problem. I also struggled to understand the use case for the mouse based ux. I am typing this comment on an iphone mini and would much rather write raw text on its keyboard than use the clicking interface shown in the fibonacci example.
- vidarh 2y ago
  The problem is communicating about code. Almost every visual / structural programming language attempts strands on how you talk about the code, and e-mail people about the code, and write about the code, or even how you efficiently input it. It's a multi-decade-long problem with many attempts.
datadrivenangel 2y ago
In the data science space, people have been going back and forth on the benefits of Notebooks, which seem very similar, and were historically stored as json under the hood. There are many many benefits to plain text.
- jtwaleson 2y ago
  I don't think notebooks are very similar to be honest. The notebooks still emit code (typically python) which is then interpreted as usual.
norir 2y ago
I think structural editors are a great feature to add on top of a language with a definite textual representation. If you can pretty print the ast (as the fib example shows), you should generally be able to parse the output to generate the structured output.
mebazaa 2y ago
My first thought: maybe competitive programmers could find it useful, if it’s a way to reduce the amount of key presses needed to write their code? (At the very least, that’s a crowd that would be motivated enough to get accustomed to that way of writing code.)
- deleted 2y ago
  [deleted]
- poutsku 2y ago
  Every year when I participate in advent of code, I have this thought: "wouldn't it be great to solve these problems directly from my phone?" (The puzzles unlock quite early morning in my timezone and opening my laptop at that time somehow annoys me.) I have wondered many times if some structural editor with a bunch of high-level library methods (Dijkstra's, for instance) would actually give a good/fast experience for most days. I haven't had the time or interest yet to try and build something yet. But perhaps I should.
  - crowdhailer 2y ago
    I'd love to get this to the point where add your own keybindings to structural edits, potentially very involved ones. It's superficially not very different to a combination of vim macros, auto completes or templates in other editors but I think the end result could be interesting as the units of change would be more meaningful. I think it's the same benefit of a hygienic macro system vs a source file manipulating version of meta programming.
lincpa 2y ago
[dead]
criddell 2y ago
Is it far enough along where you could work on the code editor in the code editor?
- bennythomsson 2y ago
  As the videos show, there are still a few usability issues. Full-screen switching somewhere else just to enter a name for example doesn't get you very far. But there is not much missing and I like the POC!
- crowdhailer 2y ago
  The code editor is for the EYG language and implemented in Gleam. So that's a blocker. The editor is probably far enough along to have a crack at it by EYG would need some UI or web library so that EYG was able to build an editor.
narag 2y ago
This is the way. Programs are not text, there's an impedance mismatch that shows at the seams. There're good reasons it hasn't been done more: text is good for troubleshooting, "every program should do one thing well" and maybe preventing lock-in. But with programming tools reaching maturity, we'll see more of this. Also there's an AI-related aspect: bots can understand structure just fine.
- taeric 2y ago
  There are good reasons it hasn't been done more, and I don't think it is just "text is good for troubleshooting." At large, text wins because the job of a program text is to communicate the program to a user in a way that a computer can also be made to use. Things that help include that text can have multiple interpretations and meanings. This is why most variable names are nouns. You can literally "ascii art" parts of code to help convey some ideas. The most important part of why "text" wins, though, is that it is all visible. Yes it implies some structure underneath that the computer is going to need. The text, though, is fully visible to users. People think they want the ability to "structurally edit," and you can sometimes make that work rather nicely. Most of the time, though, you are working with structures that are not complete. And worse, completing them will not be a process that goes in the same direction as the text. You will have to start a stack in how you are processing things to complete as you go. I'm torn on the bots comment, as I just don't know. LLMs, arguably, are leaning in on the structure that has been embedded in our symbolic text over many years. The same structure has not necessarily been developed in computer programming texts? (Also worth paying attention to how most of these goals were pursued with symbolic programming techniques back in the day. LISP, of course, but also term rewriting techniques that used to be rather magical.)
  - pjc50 2y ago
    > Most of the time, though, you are working with structures that are not complete. And worse, completing them will not be a process that goes in the same direction as the text. You will have to start a stack in how you are processing things to complete as you go. This is very insightful. A program that is being edited passes through all sorts of incomplete states. Non-structured editors will let you do things like paste mismatched braces .. which sometimes you definitely deliberately want to do! Edit: someone else makes this point at greater length https://news.ycombinator.com/item?id=42612969 https://news.ycombinator.com/item?id=42612969
rzwitserloot 2y ago
I've been toying with the idea of making something very very much lik this, but, for an existing language. Which makes it far, far more complicated; I'm sure various aspects of the design of whatever language one would build this for are needlessly convoluted due to not having been designed with this kind of editing in mind. A few concerns: * As developers, our 'world' is so defined by text, we textify things that aren't: We write marked up documentation in plain text (HTML, markdown, etc) and we despise using tools like LibreOffice to write that sort of thing, and we even 'draw' helpful graphs in `.dot` files. We do this for a good reason: It 'works' with our other tools. git itself, and the cavalcade of git-understanding frontend version control systems, are excellent at showing you diffs between textual representations. They don't do 'diffs between these 2 png files' or 'diffs between these two .docx files' well or at all. Hence, these projects either balloon, or need a pragmatic take. Either you staple 'also reinvent git' onto the project (and in passing you are increasing the hurdle to adopt or try out this stuff for any interested folk), or you think about how to adapt the idea to fit existing practices. In this specific case the latter seems like the obvious answer: The way to store the program is as a file that, if opened in a text editor, looks just like it. But, I _can_ edit files in git and it looks like a text file even if it really isn't; so what happens if I edit an eyg file with a text editor? How 'perfect' must it be? What happens if in hand-editing the file I added some whitespace that shouldn't have been there as per the canonical storage rule? These are all easily solvable details, of course. But it's where my brain went. I'm pretty sure the correct answer is 'the source file is still text and is still readable; it's just that the language has some extreme style requirements (think about a language that has no line continuation facilities whatsoever, long lines will just have to be long lines, and dictates exactly how many tabs you can insert; indenting with spaces is not allowed at all, that sort of thing). This is fine because the language is meant to be read as plain text but only edited with the structural editor. * I often try to think how it would be if I was doing my programming work with a structural editor. From time to time I do things that seem blatantly annoying if I only had a structural editor. Where I really _want_ to dip into syntactically utterly invalid gobbledygook because I'm doing some text edits which will get back to valid code but I go through invalid code. For example, block-based copy/pasting (block = multiselect/multicursor). I'm having a hard time seeing how that would work in a structural editor but I use it from time to time. It feels like a structural editor can hopefully come up with alternatives for all the use cases but I'm not sure. Should there instead be a 'fallback' option where your source file turns into raw text mode and your edit functionality is like any plain jane text editor, you do your text-based transformations and then when done exit that mode, at which point the structural editor reparses what you have. I think that's right but it means the language does, actually, need all the bells and whistles after all: A Parser, an AST, good error handling for syntax errors, and so forth.
hokkos 2y ago
I remember coding on my casio scientific calculator, you could only input full keyword by selecting from a menu, it was quick and you couldn't make syntax errors.
- nxobject 2y ago
  Perhaps it was tokenising commands to save memory. Did your calculator have keystroke programming/some form of programming?
- zabzonk 2y ago
  The Sinclair Spectrum Z80-based home computer had a keyboard that had all its BASIC keywords on the keyboard. I can't say I liked it, particularly as the keyboard was a very horrible rubber affair.
  - anonzzzies 2y ago
    People are now recreating them specifically to have the same feel out of nostalgia. I actually liked them as a kid; it was so easy to write impressive (for the time) graphical programs with BASIC without remembering all these pesky english words (I am not from an English speaking country).
unshavedyak 2y ago
Awesome! Glad to see more exploration in this space. I too am exploring this, though i am perhaps making it yet even more complicated. I want to represent relationships between AST nodes in a visual way. Eg, i love Node editors in Blender and if it was given a good keyboard based UX i could imagine editing structural code in it. I still imagine the code would be visually text, but i'm imagining that you'd be able to see a series of nodes in a code flow graph for your current editing context. I figure AST editing (to some degree) is a generally good thing, but also representing code flow somehow is also a desired thing. It's what i do in my head, so can it be visualized in a beneficial way? Difficult to say. Keyboard first will also be a challenge as i'll need to have a lot of intelligent alignment and visual compression. Thanks for the post! Love to see work in this space.
- unshavedyak 2y ago
  Oh, related in this space: https://mbuffett.com/posts/structured-editing-syntax/ https://mbuffett.com/posts/structured-editing-syntax/ I thought this blog post had an interesting idea too. Basically that a structural editor which truly knew the underlying AST could then alter the visual representation of the code entirely. Eg changing it to look like Python, or whatever insane thing. Not advocating for that of course, but the idea that the editor could be effectively transpiling the code in real time is an interesting area to explore too. I imagine not for the purpose of "Write Python that becomes Rust" or w/e, but just in general customizing the experience. Perhaps reducing clutter, /shrug
- Slackwise 2y ago
  > Glad to see more exploration in this space. I mean.... ParEdit [1] and ParInfer [2] have been around for a long time now. Structural editing is basically ancient. Lispers have been doing this for a long while, `slurp`ing and `barf`ing their parens. [1]: https://paredit.org/ https://paredit.org/ [2]: https://shaunlebron.github.io/parinfer/ https://shaunlebron.github.io/parinfer/
  - jerf 2y ago
    I have no problem with anyone jousting with the windmills; I have a problem when they don't realize they are windmills. Hundreds and probably thousands of programmers have taken a crack at this problem. That doesn't prove it's a bad idea but it is effective proof until shown otherwise that if it is a good idea it is not such a good idea that it can instantly catch up to the decades of work in text editing. And the way some people talk about it, it's clear that a lot of programmers think that it should be that good. This is up there with "everything would be solved with visual programming" and "hey, what if we could put down different syntax on top of the ASTs?". If your mental model is that it must just be because nobody has put any effort into the question, update your model, because in fact tons of effort has been expended and I'd suggest anyone interested in adding to the pile and actually solving the problems (as opposed to playing about, anyone can play with anything they like) is best served by examining the large pile of previous efforts and figuring out what they will do better than before. Creating a language designed for being manipulated this way is at least a bit more rare than just declaring structural editing is the way forward in a general sense. Still, I'm skeptical. The existence of a textual serialization of programming language concepts is not the problem, nor is the usage of said serialization. Every serious tool already immediately deserializes the text into an AST, and when you really get down to the nitty-gritty of how one represents these things in memory, it is not at all clear that there is necessary a "better" way to serialize these structures, or one that is really very different in the ways that matter. There's a number of well-known (by those who look) pitfalls built into the idea of structural editing and it is not clear at all that the fundamental disadvantages can be overcome. Maybe they can. But the evidence at this point is effectively proof it's going to take more than waving "structural editing!" and some excitement at the problem.
penteract 2y ago
Having made my own forays into making a structural editor, I'd like to sing the praises of plain text: storing something that has been created by pressing buttons at a keyboard as something very close to the sequence of buttons that have been pressed is a great idea. The magical ability of plain text is to go back in time and ask "what if I pressed a different button a minute ago/yesterday/last year?". This lets you see and fix mistaken keypresses, and even shapes the kind of mistakes you can make in a moment of confusion into ones that are quickly fixable. The direct representation of buttons pressed also really helps with teaching/learning: With plain text, for mechanical questions on the level of "What buttons do I press to make a program that looks like that?", the answer is obvious for anyone who knows how to type. Try the editor discussed in the article to appreciate how different answering that question can be in a structural editor. Seeing and explaining how someone got where they did is also straightforward ("you missed a semicolon", "you misspelt a variable name"). Ideally, those are problems that don't happen in a structural editor, but people who press a slightly wrong sequence of buttons will probably still exist. These arguments get much weaker when you aren't using a physical keyboard; I think this is the reason the article starts by talking about writing programs on a phone and a TV. In those environments structural editing has much more to offer.
- WillAdams 2y ago
  I've tried to use LEO (Literate Editor with Outlines) a couple of times, but have never found it comfortable enough to stick with. That said, the concept of Literate Programming has been one which resonated with me, and which addresses some of the same problems in a fashion which matches the way I work.
daveloyall 2y ago
Since this is a post about a programming tool, I expected links to repositories. I like to pick up a tool and turn it around in my hands before I decide if it is a good tool or not.
danielvaughn 2y ago
I’m obsessed with structural editors because it feels intuitively like the right approach to creating software. However I’ve yet to come across an interface that really clicks. Still love it though. Right now I’m building a structured editor for creating UIs, so can attest to how difficult it is to get it right. It’s a super interesting problem space.
- crowdhailer 2y ago
  I'd really like to get to the point where I have a library of transformations and you can make your own interface. Already the click ui and hot keys ui call the same `insert_function` or `assign_to` functions. Adding a ui with lots of dragging or voice control should be trivial
  - danielvaughn 2y ago
    I’ve actually moved away from GUI controls and towards something resembling a keyboard command language. Similar in spirit to vim motions, but instead of applied to text, it’s applied to an AST. I think that could be the key to making structured editing feel less “clunky”.
    - plagiarist 2y ago
      That's interesting! Do you have any of it in a public repo?
      - danielvaughn 2y ago
        I have an older repo that briefly describes it at a high level: https://github.com/matry/editor https://github.com/matry/editor I've moved on to another (private) repo, but it should be shareable soon.
        plagiarist 2y ago
        That's a cool idea. I think the arrow directions you chose for sibling and parent/child directions are very intuitive.
xigency 2y ago
This is a topic I've been contemplating heavily and it's very instructive to see the iterative progress you've made here. I think development tools like this are necessary to expand coding outside of just the desktop experience. For example, I've seen kids program some pretty impressive robotics with just a touchscreen and Blockly. Also, from a compilers perspective, it feels almost absurd to rely on plaintext. Imagine how much harder code merging is using line diff compared to diffing syntax trees. Thanks for sharing!
- crowdhailer 2y ago
  Diffing is really an interesting point. Git is quite (maybe very) good. I wonder how much better a structured approach could be but it's obviously a large amount of work that I don't yet have time for.
  - kmoser 2y ago
    A structured approach would be able to easily do things like ignore when a symbol was renamed (since that wouldn't affect the tree structure). So there are good reasons for wanting to do it that way, vs. a line-based diff.
  - aiono 2y ago
    Plain Text diffing has some obvious drawbacks: 1. If you rearrange your functions you will see a lot of additions and deletions while semantically there is no change in the program. It's just noise. 2. If you rename a variable you don't really have any actual change in places where it is referenced but text diff will again show a lot of noise. But the code references is still the same code.
    - 1718627440 2y ago
      At least to me whitespace does convey meaning and order of functions or variable renaming definitely conveys a semantic change.
      - 2y ago
cbm-vic-20 2y ago
This is neat. I wonder if this could be generalized to other languages with Tree-Sitter and LSPs. It could be helpful for programmers with accessibilty issues, among many other use cases.
- crowdhailer 2y ago
  It could be in principle. EYG was developed along with the editor so that has effected the design of the AST. Ideally if a useful/powerful/pleasant/intuitive structured editing experience was found for EYG (and it's minimal AST) then it would be proven to work and so putting the effort into doing for a program with a more complex AST would be worth the effort.
crabbone 2y ago
> Unlike traditional text-based editors, these tools treat code as a structured tree rather than a flat sequence of characters. This is such an obvious lie... Of course, even the disastrously bad editors like VSCode or IntelliJ tools understand code to have structure and allow interaction with that structure. Hell, even text editors not intended to deal with code will usually be able to deal with some structure, like identifying words, lines, perhaps even paragraphs and operating on them. > No syntax errors. Zero time is spent on missing quotes, semi-colons or other details. Usually paid for by inability to copy and paste chunks of code... not sure if it's worth the price. > Better type information. Because the program is always in a valid state the type checker can always run and give meaningful feedback. Why would you want to run typechecker all the time? What's the benefit? I run it when I'm interested in the results... otherwise it's just noise. By the time I'm interested in results, I'd get code in a decent enough shape to compile, so it won't be a problem. Other known downsides of this approach: having to access the code in places where the editor isn't available, s.a. reading diffs, debugging, running third-party tools unaware of structured format (which, given how our programming practices have developed, will be inconsistent between versions and platforms). All in all, I've seen these attempts multiple times before, and it always looks like the benefits simply aren't worth the effort. Or, maybe, too many stars need to align in order for this to actually become really useful in the day-to-day programming work, and so it doesn't happen because of some circular dependency where the tools won't be built, until the product is good enough to use, but the product is never good enough to use because there aren't any tools?
- lsaferite 2y ago
  > disastrously bad editors like VSCode or IntelliJ tools I'm curious, what code editor(s) do you hold in high esteem?
  - crabbone 2y ago
    Editors that are geared towards advanced users, i.e. they allow mostly or completely keyboard-driven interaction (to increase precision of actions), editors that are rich in the way that they understand and navigate text, i.e. they offer multiple paradigms of text navigation, be it by structure, by line, by mini-map, the "jedi" style. Editors that allow substantial automation and extensibility without a massive effort on the user end, possible ad hoc, rather than requiring a full-fledged programming project. Editors with multiple existing integrations with third parties, but also allowing for a relatively easy addition of such interactions. Editors that aren't overly reliant on third-party software to function (i.e. can function when X-Server isn't available, or when Web isn't available). Some anti-features that are preferably be missing from good editors: unnecessarily constraining user to the choices the editor designer or the editor administrator deemed necessary. Excluding particular functionality from automation or requiring substantial workarounds in order to automate particular aspects (eg. having functionality associated with a button that can only be clicked by positioning the mouse cursor over it, or, even worse, functionality that is only available through drag-and-drop). Reliance on modal windows. Automatic editing and formatting of editor-related configuration files in a way that doesn't preserve user's edits and formatting. In other words, it's possible to take a decent editor, like Emacs or Vim and turn it into a bad editor like VSCode or IDEA by doing bad things to it because the base is very extensible. But, in general, excluding the administrative / repackaging component, editors like Emacs or Vim will win in this contest.
cfiggers 2y ago
I enjoy Lisp and Lisp-alike languages specifically because with the right editor support you can do a lot of this kind of structural editing stuff without giving up the niceties of source code as text.
- mtreis86 2y ago
  Yeah in emacs with paredit/smartparens/parinfer or whatever, it is both text and structure. With parinfer you change the text and the structure gets modified to match. With the other two you can change the structure directly with a command that has a silly but descriptive name like slurp or barf.
- 0x1ceb00da 2y ago
  You don't need s-expressions for that. Jetbrains IDEs provide this functionality for all supported languages, even bash.
  - nyir 2y ago
    While they're very good compared to the alternatives, I wonder why they haven't pushed more into that direction, e.g. what I've wanted to put into a plugin when tinkering with the PSI was simple composable operations, similar to Vim operations, like "rotate if/else branch" (while correctly negating and simplifying the test expression), or "convert if to case/switch/..." (and back and only if it's feasible).
- klibertp 2y ago
  You don't need s-exps (Lisp) for this; it turns out you just need a good parsing framework. Here's the proof: https://www.masteringemacs.org/article/combobulate-structured-movement-editing-treesitter https://www.masteringemacs.org/article/combobulate-structure...
perlcommunity 2y ago
https://www.jgrasp.org/ https://www.jgrasp.org/
tcoff91 2y ago
Haven't lisp coders been doing this forever with emacs?
- _19qg 2y ago
  see for example a structure editor for Lisp, ca. 1967, described by L Peter Deutsch: https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=028a167a1c72aa817d9d5b8aa6e9099a75bd454f https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&d...
EdiX 2y ago
I used a structural text editor a while back (technically a structural text editing mode for emacs). I went in completely buying the structural text editing propaganda (it's more logical, it will make you more efficient, it's the future...) but I eventually found the experience so excruciatingly bad and stupid that I now think the whole endeavour is a complete waste of time, the idea sounds good on paper but there are some fundamental unsolvable problems with it. The problem is that, when you want to make a change to piece of source code you know its current state and you know how you want it to look like at the end. With a normal text editor you will simply make a bunch of changes that turn the text as it into what it should be, it will normally go through a series of invalid states but you don't have to think about what those states will be or care about them. With a structural text editor this is impossible, invalid states can not be represented so when you want to make a change to your source code you don't just care about the beginning and end state, you also want to find a series of valid states that takes you from the beginning to the end where each one can be reached from the previous one with an editor command. For example, let's say you have a function call: (afunc 1 2 3) and you want to turn it into a conditional function call based on a flag: (if flag (afunc 1 2 3)) with a normal text editor you: 1. type "(if flag " before the call 2. type ")" after the call It doesn't matter the order in which I do these operations or how I move the cursor. With a structural text editor, I have to: 1. create a new form before the function 2. type "if flag" 3. select the entire function call 4. cut it 5. paste it as the first branch of the if This is however only possible if the context where the function call currently is allows adding a new node before it (i.e. it's inside the equivalent of a lisp progn). If not I'd have to use a different strategy, for example: 1. select the entire function call 2. use a command to create a node around it 3. type "if flag" I did this for a few months, eventually I realized that I didn't want to think about the syntactical validity of the intermediate states my program goes through while I'm editing it, because they exist for mere seconds, and I didn't want to memorize the large palette of commands necessary to create those intermediate states. It was all additional cognitive load that accomplished nothing but satisfy someone's esthetic sense that source code should always be in a syntactically valid state. So of the benefits listed in the article: 1. "No syntax errors" is actually a major detriment of the structural editing model 2. "Better type information" is a lie, all that the structural text editor guarantees you is that the program is syntactically valid, not that it will pass type checking 3. "No keywords" is a lie, it's only true if you use a new programming language and don't store it on disk as text which opens a whole other can of worms (can't even read source code without a special text editor, won't work with github, won't work with any versioning system, won't work with grep, can't copy paste a chunk of it in an email...) 4. "Rich visualization" is an entirely different story from structural text editing but as far as I know nobody has ever shown visual programming to actually provide any kind of tangible benefit 5. "Reduced complexity" is the one I give them, yes if you are making a structured text editor you don't have to write a fault tolerant parser, it does make your (aka the text editor's implementor) life easier Fans of structural text editors usually mention that they will unlock "more powerful refactoring tools" but actually all you need for those to be "unlocked" is a dumb parser and an automatic code formatter, like "go fmt". Refactoring a function to be inlined into another is still a difficult operation to implement correctly but it isn't like the difficult part is running a parser on the source code, it's dealing with variable shadowing.
herrington_d 2y ago
> Better type information. Because the program is always in a valid state the type checker can always run and give meaningful feedback. Note that programs can be syntactically well-formed but ill-typed. For example `let x = true + 1` has valid syntax but produces an "undefined" type for the variable `x`, if the type system does not support type error recovery. A quote from the great paper from POPL 2024, https://hazel.org/papers/marking-popl24.pdf https://hazel.org/papers/marking-popl24.pdf > If a type error appears _anywhere_, the program is formally meaningless _everywhere_
- crowdhailer 2y ago
  The EYG type system does support recovery, so you will get multiple type errors if that's the case in the program.
  - herrington_d 2y ago
    My point is that type error recovery is the property of the type system. Not the property of structural code editor.
like_any_other 2y ago
For structural code editors to thrive, what is needed is a universal, simple, minimal, txt-like format for the AST, so that AST-editors are not locked to a single programming language each.
- crowdhailer 2y ago
  What requires the format to be text-like? I'm not sure I agree with this assertion. I think structural editors need to be language specific so that they can offer higher value edits that might only make sense for specific languages.
  - like_any_other 2y ago
    "Text-like" meaning easy to parse and universally understood. With language-specific (or indeed, all) AST/generic tree attributes serialized in a uniform way that does not entirely break parsing if an editor does not understand one of them. I.e. I can edit code even if my editor doesn't understand the language and doesn't give me syntax highlighting and completion. More, that that 'understanding' can be factored out of the editor core, as is done with language servers today.
- zellyn 2y ago
  You might enjoy reading up on EYG and/or listening to podcast appearances of the creator…
  - crowdhailer 2y ago
    Indeed. Here's a link to one episode https://pod.link/developer-voices/episode/e47720e8c6dc3d16603b53081bde0cb1 https://pod.link/developer-voices/episode/e47720e8c6dc3d1660...
  - WillAdams 2y ago
    Do you mean: https://eyg.run/ https://eyg.run/ Is this typo on the web page or in the language? >type missmatch given: Integer expected: String or is this an intentional phrasing to match the following text?
andrewl 2y ago
I’m always glad to see new work on tools, and amplifications to our capabilities of any sort. I played with Smalltalk, and later Pharo, a bit, but it was years ago. As I recall the editor was always aware of the state of the system. And I think it was at a level beyond what any current IDE can do. While I do not remember if it allowed you to enter invalid code, it was definitely an interesting approach. So I will be following Eyg for sure.
- xkriva11 2y ago
  In Pharo, the method needs to be syntactically valid to be saved.
  - igouy 2y ago
    Generally, in Smalltalk, ...
otikik 2y ago
I think this is the way, but tying it to a new language will not help making this initiative popuplar at all. It would need to be tied to a more popular existing language such as Python or Javascript.
dominicrose 2y ago
For some reason this reminds me of semantic highlighting. A constant is not just an identifier - like a variable - it's a constant. There's more semantic in the text than the usual syntax highlighters highlight. Some code formatters are good and exist for json or edn (extensible data notation) or even full programming languages like smalltalk or even more complex ones like typescript. In a team it's true that code can be chaotic because not everybody uses a formatter and obviously we all have different coding style, even if the tooling made it more structured. So yes it would help to enforce more structure in a seamless way so we can focus on the important stuff without creating huge diffs in pull requests.
marhee 2y ago
A mature (and very effective) structural editor is the FLUID user-interface designer, part of the FLTK UI framework (C++). It outputs C++ and works very well. I remember dismissing it as a toy back when starting to work professionally with FLTK. Soon I realised I was mistaken; it is excellent to create UI's with for C++/FLKT.
codery 2y ago
https://github.com/disconcision/fructure https://github.com/disconcision/fructure https://www.youtube.com/watch?v=CnbVCNIh1NA https://www.youtube.com/watch?v=CnbVCNIh1NA
eternityforest 2y ago
I was able to use this without reading the manual, so I suppose it passes my favorite test of UI! But I do wish it was more clear what you have selected, and what things are selectable.

242 points by crowdhailer 2y ago

I have wanted something like this for so long that I can’t remember when I started. Strongly coupling source code to plain text is a lost opportunity to improve tooling.

jtwaleson 2y ago
Same! I think we need to free "code" from files, characters, tabs/spaces and variable names. Put millions of ASTs in a database. Imagine how much cheaper and better AI code assistants can be if they can only generate valid code with their "next token".
- zokier 2y ago
  At some point the AST needs to be serialized as bytes on disk. In the big picture, the cost of having that serialization format be also human readable is relatively small as long as the syntax is designed to be sane (i.e. not c++/perl)
- norman784 2y ago
  > Put millions of ASTs in a database. That's basically what unison does, they store the source code in binary format in an sqlite database.
School-Cotton 2y ago
The emacs mode paredit is basically structural editing for lisp code (and some of its more modern competitors like Smartparens support other languages too, but not as well as Lisp). It's great.
norir 2y ago
I disagree that plain text is an obstacle towards better tooling. You can do most of what the author describes with direct plain text edits pipelined into an analysis program. But I also think that the ux the author presents is undesirable. When writing new code, one must pass through a phase of broken code to reach the working code. I would not want an editor blocking me from typing a word that isn't yet defined but I do want it to notify me instantly that there is a problem. I also struggled to understand the use case for the mouse based ux. I am typing this comment on an iphone mini and would much rather write raw text on its keyboard than use the clicking interface shown in the fibonacci example.
vidarh 2y ago
The problem is communicating about code. Almost every visual / structural programming language attempts strands on how you talk about the code, and e-mail people about the code, and write about the code, or even how you efficiently input it. It's a multi-decade-long problem with many attempts.

datadrivenangel 2y ago

In the data science space, people have been going back and forth on the benefits of Notebooks, which seem very similar, and were historically stored as json under the hood. There are many many benefits to plain text.

jtwaleson 2y ago
I don't think notebooks are very similar to be honest. The notebooks still emit code (typically python) which is then interpreted as usual.

norir 2y ago

I think structural editors are a great feature to add on top of a language with a definite textual representation. If you can pretty print the ast (as the fib example shows), you should generally be able to parse the output to generate the structured output.

mebazaa 2y ago

My first thought: maybe competitive programmers could find it useful, if it’s a way to reduce the amount of key presses needed to write their code? (At the very least, that’s a crowd that would be motivated enough to get accustomed to that way of writing code.)

deleted 2y ago
[deleted]
poutsku 2y ago
Every year when I participate in advent of code, I have this thought: "wouldn't it be great to solve these problems directly from my phone?" (The puzzles unlock quite early morning in my timezone and opening my laptop at that time somehow annoys me.) I have wondered many times if some structural editor with a bunch of high-level library methods (Dijkstra's, for instance) would actually give a good/fast experience for most days. I haven't had the time or interest yet to try and build something yet. But perhaps I should.
- crowdhailer 2y ago
  I'd love to get this to the point where add your own keybindings to structural edits, potentially very involved ones. It's superficially not very different to a combination of vim macros, auto completes or templates in other editors but I think the end result could be interesting as the units of change would be more meaningful. I think it's the same benefit of a hygienic macro system vs a source file manipulating version of meta programming.

lincpa 2y ago

[dead]

criddell 2y ago

Is it far enough along where you could work on the code editor in the code editor?

bennythomsson 2y ago
As the videos show, there are still a few usability issues. Full-screen switching somewhere else just to enter a name for example doesn't get you very far. But there is not much missing and I like the POC!
crowdhailer 2y ago
The code editor is for the EYG language and implemented in Gleam. So that's a blocker. The editor is probably far enough along to have a crack at it by EYG would need some UI or web library so that EYG was able to build an editor.

narag 2y ago

This is the way. Programs are not text, there's an impedance mismatch that shows at the seams. There're good reasons it hasn't been done more: text is good for troubleshooting, "every program should do one thing well" and maybe preventing lock-in. But with programming tools reaching maturity, we'll see more of this. Also there's an AI-related aspect: bots can understand structure just fine.

taeric 2y ago
There are good reasons it hasn't been done more, and I don't think it is just "text is good for troubleshooting." At large, text wins because the job of a program text is to communicate the program to a user in a way that a computer can also be made to use. Things that help include that text can have multiple interpretations and meanings. This is why most variable names are nouns. You can literally "ascii art" parts of code to help convey some ideas. The most important part of why "text" wins, though, is that it is all visible. Yes it implies some structure underneath that the computer is going to need. The text, though, is fully visible to users. People think they want the ability to "structurally edit," and you can sometimes make that work rather nicely. Most of the time, though, you are working with structures that are not complete. And worse, completing them will not be a process that goes in the same direction as the text. You will have to start a stack in how you are processing things to complete as you go. I'm torn on the bots comment, as I just don't know. LLMs, arguably, are leaning in on the structure that has been embedded in our symbolic text over many years. The same structure has not necessarily been developed in computer programming texts? (Also worth paying attention to how most of these goals were pursued with symbolic programming techniques back in the day. LISP, of course, but also term rewriting techniques that used to be rather magical.)
- pjc50 2y ago
  > Most of the time, though, you are working with structures that are not complete. And worse, completing them will not be a process that goes in the same direction as the text. You will have to start a stack in how you are processing things to complete as you go. This is very insightful. A program that is being edited passes through all sorts of incomplete states. Non-structured editors will let you do things like paste mismatched braces .. which sometimes you definitely deliberately want to do! Edit: someone else makes this point at greater length https://news.ycombinator.com/item?id=42612969 https://news.ycombinator.com/item?id=42612969

rzwitserloot 2y ago

I've been toying with the idea of making something very very much lik this, but, for an existing language. Which makes it far, far more complicated; I'm sure various aspects of the design of whatever language one would build this for are needlessly convoluted due to not having been designed with this kind of editing in mind. A few concerns: * As developers, our 'world' is so defined by text, we textify things that aren't: We write marked up documentation in plain text (HTML, markdown, etc) and we despise using tools like LibreOffice to write that sort of thing, and we even 'draw' helpful graphs in `.dot` files. We do this for a good reason: It 'works' with our other tools. git itself, and the cavalcade of git-understanding frontend version control systems, are excellent at showing you diffs between textual representations. They don't do 'diffs between these 2 png files' or 'diffs between these two .docx files' well or at all. Hence, these projects either balloon, or need a pragmatic take. Either you staple 'also reinvent git' onto the project (and in passing you are increasing the hurdle to adopt or try out this stuff for any interested folk), or you think about how to adapt the idea to fit existing practices. In this specific case the latter seems like the obvious answer: The way to store the program is as a file that, if opened in a text editor, looks just like it. But, I _can_ edit files in git and it looks like a text file even if it really isn't; so what happens if I edit an eyg file with a text editor? How 'perfect' must it be? What happens if in hand-editing the file I added some whitespace that shouldn't have been there as per the canonical storage rule? These are all easily solvable details, of course. But it's where my brain went. I'm pretty sure the correct answer is 'the source file is still text and is still readable; it's just that the language has some extreme style requirements (think about a language that has no line continuation facilities whatsoever, long lines will just have to be long lines, and dictates exactly how many tabs you can insert; indenting with spaces is not allowed at all, that sort of thing). This is fine because the language is meant to be read as plain text but only edited with the structural editor. * I often try to think how it would be if I was doing my programming work with a structural editor. From time to time I do things that seem blatantly annoying if I only had a structural editor. Where I really _want_ to dip into syntactically utterly invalid gobbledygook because I'm doing some text edits which will get back to valid code but I go through invalid code. For example, block-based copy/pasting (block = multiselect/multicursor). I'm having a hard time seeing how that would work in a structural editor but I use it from time to time. It feels like a structural editor can hopefully come up with alternatives for all the use cases but I'm not sure. Should there instead be a 'fallback' option where your source file turns into raw text mode and your edit functionality is like any plain jane text editor, you do your text-based transformations and then when done exit that mode, at which point the structural editor reparses what you have. I think that's right but it means the language does, actually, need all the bells and whistles after all: A Parser, an AST, good error handling for syntax errors, and so forth.

hokkos 2y ago

I remember coding on my casio scientific calculator, you could only input full keyword by selecting from a menu, it was quick and you couldn't make syntax errors.

nxobject 2y ago
Perhaps it was tokenising commands to save memory. Did your calculator have keystroke programming/some form of programming?
zabzonk 2y ago
The Sinclair Spectrum Z80-based home computer had a keyboard that had all its BASIC keywords on the keyboard. I can't say I liked it, particularly as the keyboard was a very horrible rubber affair.
- anonzzzies 2y ago
  People are now recreating them specifically to have the same feel out of nostalgia. I actually liked them as a kid; it was so easy to write impressive (for the time) graphical programs with BASIC without remembering all these pesky english words (I am not from an English speaking country).

unshavedyak 2y ago

Awesome! Glad to see more exploration in this space. I too am exploring this, though i am perhaps making it yet even more complicated. I want to represent relationships between AST nodes in a visual way. Eg, i love Node editors in Blender and if it was given a good keyboard based UX i could imagine editing structural code in it. I still imagine the code would be visually text, but i'm imagining that you'd be able to see a series of nodes in a code flow graph for your current editing context. I figure AST editing (to some degree) is a generally good thing, but also representing code flow somehow is also a desired thing. It's what i do in my head, so can it be visualized in a beneficial way? Difficult to say. Keyboard first will also be a challenge as i'll need to have a lot of intelligent alignment and visual compression. Thanks for the post! Love to see work in this space.

unshavedyak 2y ago
Oh, related in this space: https://mbuffett.com/posts/structured-editing-syntax/ https://mbuffett.com/posts/structured-editing-syntax/ I thought this blog post had an interesting idea too. Basically that a structural editor which truly knew the underlying AST could then alter the visual representation of the code entirely. Eg changing it to look like Python, or whatever insane thing. Not advocating for that of course, but the idea that the editor could be effectively transpiling the code in real time is an interesting area to explore too. I imagine not for the purpose of "Write Python that becomes Rust" or w/e, but just in general customizing the experience. Perhaps reducing clutter, /shrug
Slackwise 2y ago
> Glad to see more exploration in this space. I mean.... ParEdit [1] and ParInfer [2] have been around for a long time now. Structural editing is basically ancient. Lispers have been doing this for a long while, `slurp`ing and `barf`ing their parens. [1]: https://paredit.org/ https://paredit.org/ [2]: https://shaunlebron.github.io/parinfer/ https://shaunlebron.github.io/parinfer/
- jerf 2y ago
  I have no problem with anyone jousting with the windmills; I have a problem when they don't realize they are windmills. Hundreds and probably thousands of programmers have taken a crack at this problem. That doesn't prove it's a bad idea but it is effective proof until shown otherwise that if it is a good idea it is not such a good idea that it can instantly catch up to the decades of work in text editing. And the way some people talk about it, it's clear that a lot of programmers think that it should be that good. This is up there with "everything would be solved with visual programming" and "hey, what if we could put down different syntax on top of the ASTs?". If your mental model is that it must just be because nobody has put any effort into the question, update your model, because in fact tons of effort has been expended and I'd suggest anyone interested in adding to the pile and actually solving the problems (as opposed to playing about, anyone can play with anything they like) is best served by examining the large pile of previous efforts and figuring out what they will do better than before. Creating a language designed for being manipulated this way is at least a bit more rare than just declaring structural editing is the way forward in a general sense. Still, I'm skeptical. The existence of a textual serialization of programming language concepts is not the problem, nor is the usage of said serialization. Every serious tool already immediately deserializes the text into an AST, and when you really get down to the nitty-gritty of how one represents these things in memory, it is not at all clear that there is necessary a "better" way to serialize these structures, or one that is really very different in the ways that matter. There's a number of well-known (by those who look) pitfalls built into the idea of structural editing and it is not clear at all that the fundamental disadvantages can be overcome. Maybe they can. But the evidence at this point is effectively proof it's going to take more than waving "structural editing!" and some excitement at the problem.

penteract 2y ago

Having made my own forays into making a structural editor, I'd like to sing the praises of plain text: storing something that has been created by pressing buttons at a keyboard as something very close to the sequence of buttons that have been pressed is a great idea. The magical ability of plain text is to go back in time and ask "what if I pressed a different button a minute ago/yesterday/last year?". This lets you see and fix mistaken keypresses, and even shapes the kind of mistakes you can make in a moment of confusion into ones that are quickly fixable. The direct representation of buttons pressed also really helps with teaching/learning: With plain text, for mechanical questions on the level of "What buttons do I press to make a program that looks like that?", the answer is obvious for anyone who knows how to type. Try the editor discussed in the article to appreciate how different answering that question can be in a structural editor. Seeing and explaining how someone got where they did is also straightforward ("you missed a semicolon", "you misspelt a variable name"). Ideally, those are problems that don't happen in a structural editor, but people who press a slightly wrong sequence of buttons will probably still exist. These arguments get much weaker when you aren't using a physical keyboard; I think this is the reason the article starts by talking about writing programs on a phone and a TV. In those environments structural editing has much more to offer.

WillAdams 2y ago
I've tried to use LEO (Literate Editor with Outlines) a couple of times, but have never found it comfortable enough to stick with. That said, the concept of Literate Programming has been one which resonated with me, and which addresses some of the same problems in a fashion which matches the way I work.

daveloyall 2y ago

Since this is a post about a programming tool, I expected links to repositories. I like to pick up a tool and turn it around in my hands before I decide if it is a good tool or not.

danielvaughn 2y ago

I’m obsessed with structural editors because it feels intuitively like the right approach to creating software. However I’ve yet to come across an interface that really clicks. Still love it though. Right now I’m building a structured editor for creating UIs, so can attest to how difficult it is to get it right. It’s a super interesting problem space.

crowdhailer 2y ago
I'd really like to get to the point where I have a library of transformations and you can make your own interface. Already the click ui and hot keys ui call the same `insert_function` or `assign_to` functions. Adding a ui with lots of dragging or voice control should be trivial
- danielvaughn 2y ago
  I’ve actually moved away from GUI controls and towards something resembling a keyboard command language. Similar in spirit to vim motions, but instead of applied to text, it’s applied to an AST. I think that could be the key to making structured editing feel less “clunky”.
  - plagiarist 2y ago
    That's interesting! Do you have any of it in a public repo?
    - danielvaughn 2y ago
      I have an older repo that briefly describes it at a high level: https://github.com/matry/editor https://github.com/matry/editor I've moved on to another (private) repo, but it should be shareable soon.
      - plagiarist 2y ago
        That's a cool idea. I think the arrow directions you chose for sibling and parent/child directions are very intuitive.

xigency 2y ago

This is a topic I've been contemplating heavily and it's very instructive to see the iterative progress you've made here. I think development tools like this are necessary to expand coding outside of just the desktop experience. For example, I've seen kids program some pretty impressive robotics with just a touchscreen and Blockly. Also, from a compilers perspective, it feels almost absurd to rely on plaintext. Imagine how much harder code merging is using line diff compared to diffing syntax trees. Thanks for sharing!

crowdhailer 2y ago
Diffing is really an interesting point. Git is quite (maybe very) good. I wonder how much better a structured approach could be but it's obviously a large amount of work that I don't yet have time for.
- kmoser 2y ago
  A structured approach would be able to easily do things like ignore when a symbol was renamed (since that wouldn't affect the tree structure). So there are good reasons for wanting to do it that way, vs. a line-based diff.
- aiono 2y ago
  Plain Text diffing has some obvious drawbacks: 1. If you rearrange your functions you will see a lot of additions and deletions while semantically there is no change in the program. It's just noise. 2. If you rename a variable you don't really have any actual change in places where it is referenced but text diff will again show a lot of noise. But the code references is still the same code.
  - 1718627440 2y ago
    At least to me whitespace does convey meaning and order of functions or variable renaming definitely conveys a semantic change.
    - 2y ago

cbm-vic-20 2y ago

This is neat. I wonder if this could be generalized to other languages with Tree-Sitter and LSPs. It could be helpful for programmers with accessibilty issues, among many other use cases.

crowdhailer 2y ago
It could be in principle. EYG was developed along with the editor so that has effected the design of the AST. Ideally if a useful/powerful/pleasant/intuitive structured editing experience was found for EYG (and it's minimal AST) then it would be proven to work and so putting the effort into doing for a program with a more complex AST would be worth the effort.

crabbone 2y ago

> Unlike traditional text-based editors, these tools treat code as a structured tree rather than a flat sequence of characters. This is such an obvious lie... Of course, even the disastrously bad editors like VSCode or IntelliJ tools understand code to have structure and allow interaction with that structure. Hell, even text editors not intended to deal with code will usually be able to deal with some structure, like identifying words, lines, perhaps even paragraphs and operating on them. > No syntax errors. Zero time is spent on missing quotes, semi-colons or other details. Usually paid for by inability to copy and paste chunks of code... not sure if it's worth the price. > Better type information. Because the program is always in a valid state the type checker can always run and give meaningful feedback. Why would you want to run typechecker all the time? What's the benefit? I run it when I'm interested in the results... otherwise it's just noise. By the time I'm interested in results, I'd get code in a decent enough shape to compile, so it won't be a problem. Other known downsides of this approach: having to access the code in places where the editor isn't available, s.a. reading diffs, debugging, running third-party tools unaware of structured format (which, given how our programming practices have developed, will be inconsistent between versions and platforms). All in all, I've seen these attempts multiple times before, and it always looks like the benefits simply aren't worth the effort. Or, maybe, too many stars need to align in order for this to actually become really useful in the day-to-day programming work, and so it doesn't happen because of some circular dependency where the tools won't be built, until the product is good enough to use, but the product is never good enough to use because there aren't any tools?

lsaferite 2y ago
> disastrously bad editors like VSCode or IntelliJ tools I'm curious, what code editor(s) do you hold in high esteem?
- crabbone 2y ago
  Editors that are geared towards advanced users, i.e. they allow mostly or completely keyboard-driven interaction (to increase precision of actions), editors that are rich in the way that they understand and navigate text, i.e. they offer multiple paradigms of text navigation, be it by structure, by line, by mini-map, the "jedi" style. Editors that allow substantial automation and extensibility without a massive effort on the user end, possible ad hoc, rather than requiring a full-fledged programming project. Editors with multiple existing integrations with third parties, but also allowing for a relatively easy addition of such interactions. Editors that aren't overly reliant on third-party software to function (i.e. can function when X-Server isn't available, or when Web isn't available). Some anti-features that are preferably be missing from good editors: unnecessarily constraining user to the choices the editor designer or the editor administrator deemed necessary. Excluding particular functionality from automation or requiring substantial workarounds in order to automate particular aspects (eg. having functionality associated with a button that can only be clicked by positioning the mouse cursor over it, or, even worse, functionality that is only available through drag-and-drop). Reliance on modal windows. Automatic editing and formatting of editor-related configuration files in a way that doesn't preserve user's edits and formatting. In other words, it's possible to take a decent editor, like Emacs or Vim and turn it into a bad editor like VSCode or IDEA by doing bad things to it because the base is very extensible. But, in general, excluding the administrative / repackaging component, editors like Emacs or Vim will win in this contest.

cfiggers 2y ago

I enjoy Lisp and Lisp-alike languages specifically because with the right editor support you can do a lot of this kind of structural editing stuff without giving up the niceties of source code as text.

mtreis86 2y ago
Yeah in emacs with paredit/smartparens/parinfer or whatever, it is both text and structure. With parinfer you change the text and the structure gets modified to match. With the other two you can change the structure directly with a command that has a silly but descriptive name like slurp or barf.
0x1ceb00da 2y ago
You don't need s-expressions for that. Jetbrains IDEs provide this functionality for all supported languages, even bash.
- nyir 2y ago
  While they're very good compared to the alternatives, I wonder why they haven't pushed more into that direction, e.g. what I've wanted to put into a plugin when tinkering with the PSI was simple composable operations, similar to Vim operations, like "rotate if/else branch" (while correctly negating and simplifying the test expression), or "convert if to case/switch/..." (and back and only if it's feasible).
klibertp 2y ago
You don't need s-exps (Lisp) for this; it turns out you just need a good parsing framework. Here's the proof: https://www.masteringemacs.org/article/combobulate-structured-movement-editing-treesitter https://www.masteringemacs.org/article/combobulate-structure...

perlcommunity 2y ago

https://www.jgrasp.org/ https://www.jgrasp.org/

tcoff91 2y ago

Haven't lisp coders been doing this forever with emacs?

_19qg 2y ago
see for example a structure editor for Lisp, ca. 1967, described by L Peter Deutsch: https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=028a167a1c72aa817d9d5b8aa6e9099a75bd454f https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&d...

EdiX 2y ago

I used a structural text editor a while back (technically a structural text editing mode for emacs). I went in completely buying the structural text editing propaganda (it's more logical, it will make you more efficient, it's the future...) but I eventually found the experience so excruciatingly bad and stupid that I now think the whole endeavour is a complete waste of time, the idea sounds good on paper but there are some fundamental unsolvable problems with it. The problem is that, when you want to make a change to piece of source code you know its current state and you know how you want it to look like at the end. With a normal text editor you will simply make a bunch of changes that turn the text as it into what it should be, it will normally go through a series of invalid states but you don't have to think about what those states will be or care about them. With a structural text editor this is impossible, invalid states can not be represented so when you want to make a change to your source code you don't just care about the beginning and end state, you also want to find a series of valid states that takes you from the beginning to the end where each one can be reached from the previous one with an editor command. For example, let's say you have a function call: (afunc 1 2 3) and you want to turn it into a conditional function call based on a flag: (if flag (afunc 1 2 3)) with a normal text editor you: 1. type "(if flag " before the call 2. type ")" after the call It doesn't matter the order in which I do these operations or how I move the cursor. With a structural text editor, I have to: 1. create a new form before the function 2. type "if flag" 3. select the entire function call 4. cut it 5. paste it as the first branch of the if This is however only possible if the context where the function call currently is allows adding a new node before it (i.e. it's inside the equivalent of a lisp progn). If not I'd have to use a different strategy, for example: 1. select the entire function call 2. use a command to create a node around it 3. type "if flag" I did this for a few months, eventually I realized that I didn't want to think about the syntactical validity of the intermediate states my program goes through while I'm editing it, because they exist for mere seconds, and I didn't want to memorize the large palette of commands necessary to create those intermediate states. It was all additional cognitive load that accomplished nothing but satisfy someone's esthetic sense that source code should always be in a syntactically valid state. So of the benefits listed in the article: 1. "No syntax errors" is actually a major detriment of the structural editing model 2. "Better type information" is a lie, all that the structural text editor guarantees you is that the program is syntactically valid, not that it will pass type checking 3. "No keywords" is a lie, it's only true if you use a new programming language and don't store it on disk as text which opens a whole other can of worms (can't even read source code without a special text editor, won't work with github, won't work with any versioning system, won't work with grep, can't copy paste a chunk of it in an email...) 4. "Rich visualization" is an entirely different story from structural text editing but as far as I know nobody has ever shown visual programming to actually provide any kind of tangible benefit 5. "Reduced complexity" is the one I give them, yes if you are making a structured text editor you don't have to write a fault tolerant parser, it does make your (aka the text editor's implementor) life easier Fans of structural text editors usually mention that they will unlock "more powerful refactoring tools" but actually all you need for those to be "unlocked" is a dumb parser and an automatic code formatter, like "go fmt". Refactoring a function to be inlined into another is still a difficult operation to implement correctly but it isn't like the difficult part is running a parser on the source code, it's dealing with variable shadowing.

herrington_d 2y ago

> Better type information. Because the program is always in a valid state the type checker can always run and give meaningful feedback. Note that programs can be syntactically well-formed but ill-typed. For example `let x = true + 1` has valid syntax but produces an "undefined" type for the variable `x`, if the type system does not support type error recovery. A quote from the great paper from POPL 2024, https://hazel.org/papers/marking-popl24.pdf https://hazel.org/papers/marking-popl24.pdf > If a type error appears _anywhere_, the program is formally meaningless _everywhere_

crowdhailer 2y ago
The EYG type system does support recovery, so you will get multiple type errors if that's the case in the program.
- herrington_d 2y ago
  My point is that type error recovery is the property of the type system. Not the property of structural code editor.

like_any_other 2y ago

For structural code editors to thrive, what is needed is a universal, simple, minimal, txt-like format for the AST, so that AST-editors are not locked to a single programming language each.

crowdhailer 2y ago
What requires the format to be text-like? I'm not sure I agree with this assertion. I think structural editors need to be language specific so that they can offer higher value edits that might only make sense for specific languages.
- like_any_other 2y ago
  "Text-like" meaning easy to parse and universally understood. With language-specific (or indeed, all) AST/generic tree attributes serialized in a uniform way that does not entirely break parsing if an editor does not understand one of them. I.e. I can edit code even if my editor doesn't understand the language and doesn't give me syntax highlighting and completion. More, that that 'understanding' can be factored out of the editor core, as is done with language servers today.
zellyn 2y ago
You might enjoy reading up on EYG and/or listening to podcast appearances of the creator…
- crowdhailer 2y ago
  Indeed. Here's a link to one episode https://pod.link/developer-voices/episode/e47720e8c6dc3d16603b53081bde0cb1 https://pod.link/developer-voices/episode/e47720e8c6dc3d1660...
- WillAdams 2y ago
  Do you mean: https://eyg.run/ https://eyg.run/ Is this typo on the web page or in the language? >type missmatch given: Integer expected: String or is this an intentional phrasing to match the following text?

andrewl 2y ago

I’m always glad to see new work on tools, and amplifications to our capabilities of any sort. I played with Smalltalk, and later Pharo, a bit, but it was years ago. As I recall the editor was always aware of the state of the system. And I think it was at a level beyond what any current IDE can do. While I do not remember if it allowed you to enter invalid code, it was definitely an interesting approach. So I will be following Eyg for sure.

xkriva11 2y ago
In Pharo, the method needs to be syntactically valid to be saved.
- igouy 2y ago
  Generally, in Smalltalk, ...

otikik 2y ago

I think this is the way, but tying it to a new language will not help making this initiative popuplar at all. It would need to be tied to a more popular existing language such as Python or Javascript.

dominicrose 2y ago

For some reason this reminds me of semantic highlighting. A constant is not just an identifier - like a variable - it's a constant. There's more semantic in the text than the usual syntax highlighters highlight. Some code formatters are good and exist for json or edn (extensible data notation) or even full programming languages like smalltalk or even more complex ones like typescript. In a team it's true that code can be chaotic because not everybody uses a formatter and obviously we all have different coding style, even if the tooling made it more structured. So yes it would help to enforce more structure in a seamless way so we can focus on the important stuff without creating huge diffs in pull requests.

marhee 2y ago

A mature (and very effective) structural editor is the FLUID user-interface designer, part of the FLTK UI framework (C++). It outputs C++ and works very well. I remember dismissing it as a toy back when starting to work professionally with FLTK. Soon I realised I was mistaken; it is excellent to create UI's with for C++/FLKT.

codery 2y ago

https://github.com/disconcision/fructure https://github.com/disconcision/fructure https://www.youtube.com/watch?v=CnbVCNIh1NA https://www.youtube.com/watch?v=CnbVCNIh1NA

eternityforest 2y ago

I was able to use this without reading the manual, so I suppose it passes my favorite test of UI! But I do wish it was more clear what you have selected, and what things are selectable.