4 ms·

The problem with these environments is that humans routinely want to write syntactically incorrect code as they transition from one valid program to another. An

The problem with these environments is that humans routinely want to write syntactically incorrect code as they transition from one valid program to another. An AST-based editor like this enforces that every intermediate step must parse as a valid program. This restriction makes making changes a massive pain in the ass, because you can no longer take the shortest route from where you are to where you want to be. For example, imagine you're restructuring an if-else sequence into a switch statement. You're probably going to change that first "if" to a "switch", then go down changing all the "else"s into "case"s. This is eminently sensible, but an AST-based editor will stop you dead if you try to do this, because the intermediate states are not valid ASTs. The only system slightly like this I've seen in widespread use is Paredit, which prevents you typing anything that isn't a valid s-expression. This only works because s-expression syntax is so minimal, and most of Lisp's actual syntax is layered on top with macros. Even if you're using Paredit, your spit/slurp/kill operations will produce invalid intermediate code that would fail to compile.

westoncb 10y ago
That's an interesting point. I've heard other people bring up this issue in a vague way, but your example makes it clear why people would have a concern about this. At the same time, however, I'm not convinced it's a serious issue—it's more important to save work and avoid re-typing the shell of a switch when you have to type it one character at a time. If you're mapping language constructs to single keys (this was the way I went), you can crank out a new multi-block switch statement in a few keystrokes; then (in my editor anyway), you can just drag the code blocks from the if statements into the new structure. Even better, since the editor has easy access to info about language structure, it could include automatic transformations to transition between structurally similar language constructs. I think it seems unnatural at first because we're so used to typing characters out one at a time, and it seems like we're throwing that skill out the window when considering an alternate form of editor. I still don't really see an intrinsic limit though. I'd appreciate any other counter-example you may have (a lot of why I don't continue working on my own project at the moment is concern that these are lurking and I'll only discover them after lots more wasted effort—so if I could definitively rule out that this project is a good idea, that'd be great.)
- meredydd 10y ago
  I'd say the opposite - in your shoes, I'd just try it, doomed or not! (But then again building an IDE is My Thing atm - check my profile.) Building one of these systems is going to be really fun. If you don't have the time/inclination to just do it cause it's a cool project, one option is to "paper prototype" the feasibility of the transforms you'd need. Next time you're writing any code in the first language you want to support in that AST editor, take a screen recording of at least half an hour's coding. Watch it back later, and keep a count of how many times you: * transition from one valid program to another in a simple way (eg just writing a line of code from scratch that's valid first time) * transition in a way that would require abandoning your keyboard and reaching for your mouse in your editor (NB this is a seriously slow operation, usually taking a couple of seconds or more). * transition in a complex way your editor would support if it had that kind of transform built in (eg if/else-> switch), and work out exactly how clever that transform would have to be (would it work if the if/elses weren't simple equality checks? If they weren't, how would you make the transform if you weren't allowed any non-compiling code?) Then enumerate the distinct such transforms you'd need to cover that half-hour of typing. (You will probably discover a Pareto-type distribution - the question is how tight the head/long the tail is. My guess is you'd need a huge number of special transforms to cover 95% of your edits, but data beats guessing) * Jump around between statements (eg half-write something, leave it in an utterly broken state, then go actually define the variable/function you're using, then jump back and finish your thought). Your editor would have to permit this somehow or it will be really frustrating to use.
  - westoncb 10y ago
    Thanks for the reply meredydd. That does seem like a good approach. Actually, I spent 1.5 years building an editor in this style (see link in my original post here) while working at a grocery store :) Unfortunately, while I can see now that I should have first been super focused on validating the concept—I instead just kind of ran with it, assuming it was going to work, and built this massive, probably over-engineered, framework for generating editors for given grammars (with my starting point being: not even knowing what a grammar was, thinking I'd have to invent some kind of 'linguistic constraint description' format ;) ). As it stands the editing portion works well enough for a demo, but the program never reached the point where I could write code with it, so a lot of these questions are still un answered for me. I think doing the paper prototype on these edit actions would be a good pre-coding validation step. I did check out Anvil briefly. My two second, potentially incorrect summary would be VB for web apps. Is that close? Are you guys doing anything special for working with text itself?

westoncb 10y ago

That's an interesting point. I've heard other people bring up this issue in a vague way, but your example makes it clear why people would have a concern about this. At the same time, however, I'm not convinced it's a serious issue—it's more important to save work and avoid re-typing the shell of a switch when you have to type it one character at a time. If you're mapping language constructs to single keys (this was the way I went), you can crank out a new multi-block switch statement in a few keystrokes; then (in my editor anyway), you can just drag the code blocks from the if statements into the new structure. Even better, since the editor has easy access to info about language structure, it could include automatic transformations to transition between structurally similar language constructs. I think it seems unnatural at first because we're so used to typing characters out one at a time, and it seems like we're throwing that skill out the window when considering an alternate form of editor. I still don't really see an intrinsic limit though. I'd appreciate any other counter-example you may have (a lot of why I don't continue working on my own project at the moment is concern that these are lurking and I'll only discover them after lots more wasted effort—so if I could definitively rule out that this project is a good idea, that'd be great.)

meredydd 10y ago
I'd say the opposite - in your shoes, I'd just try it, doomed or not! (But then again building an IDE is My Thing atm - check my profile.) Building one of these systems is going to be really fun. If you don't have the time/inclination to just do it cause it's a cool project, one option is to "paper prototype" the feasibility of the transforms you'd need. Next time you're writing any code in the first language you want to support in that AST editor, take a screen recording of at least half an hour's coding. Watch it back later, and keep a count of how many times you: * transition from one valid program to another in a simple way (eg just writing a line of code from scratch that's valid first time) * transition in a way that would require abandoning your keyboard and reaching for your mouse in your editor (NB this is a seriously slow operation, usually taking a couple of seconds or more). * transition in a complex way your editor would support if it had that kind of transform built in (eg if/else-> switch), and work out exactly how clever that transform would have to be (would it work if the if/elses weren't simple equality checks? If they weren't, how would you make the transform if you weren't allowed any non-compiling code?) Then enumerate the distinct such transforms you'd need to cover that half-hour of typing. (You will probably discover a Pareto-type distribution - the question is how tight the head/long the tail is. My guess is you'd need a huge number of special transforms to cover 95% of your edits, but data beats guessing) * Jump around between statements (eg half-write something, leave it in an utterly broken state, then go actually define the variable/function you're using, then jump back and finish your thought). Your editor would have to permit this somehow or it will be really frustrating to use.
- westoncb 10y ago
  Thanks for the reply meredydd. That does seem like a good approach. Actually, I spent 1.5 years building an editor in this style (see link in my original post here) while working at a grocery store :) Unfortunately, while I can see now that I should have first been super focused on validating the concept—I instead just kind of ran with it, assuming it was going to work, and built this massive, probably over-engineered, framework for generating editors for given grammars (with my starting point being: not even knowing what a grammar was, thinking I'd have to invent some kind of 'linguistic constraint description' format ;) ). As it stands the editing portion works well enough for a demo, but the program never reached the point where I could write code with it, so a lot of these questions are still un answered for me. I think doing the paper prototype on these edit actions would be a good pre-coding validation step. I did check out Anvil briefly. My two second, potentially incorrect summary would be VB for web apps. Is that close? Are you guys doing anything special for working with text itself?