8 ms·

Frame-Based Editing [pdf]

140 points by panic 9y ago

sillysaurus3 9y ago
This is pretty incredible work wrapped in an unassuming name. It basically presents an entirely new way to interact with any programming language. You could make an emacs mode that worked this way, for example. I could see this being one of the fundamental ways of overcoming Lisp bias. Lisp still isn't mainstream. Clojure was a nice attempt, but it fell short. You can find companies that use Clojure, but it's not the lingua franca of any domain (except perhaps text editors). If you were to expose a way to write Lisp without dealing with any parens at all, it might have a chance of sparking the interest of younger programers long enough to seduce them to Lisp's other benefits: when you write in Lisp, you're writing in the abstract syntax tree normally generated by other languages. This allows you to write macros, which transform the tree in arbitrary ways. It's trivial to write a program to analyze your entire codebase in arbitrary ways (what are the most popular function calls? what's my dependency graph look like? which functions are unreferenced?) which is normally a herculean effort in other languages. And it all comes down to syntax. `(if a (b) (or c d))` is so utterly foreign to most programmers compared to `if (a) { return b(); } else { return c || d; }` that it's nearly impossible to overcome the inertia long enough to persuade them that the tradeoff in readability is worth the power you get. I think this work could be helpful here. When you throw a newcomer in front of a Lisp editor and say "Write a program," what goes through their minds? "How do I write an if statement?" "How do I call a function?" "How do I set a variable?" "How do I loop while a condition is true?" All of these are abstract concepts, not specific to any language, let alone Lisp. That'd let them avoid dealing with parens altogether. Then you can introduce the idea of a macro, which just combines new ways of using existing concepts. Regarding color: I've often wished for colored quasiquotation. `(foo `(bar `(,baz))) would be more useful if your editor displayed the background color differently for each level of quasiquation. It would also de-mystify statements like (fn (x) `(define-macro foo (y) `(define ,y ,',x))) which can otherwise be incredibly difficult to reason about. (Thankfully nested quasiquotation is rare, but suffice to say I wish editors used color more effectively rather than stylistically.) Sadly, emacs is one of the only editors flexible enough to implement the concepts presented in this paper, and the intersection between emacs users and novice programmers is about as large as the intersection between hackers and MBAs. But you can flip it around: emacs is flexible enough. That means you can ship a custom preconfigured emacs designed specifically for Lisp hacking, set up to mimic Atom/Coda/Sublime/whatever.
- pvg 9y ago
  And it all comes down to syntax. `(if a (b) (or c d))` is so utterly foreign to most programmers compared to `if (a) { return b(); } else { return c || d; }` That's a very strong claim without much evidence. Most programmers routinely deal with multiple syntaxes many of which are weirder than and less readable than lisp. A more likely thing is that there just isn't a sufficiently useful lisp.
  - deleted 9y ago
    [deleted]
  - KirinDave 9y ago
    I'm fairly sure it's safe to say the C/Algol derivative syntax is more familiar to programmers as the most popular and used programming language in the world uses it. Don't worry, we can fix it.
    - pvg 9y ago
      The claim was not about popularity or even familiarity, it was that Lisp syntax is so utterly alien, it's the primary thing that stands in the way of lisp adoption. I'm not worried but I appreciate the concern, I guess.
      - kazinator 9y ago
        Right, just like syntax like FOO=$(command ${path#${path#/*}}) stands in the way of Unix shell adoption. Also, this one hinders HTML adoption: <foo bar="attr"></foo> This kind of thing (a direct snippet from the Linux kernel Makefile) hinders GNU Make adoption: $(if $(wildcard $(objtree)/.config),, $(error No .config exists, config your kernel first!))
        KirinDave 9y ago
        The world is unjust, that's for sure. I look at the loader code I use in haskell w/ optparse-applicative and how much better it is than anything I've used in my damn life and I say, "Why is this considered unusable?"
kot-behemoth 9y ago
There is an app called Lisping which allows writing a variety of Lisps (Scheme and Clojure) on iPads. The great thing about it was that there was minimal typing - the interface made the AST very clickable, with minimal cursor movement. When I tried it, it was such a great way to edit code on an iPad. Demo video: https://www.youtube.com/watch?v=nHh00VPT7L4 https://www.youtube.com/watch?v=nHh00VPT7L4
- deleted 9y ago
  [deleted]
- edmccard 9y ago
  >the interface made the AST very clickable, with minimal cursor movement. Every so often the concept of visual programming comes up, where people wish that instead of editing text they could somehow directly manipulate an AST. Wouldn't it be funny if it turned out that it was Lisp they were looking for all along?
  - maxiepoo 9y ago
    Lisp is certainly amenable to this, and structured editors for lisp have been around for a long time, but the concept is not specific to lisp except inasmuch as it's much easier to implement since the syntax is so similar to the AST. For instance, I like to use Paredit [1] for elisp and Racket hacking, and there's a similar mode for Haskell in emacs [2]. [1] https://www.emacswiki.org/emacs/ParEdit https://www.emacswiki.org/emacs/ParEdit [2] https://github.com/chrisdone/structured-haskell-mode https://github.com/chrisdone/structured-haskell-mode
    - specialist 9y ago
      "...much easier to implement since the syntax is so similar to the AST." Hmmm. Updating my grammars to ANTLR 4.x's LL(Star) has been very rewarding. The resulting parse tree and AST are the same, if done carefully. I've never been smart enough to create a structured, incremental editor. Eg two-way editing. Your comment nudges me to pondering trying again.
- 9y ago
fcorsitsaroaway 9y ago
Literally sets an approach that's hit its limits in stone. [And yet people complain about punctuation.] Looking forward there will be books on programming with FBE. fbeop will be all the rage. Naturally we'll discuss the form of code. Is that bulge too far out? What is better: a plain of declarations that meets a mountain range of control loops and flow, or, is it better to "sprinkle the frame" with mini frames of plains and hills? There will be debates. Conferences will be held. Papers will be presented with analysis of old school code rendered in FBE. There will also be attempts to finally bring generics to Go, with FBE frontends with little angular < slots >. Efforts begin to teach monkeys to code. [p.s. and where exactly is that crab in Fig. 1? Very confusing ..]
brad0 9y ago
Is this essentially wrapping each scoped segment of code into a "frame" for manupulation using a mouse? I must be misunderstanding, how can this be more useful than plain text? Eliminating syntax errors?
- cycomachead 9y ago
  Syntax errors can be a large impediment to learning to program, so that's one big advantage. Frame-based editing has some other advantages, some of which you'd see in more complex IDEs like being context aware of what the user is truing to do.
- Jtsummers 9y ago
  Not just with a mouse. The frames can be detected automatically. Imagine a typical IDE where: while (...) { Automatically inserted the next `}` and moved the cursor between it. But in the frame based editor, the `{` and `}` are elided from the presentation, replaced with a colored block or frame around the inner kernel of the while loop. +++ while (condition) + --- if (condition) + - *** actions + ------------------ ++++++++++++++++++++++ Scratch (the example in Figure 1) is definitely a mouse driven interface, but beyond that and its visual presentation, it's more a cousin to the frame based model than a precise representation. It gives you a better (arguable?), structural way to view code, and to edit it with both text input and mouse.
  - a3n 9y ago
    Yes, it's arguable. :) (curmudgeon)A well-formed program is "defined" as what the compiler expects. Like literate programming, this hides from me what's being presented to the compiler, and sets me adrift from what the program actually is.(/curmudgeon) It also reminds me vaguely of writing with Framemaker.
    - edmccard 9y ago
      >this hides from me what's being presented to the compiler Isn't that the whole point of abstractions? Like, when you write a function call, you see "foo()" but the compiler sees all the individual instructions in the function. The same with macros, templates, etc. And on a purely visual level, the example 'jtsummers gave just seems to be an extension of code folding and other tools that let you focus on one part of the code while ignoring another.
KirinDave 9y ago
I was much more excited about this before I read the article.
hcs 9y ago
Coincidentally I just started working on an editor for touch screens that relies on displaying nesting like this. I also worked on a syntactic zoom which I'm sort of proud of, but don't know how practical it could be yet. Editing is still in progress but I have a read-only demo at https://gashlin.net/tests/ecola/hn/ https://gashlin.net/tests/ecola/hn/
- binhqx 9y ago
  On a ms surface running chrome 59, that pinch zoom feels nice
escherize 9y ago
Yeah, it's a pretty good idea. I've been using lisp and expand-region[1] and it just makes this frame-based editing dream come alive. I've just recorded a video where I'll refactor some code for fun. This code adds two arrays one has strings in it like ["1" "2" "3"] the other has numbers like [1 2 3]. Video: https://s3.us-east-2.amazonaws.com/photoblobs/2017-06-22_003949.mp4 https://s3.us-east-2.amazonaws.com/photoblobs/2017-06-22_003... I do have a couple typos / mis-compiles but that's sort of par for the course. Hope you enjoy! [1] - https://github.com/magnars/expand-region.el https://github.com/magnars/expand-region.el
- Huege 9y ago
  Exactly
deleted 9y ago
[deleted]
speps 9y ago
Reminds me of Construct : https://static2.scirra.net/images/fresh/c2/gallery/fullsize/jpg/eventsheet-edit-01.jpg https://static2.scirra.net/images/fresh/c2/gallery/fullsize/...
- broxp 9y ago
  Or programming in the RPG Maker: http://d289qh4hsbjjw7.cloudfront.net/rpgmaker-20130522223546811/files/screenshot-rpg-maker-vx-13.jpg http://d289qh4hsbjjw7.cloudfront.net/rpgmaker-20130522223546...
broxp 9y ago
Regarding coloring, I created a simple Ruby editor with syntax highlighting that features... - Indentation based coloring of whitespace - Nesting-based coloring of parentheses You can try it out here: https://broxp.github.io/ruby-ide https://broxp.github.io/ruby-ide Of course, these ideas are not new. There are plugins for IDEs for some languages, one that I recall is the F# depth colorizer: https://cyanbyfuchsia.wordpress.com/2013/11/18/cool-visual-studio-extensions-for-f-developer/ https://cyanbyfuchsia.wordpress.com/2013/11/18/cool-visual-s... And there is parentheses highlighting in Eclipse (when touched by the cursor).
gr__or 9y ago
I wrote a similar post, though much shorter and less detailed. Nevertheless eerily similar: https://medium.com/@grgtwt/code-is-not-just-text-1082981ae27f https://medium.com/@grgtwt/code-is-not-just-text-1082981ae27... Greenfoot really was the biggest inspiration, but I'd really like to build something more general purporse. Aka pass in a grammar file and then you can use the editor for that language.
auggierose 9y ago
The core thesis of this work is "that text is an inappropriate way to model structured program code". I think that is wrong, text is a great way to model structured program code. That doesn't mean that the ways we use text these days for programming couldn't be improved and augmented. For example, the example in Figure 3 is very pretty and readable. But there is no reason why this couldn't be what you see when still editing everything as text. Basically, it could be a more advanced form of syntax highlighting. The reason why this is difficult/impossible to achieve with Java is because of Java's syntax. Therefore I think when approaching programming from a user interface point of view, it is crucial to also work on a better syntax for the programming language itself. Ideally, work like described in this paper should go hand in hand with work to develop better syntaxes.
- falcolas 9y ago
  IMO, the big problem with text as a representation of (imperative) code is that it only represents the program's execution flow; the visualization of the flow of data must be done in a person's mental model. The side effect of this is that a programmer must build the proper mental model anytime they want to make a change, or they risk mucking up the data. Object orientated programming was an attempt at localizing data changes, at the cost of complicating the overall flow of the program's execution. Representing the flow of data and not the program is one advantage that many functional languages have, at the cost of removing the visualization of the program's flow. Often this doesn't matter, but sometimes it does. When it does matter, you're faced with the same challenge for the programmer. So ultimately, even the best syntax can only help you model one thing - either the flow of the data or the flow of the program - and the other flow must be modeled internally. It would be cool if both could be visualized by our tooling simultaneously.
eecc 9y ago
https://www.jetbrains.com/mps/ https://www.jetbrains.com/mps/ ?

140 points by panic 9y ago

This is pretty incredible work wrapped in an unassuming name. It basically presents an entirely new way to interact with any programming language. You could make an emacs mode that worked this way, for example. I could see this being one of the fundamental ways of overcoming Lisp bias. Lisp still isn't mainstream. Clojure was a nice attempt, but it fell short. You can find companies that use Clojure, but it's not the lingua franca of any domain (except perhaps text editors). If you were to expose a way to write Lisp without dealing with any parens at all, it might have a chance of sparking the interest of younger programers long enough to seduce them to Lisp's other benefits: when you write in Lisp, you're writing in the abstract syntax tree normally generated by other languages. This allows you to write macros, which transform the tree in arbitrary ways. It's trivial to write a program to analyze your entire codebase in arbitrary ways (what are the most popular function calls? what's my dependency graph look like? which functions are unreferenced?) which is normally a herculean effort in other languages. And it all comes down to syntax. `(if a (b) (or c d))` is so utterly foreign to most programmers compared to `if (a) { return b(); } else { return c || d; }` that it's nearly impossible to overcome the inertia long enough to persuade them that the tradeoff in readability is worth the power you get. I think this work could be helpful here. When you throw a newcomer in front of a Lisp editor and say "Write a program," what goes through their minds? "How do I write an if statement?" "How do I call a function?" "How do I set a variable?" "How do I loop while a condition is true?" All of these are abstract concepts, not specific to any language, let alone Lisp. That'd let them avoid dealing with parens altogether. Then you can introduce the idea of a macro, which just combines new ways of using existing concepts. Regarding color: I've often wished for colored quasiquotation. `(foo `(bar `(,baz))) would be more useful if your editor displayed the background color differently for each level of quasiquation. It would also de-mystify statements like (fn (x) `(define-macro foo (y) `(define ,y ,',x))) which can otherwise be incredibly difficult to reason about. (Thankfully nested quasiquotation is rare, but suffice to say I wish editors used color more effectively rather than stylistically.) Sadly, emacs is one of the only editors flexible enough to implement the concepts presented in this paper, and the intersection between emacs users and novice programmers is about as large as the intersection between hackers and MBAs. But you can flip it around: emacs is flexible enough. That means you can ship a custom preconfigured emacs designed specifically for Lisp hacking, set up to mimic Atom/Coda/Sublime/whatever.

pvg 9y ago
And it all comes down to syntax. `(if a (b) (or c d))` is so utterly foreign to most programmers compared to `if (a) { return b(); } else { return c || d; }` That's a very strong claim without much evidence. Most programmers routinely deal with multiple syntaxes many of which are weirder than and less readable than lisp. A more likely thing is that there just isn't a sufficiently useful lisp.
- deleted 9y ago
  [deleted]
- KirinDave 9y ago
  I'm fairly sure it's safe to say the C/Algol derivative syntax is more familiar to programmers as the most popular and used programming language in the world uses it. Don't worry, we can fix it.
  - pvg 9y ago
    The claim was not about popularity or even familiarity, it was that Lisp syntax is so utterly alien, it's the primary thing that stands in the way of lisp adoption. I'm not worried but I appreciate the concern, I guess.
    - kazinator 9y ago
      Right, just like syntax like FOO=$(command ${path#${path#/*}}) stands in the way of Unix shell adoption. Also, this one hinders HTML adoption: <foo bar="attr"></foo> This kind of thing (a direct snippet from the Linux kernel Makefile) hinders GNU Make adoption: $(if $(wildcard $(objtree)/.config),, $(error No .config exists, config your kernel first!))
      - KirinDave 9y ago
        The world is unjust, that's for sure. I look at the loader code I use in haskell w/ optparse-applicative and how much better it is than anything I've used in my damn life and I say, "Why is this considered unusable?"

kot-behemoth 9y ago

There is an app called Lisping which allows writing a variety of Lisps (Scheme and Clojure) on iPads. The great thing about it was that there was minimal typing - the interface made the AST very clickable, with minimal cursor movement. When I tried it, it was such a great way to edit code on an iPad. Demo video: https://www.youtube.com/watch?v=nHh00VPT7L4 https://www.youtube.com/watch?v=nHh00VPT7L4

deleted 9y ago
[deleted]
edmccard 9y ago
>the interface made the AST very clickable, with minimal cursor movement. Every so often the concept of visual programming comes up, where people wish that instead of editing text they could somehow directly manipulate an AST. Wouldn't it be funny if it turned out that it was Lisp they were looking for all along?
- maxiepoo 9y ago
  Lisp is certainly amenable to this, and structured editors for lisp have been around for a long time, but the concept is not specific to lisp except inasmuch as it's much easier to implement since the syntax is so similar to the AST. For instance, I like to use Paredit [1] for elisp and Racket hacking, and there's a similar mode for Haskell in emacs [2]. [1] https://www.emacswiki.org/emacs/ParEdit https://www.emacswiki.org/emacs/ParEdit [2] https://github.com/chrisdone/structured-haskell-mode https://github.com/chrisdone/structured-haskell-mode
  - specialist 9y ago
    "...much easier to implement since the syntax is so similar to the AST." Hmmm. Updating my grammars to ANTLR 4.x's LL(Star) has been very rewarding. The resulting parse tree and AST are the same, if done carefully. I've never been smart enough to create a structured, incremental editor. Eg two-way editing. Your comment nudges me to pondering trying again.
9y ago

fcorsitsaroaway 9y ago

Literally sets an approach that's hit its limits in stone. [And yet people complain about punctuation.] Looking forward there will be books on programming with FBE. fbeop will be all the rage. Naturally we'll discuss the form of code. Is that bulge too far out? What is better: a plain of declarations that meets a mountain range of control loops and flow, or, is it better to "sprinkle the frame" with mini frames of plains and hills? There will be debates. Conferences will be held. Papers will be presented with analysis of old school code rendered in FBE. There will also be attempts to finally bring generics to Go, with FBE frontends with little angular < slots >. Efforts begin to teach monkeys to code. [p.s. and where exactly is that crab in Fig. 1? Very confusing ..]

brad0 9y ago

Is this essentially wrapping each scoped segment of code into a "frame" for manupulation using a mouse? I must be misunderstanding, how can this be more useful than plain text? Eliminating syntax errors?

cycomachead 9y ago
Syntax errors can be a large impediment to learning to program, so that's one big advantage. Frame-based editing has some other advantages, some of which you'd see in more complex IDEs like being context aware of what the user is truing to do.
Jtsummers 9y ago
Not just with a mouse. The frames can be detected automatically. Imagine a typical IDE where: while (...) { Automatically inserted the next `}` and moved the cursor between it. But in the frame based editor, the `{` and `}` are elided from the presentation, replaced with a colored block or frame around the inner kernel of the while loop. +++ while (condition) + --- if (condition) + - *** actions + ------------------ ++++++++++++++++++++++ Scratch (the example in Figure 1) is definitely a mouse driven interface, but beyond that and its visual presentation, it's more a cousin to the frame based model than a precise representation. It gives you a better (arguable?), structural way to view code, and to edit it with both text input and mouse.
- a3n 9y ago
  Yes, it's arguable. :) (curmudgeon)A well-formed program is "defined" as what the compiler expects. Like literate programming, this hides from me what's being presented to the compiler, and sets me adrift from what the program actually is.(/curmudgeon) It also reminds me vaguely of writing with Framemaker.
  - edmccard 9y ago
    >this hides from me what's being presented to the compiler Isn't that the whole point of abstractions? Like, when you write a function call, you see "foo()" but the compiler sees all the individual instructions in the function. The same with macros, templates, etc. And on a purely visual level, the example 'jtsummers gave just seems to be an extension of code folding and other tools that let you focus on one part of the code while ignoring another.

KirinDave 9y ago

I was much more excited about this before I read the article.

hcs 9y ago

Coincidentally I just started working on an editor for touch screens that relies on displaying nesting like this. I also worked on a syntactic zoom which I'm sort of proud of, but don't know how practical it could be yet. Editing is still in progress but I have a read-only demo at https://gashlin.net/tests/ecola/hn/ https://gashlin.net/tests/ecola/hn/

binhqx 9y ago
On a ms surface running chrome 59, that pinch zoom feels nice

escherize 9y ago

Yeah, it's a pretty good idea. I've been using lisp and expand-region[1] and it just makes this frame-based editing dream come alive. I've just recorded a video where I'll refactor some code for fun. This code adds two arrays one has strings in it like ["1" "2" "3"] the other has numbers like [1 2 3]. Video: https://s3.us-east-2.amazonaws.com/photoblobs/2017-06-22_003949.mp4 https://s3.us-east-2.amazonaws.com/photoblobs/2017-06-22_003... I do have a couple typos / mis-compiles but that's sort of par for the course. Hope you enjoy! [1] - https://github.com/magnars/expand-region.el https://github.com/magnars/expand-region.el

Huege 9y ago
Exactly

deleted 9y ago

[deleted]

speps 9y ago

Reminds me of Construct : https://static2.scirra.net/images/fresh/c2/gallery/fullsize/jpg/eventsheet-edit-01.jpg https://static2.scirra.net/images/fresh/c2/gallery/fullsize/...

broxp 9y ago
Or programming in the RPG Maker: http://d289qh4hsbjjw7.cloudfront.net/rpgmaker-20130522223546811/files/screenshot-rpg-maker-vx-13.jpg http://d289qh4hsbjjw7.cloudfront.net/rpgmaker-20130522223546...

broxp 9y ago

Regarding coloring, I created a simple Ruby editor with syntax highlighting that features... - Indentation based coloring of whitespace - Nesting-based coloring of parentheses You can try it out here: https://broxp.github.io/ruby-ide https://broxp.github.io/ruby-ide Of course, these ideas are not new. There are plugins for IDEs for some languages, one that I recall is the F# depth colorizer: https://cyanbyfuchsia.wordpress.com/2013/11/18/cool-visual-studio-extensions-for-f-developer/ https://cyanbyfuchsia.wordpress.com/2013/11/18/cool-visual-s... And there is parentheses highlighting in Eclipse (when touched by the cursor).

gr__or 9y ago

I wrote a similar post, though much shorter and less detailed. Nevertheless eerily similar: https://medium.com/@grgtwt/code-is-not-just-text-1082981ae27f https://medium.com/@grgtwt/code-is-not-just-text-1082981ae27... Greenfoot really was the biggest inspiration, but I'd really like to build something more general purporse. Aka pass in a grammar file and then you can use the editor for that language.

auggierose 9y ago

The core thesis of this work is "that text is an inappropriate way to model structured program code". I think that is wrong, text is a great way to model structured program code. That doesn't mean that the ways we use text these days for programming couldn't be improved and augmented. For example, the example in Figure 3 is very pretty and readable. But there is no reason why this couldn't be what you see when still editing everything as text. Basically, it could be a more advanced form of syntax highlighting. The reason why this is difficult/impossible to achieve with Java is because of Java's syntax. Therefore I think when approaching programming from a user interface point of view, it is crucial to also work on a better syntax for the programming language itself. Ideally, work like described in this paper should go hand in hand with work to develop better syntaxes.

falcolas 9y ago
IMO, the big problem with text as a representation of (imperative) code is that it only represents the program's execution flow; the visualization of the flow of data must be done in a person's mental model. The side effect of this is that a programmer must build the proper mental model anytime they want to make a change, or they risk mucking up the data. Object orientated programming was an attempt at localizing data changes, at the cost of complicating the overall flow of the program's execution. Representing the flow of data and not the program is one advantage that many functional languages have, at the cost of removing the visualization of the program's flow. Often this doesn't matter, but sometimes it does. When it does matter, you're faced with the same challenge for the programmer. So ultimately, even the best syntax can only help you model one thing - either the flow of the data or the flow of the program - and the other flow must be modeled internally. It would be cool if both could be visualized by our tooling simultaneously.

eecc 9y ago

https://www.jetbrains.com/mps/ https://www.jetbrains.com/mps/ ?