6 ms·

Tim Teitelbaum, one of the first researchers on IDEs summarized problems like this in a quote from one of his papers. "Programs are not text; they are hierarch

Tim Teitelbaum, one of the first researchers on IDEs summarized problems like this in a quote from one of his papers. "Programs are not text; they are hierarchical compositions of computational structures and should be edited, executed, and debugged in an environment that consistently acknowledges and reinforces this viewpoint." It is unfortunate that most major code editors and IDEs today do not store units of code (like Lisp SEXPs) as databases to be updated as you edit, which would make problems like this or other operations like metaprogramming, formal analysis, or documentation much easier to solve.

skybrian 9y ago
I'm not sure why you think they don't? Most IDE's do typically build a language-specific search index and keep it up to date as you edit. (That's the main difference between an IDE and a text editor, though the line is blurrier these days.) Having taken Teitelbaum's compiler class as an undergrad, I was happy to ditch the IDE he inflicted on us and go back to text editors. Structured code editing is a tricky UI problem and I didn't find a really good IDE until many years later. There's no one way to edit code. Sometimes refactoring tools work well, but typing text can be quite efficient too. Getting locked into a tree editor at the expression level is no fun.
- olewhalehunter 9y ago
  >I'm not sure why you think they don't? If they are they either aren't offering user/programmer access to that database, they aren't doing semantic/type binds, or aren't advertising those features well. >I was happy to ditch the IDE he inflicted on us and go back to text editors. Structured code editing is a tricky UI problem It's reconcilable with normal text editing, just update the structure once its valid. I agree that things like block/visual programming can be absurd.
  - skybrian 9y ago
    I'm not sure what you mean by "semantic/type binds", but if you're writing a plugin, IDE's like Eclipse and IDEA do give you access to program syntax via a Java API, and for many languages there is also type-aware indexing. Typically this is exposed to the user as specific queries (such as "go to definition" or "find all usages") and updates ("rename method"). From a UI perspective, more features can be added by writing more plugins and/or improving them. But this indexing is only on one user's workstation and tends not to scale up well. Updating dependencies or switching to a different branch means rebuilding large parts of the index. Also, part of the problem is that there is little standardization. Many ecosystems are language, platform, build tool, and/or editor-specific. When you do something new you end up reinventing the wheel.
- imtringued 9y ago
  Especially in statically typed programs you want to "break" your program for refactoring purposes. Usually I look at a function or class and create new functions/classes and then rename the old one and then fix all the errors one by one by using the new code.
skrebbel 9y ago
There are editors that do this. Notable, Jetbrains MPS comes to mind. It looks like a text editor but once you use it you quickly notice that you're actually editing the abstract syntax tree directly. It's cool, but it has some major downsides too. For example, MPS stores the source as XML, not text (since it isn't text, it's a tree). This makes lots of basic tools we've taken for granted a lot harder, such as git merging etc. They've had to make a custom mergetool just to make basic collaborative coding feasible. I bet there's other ways around that, all I'm saying is that text has major, major upsides because of the enormous ecosystem support.
- sitkack 9y ago
  It would be nice to have a VCS that could work on the native MPS data structures. Text is probably the next biggest mistake in programmer productivity after null.
  - icebraining 9y ago
    But why? Tools can parse text just fine, and create its MPS structures in the background to do whatever it needs. Why expose this to the programmer?
    - sitkack 9y ago
      Rich structured editors free one from text and are able to encode other information that is not currently recorded in text formats. Directly operating on structures would free languages from parsing, correctness checking and compiling could occur at every semantically correct operation. With a rich structure editor that can do merges, the undo history of edit and refactor operations could be persisted and merged into the VCS. Currently this isn't possible. Text is a projection for the page and a lowest common format.
      - icebraining 9y ago
        Directly operating on structures would free languages from parsing, correctness checking and compiling could occur at every semantically correct operation. Directly operating on structures would mean that you'd have to write an editor, which had to enforce correctness as well. And then you'd have to write a generator to save those structures in some kind of format that could be written to a file and passed around, and a parser to read such format. And check for correctness again, since who knows what generated that file. As for constant compilation, that already exists, many IDEs have it. That's because parsing text is not actually hard, the other stages are. With a rich structure editor that can do merges, the undo history of edit and refactor operations could be persisted and merged into the VCS. Currently this isn't possible. Of course it is, you could write a plugin for any IDE that would record edits and refactor operations and save those in or alongside the text (much like they've have to be save alongside the AST). Of course, that doesn't help if the user does a manual refactor, but that's no different than they choosing a node in the rich editor, deleting it, then manually recreating it in its refactored form.
sitkack 9y ago
For those looking for the Teitelbaum paper mentioned above [0] [0] The Cornell Program Synthesizer: A Syntax Directed Programming Environment https://core.ac.uk/download/pdf/21750999.pdf?repositoryId=145 https://core.ac.uk/download/pdf/21750999.pdf?repositoryId=14...
pkaye 9y ago
I think Microsoft did that in their Roslyn dot.net compiler. There is a server that is fed source code changes and it updates the AST internally incrementally. The text editor can then make queries against that AST. I believe Microsoft also took a portion of this as added it to Visual Studio Code.

skybrian 9y ago

I'm not sure why you think they don't? Most IDE's do typically build a language-specific search index and keep it up to date as you edit. (That's the main difference between an IDE and a text editor, though the line is blurrier these days.) Having taken Teitelbaum's compiler class as an undergrad, I was happy to ditch the IDE he inflicted on us and go back to text editors. Structured code editing is a tricky UI problem and I didn't find a really good IDE until many years later. There's no one way to edit code. Sometimes refactoring tools work well, but typing text can be quite efficient too. Getting locked into a tree editor at the expression level is no fun.

olewhalehunter 9y ago
>I'm not sure why you think they don't? If they are they either aren't offering user/programmer access to that database, they aren't doing semantic/type binds, or aren't advertising those features well. >I was happy to ditch the IDE he inflicted on us and go back to text editors. Structured code editing is a tricky UI problem It's reconcilable with normal text editing, just update the structure once its valid. I agree that things like block/visual programming can be absurd.
- skybrian 9y ago
  I'm not sure what you mean by "semantic/type binds", but if you're writing a plugin, IDE's like Eclipse and IDEA do give you access to program syntax via a Java API, and for many languages there is also type-aware indexing. Typically this is exposed to the user as specific queries (such as "go to definition" or "find all usages") and updates ("rename method"). From a UI perspective, more features can be added by writing more plugins and/or improving them. But this indexing is only on one user's workstation and tends not to scale up well. Updating dependencies or switching to a different branch means rebuilding large parts of the index. Also, part of the problem is that there is little standardization. Many ecosystems are language, platform, build tool, and/or editor-specific. When you do something new you end up reinventing the wheel.
imtringued 9y ago
Especially in statically typed programs you want to "break" your program for refactoring purposes. Usually I look at a function or class and create new functions/classes and then rename the old one and then fix all the errors one by one by using the new code.

skrebbel 9y ago

There are editors that do this. Notable, Jetbrains MPS comes to mind. It looks like a text editor but once you use it you quickly notice that you're actually editing the abstract syntax tree directly. It's cool, but it has some major downsides too. For example, MPS stores the source as XML, not text (since it isn't text, it's a tree). This makes lots of basic tools we've taken for granted a lot harder, such as git merging etc. They've had to make a custom mergetool just to make basic collaborative coding feasible. I bet there's other ways around that, all I'm saying is that text has major, major upsides because of the enormous ecosystem support.

sitkack 9y ago
It would be nice to have a VCS that could work on the native MPS data structures. Text is probably the next biggest mistake in programmer productivity after null.
- icebraining 9y ago
  But why? Tools can parse text just fine, and create its MPS structures in the background to do whatever it needs. Why expose this to the programmer?
  - sitkack 9y ago
    Rich structured editors free one from text and are able to encode other information that is not currently recorded in text formats. Directly operating on structures would free languages from parsing, correctness checking and compiling could occur at every semantically correct operation. With a rich structure editor that can do merges, the undo history of edit and refactor operations could be persisted and merged into the VCS. Currently this isn't possible. Text is a projection for the page and a lowest common format.
    - icebraining 9y ago
      Directly operating on structures would free languages from parsing, correctness checking and compiling could occur at every semantically correct operation. Directly operating on structures would mean that you'd have to write an editor, which had to enforce correctness as well. And then you'd have to write a generator to save those structures in some kind of format that could be written to a file and passed around, and a parser to read such format. And check for correctness again, since who knows what generated that file. As for constant compilation, that already exists, many IDEs have it. That's because parsing text is not actually hard, the other stages are. With a rich structure editor that can do merges, the undo history of edit and refactor operations could be persisted and merged into the VCS. Currently this isn't possible. Of course it is, you could write a plugin for any IDE that would record edits and refactor operations and save those in or alongside the text (much like they've have to be save alongside the AST). Of course, that doesn't help if the user does a manual refactor, but that's no different than they choosing a node in the rich editor, deleting it, then manually recreating it in its refactored form.

sitkack 9y ago

For those looking for the Teitelbaum paper mentioned above [0] [0] The Cornell Program Synthesizer: A Syntax Directed Programming Environment https://core.ac.uk/download/pdf/21750999.pdf?repositoryId=145 https://core.ac.uk/download/pdf/21750999.pdf?repositoryId=14...

pkaye 9y ago

I think Microsoft did that in their Roslyn dot.net compiler. There is a server that is fed source code changes and it updates the AST internally incrementally. The text editor can then make queries against that AST. I believe Microsoft also took a portion of this as added it to Visual Studio Code.