4 ms·

Dion 2020 IDE Research Demo

41 points by archagon 5y ago

armchairhacker 5y ago
it looks like they want a code representation better than text tbh basic text is a fine storage tool, we just need better incremental analyses and tools to display and manipulate that text as more
- deleted 5y ago
  [deleted]
- omneity 5y ago
  > better incremental analyses and tools to display and manipulate that text as more This is my exact impression of Dion after watching their demos. This link might give you a better idea: https://dion.systems/gallery.html https://dion.systems/gallery.html
  - shepardrtc 5y ago
    They should just lead with the gallery. A picture is worth a thousand words and all. Because honestly now I'm really excited to see what they can do.
  - aidenn0 5y ago
    They do claim "Text is not the source format but just the way it is rendered on the screen" at that page, which supports the comment in TFA that they wish to abandon text files as representation for code. That seems rather silly to me; if everyone wants to interact with it as source, and source can be an unambiguous representation of its structure, why store it in some other format?
necovek 5y ago
I can see where they are going with things like Metadesk, but it's ultimately just another file format (text or otherwise) to represent rich structured data. The endgoal is probably to allow neater refactoring opportunities and raise the level of efficiency by standardising on an easier-to-parse-into-AST-like-structure syntax, enabling generic tools to be built that will deal with such structures directly. While I sense that there is something smart that could be done in that area, there are already plenty of de facto standardized languages for structured data (most notably XML), and already a bunch of real world programming languages that work. So I think it'd be easier to grasp their goals if they started off of an existing language ecosystem (that language spec is the definition for their AST parser), and attempted to build the tools they want: this would have more quickly formalized what's missing in the source code format for it to remain human editable, but at the same time easier and richer to parse. Again, it's all a bit vague since you can build a development environment based on text files where you are not seeing all of that generated text at all times (if ever). As the goal seems to be to find improvements to the process of program development while being unconstrained by complexity of (re)parsing text files repeatedly, I would have started with trying to add those improvements and ignoring the slowness/cost of parsing (or simply used an existing language that's cheap to parse, like Lisp).
- throwaway34241 5y ago
  I agree, it seems like an editor based on direct AST manipulation could work pretty well with existing languages (I actually prototyped one a long time ago in university). Some languages like C with text-based preprocessors might cause some headaches but there are others that probably wouldn't be too bad. There's pretty far you can go with text-based formats since you aren't obligated to display the file exactly as it's stored on disk (and many current IDEs do minor code folding things). For example embedded images can be displayed inline in the source code, but be stored as some loadImage() function on disk. You could even have some comments with base64 binary data if you really needed - at that point binary vs text is mostly a performance issue, but parsing is usually pretty fast so being text-based might still have an advantage because of better interoperability with source control etc.
junon 5y ago
I'd much rather they do this with text and then use something like tree sitter to do the zooming and separation thing because I've always thought that would be useful, too. Looks neat, not sure how practical it'll be.
zokier 5y ago
Sounds a lot like sexprs(/LISP), something they seem to avoid mentioning. Overall I'm not convinced about the need for new file format, I don't really believe that parsing is a bottleneck in IDEs (unless we are talking about C++...), especially with tools like tree-sitter or rust-analyzer. Furthermore parsing should be easily cacheable if performance is a concern. Going the other way, tools like gofmt etc already pretty much standardize a canonical serialization format for any particular AST. That being said, I'm fan of the concept of structure editing etc, so I'm still interested to see what they come up with.
pizza 5y ago
What I need for a metaprogramming tool is a code generator that uses CodeQL as input, then allows me to specify very terse AST manipulation/transformation functions, then generate my output code for me. The one real flaw about CodeQL imo is that they don’t let me generate code.. Like suppose I found a very nice library whose only fatal flaw was that it only wrote its output to files as opposed to general buffers/streams. And I had another library that was godawful but it had a very nice, modular, network streaming routine. I want this: from nice_library, awful_library select concat(nice_library_routine_upto_filwriting_entrypoint, awful_library_routine_networkIO_entrypoint) It’s likely gonna need more information to make it possible but this is the sketch of my intent In fact a gui like Scratch may be just the type of graphical user interface that would make this nice.. think colorful affordances that prevent you from writing a buggy codeql query and iterating pointlessly when everything could be visually typechecked. You could even call it Frankenstein :) Would that become closer to reality with this?
duped 5y ago
I like seeing raw ambition in these projects, my reading is that this is not "let's make a better IDE" but "let's fundamentally change how software tools interact". There are numerous projects that I have worked on that would benefit from semantic code storage (and have used LSIF[1], which only goes so far). I think the demos are neat but really don't hit on the massive productivity increases, this is the stuff that people would throw money at : - Language-aware-yet-independent and polyglot build systems that don't need new frontends for new languages. - A CI platform that doesn't need configuration - Semantic fuzzing [1] https://lsif.dev/ https://lsif.dev/
abnercoimbre 5y ago
If you want to see their demo that made a pretty big splash it's available on Handmade Seattle. [0] Usual disclosure that I am the organizer for these things. [0] https://media.handmade-seattle.com/dion-systems https://media.handmade-seattle.com/dion-systems
Cilvic 5y ago
I think this is pretty cool. I've been thinking about this "tree" in context of presentations which are structure + content/facts + layout/visualization. Basically all presentation content is just a big tree/hierarchy of facts (text and numbers) and some applied layout. Many of the problems are similar: - you want to look at the problem on different levels "high level" story line, low level drilling into section 4.3.2, seamless zoom - you'd want to have different representations of the same information. During problem solving the information you are collecting is constantly growing and it's a back and forth between adding data, shaping it differently (lists become tables etc.) - easy entity/renaming, search and replace for facts + audit trails "where does the number come from" etc. - CSS / re-styling (why does it matter if important words are blue or red, that should just be a CSS theme you switch out) - there also exists "structs" which are structured entities like say "these are the 40 offices" the client has, and we want to represent them in various forms - ... I wonder if notion could be turned into a modern problem solving / result presentation maker. It's very flexible in surfacing the same content in different formats. More related to software development and their demo: Is any additional information in the tree, that is not /can not be in the text rendering. I get that the text rendering can be "loss-y" meaning that if it's badly rendered, information can be lost / the text has errors. But if you have the information, and you render it into a certain text representation that is error free. Then the program should also be able to parse it back into a tree? What are the situations were storing it in "tree form" are actually superior? Isn't the "tree" just a more compressed, less error prone and stricter text?

41 points by archagon 5y ago

it looks like they want a code representation better than text tbh basic text is a fine storage tool, we just need better incremental analyses and tools to display and manipulate that text as more

deleted 5y ago
[deleted]
omneity 5y ago
> better incremental analyses and tools to display and manipulate that text as more This is my exact impression of Dion after watching their demos. This link might give you a better idea: https://dion.systems/gallery.html https://dion.systems/gallery.html
- shepardrtc 5y ago
  They should just lead with the gallery. A picture is worth a thousand words and all. Because honestly now I'm really excited to see what they can do.
- aidenn0 5y ago
  They do claim "Text is not the source format but just the way it is rendered on the screen" at that page, which supports the comment in TFA that they wish to abandon text files as representation for code. That seems rather silly to me; if everyone wants to interact with it as source, and source can be an unambiguous representation of its structure, why store it in some other format?

necovek 5y ago

I can see where they are going with things like Metadesk, but it's ultimately just another file format (text or otherwise) to represent rich structured data. The endgoal is probably to allow neater refactoring opportunities and raise the level of efficiency by standardising on an easier-to-parse-into-AST-like-structure syntax, enabling generic tools to be built that will deal with such structures directly. While I sense that there is something smart that could be done in that area, there are already plenty of de facto standardized languages for structured data (most notably XML), and already a bunch of real world programming languages that work. So I think it'd be easier to grasp their goals if they started off of an existing language ecosystem (that language spec is the definition for their AST parser), and attempted to build the tools they want: this would have more quickly formalized what's missing in the source code format for it to remain human editable, but at the same time easier and richer to parse. Again, it's all a bit vague since you can build a development environment based on text files where you are not seeing all of that generated text at all times (if ever). As the goal seems to be to find improvements to the process of program development while being unconstrained by complexity of (re)parsing text files repeatedly, I would have started with trying to add those improvements and ignoring the slowness/cost of parsing (or simply used an existing language that's cheap to parse, like Lisp).

throwaway34241 5y ago
I agree, it seems like an editor based on direct AST manipulation could work pretty well with existing languages (I actually prototyped one a long time ago in university). Some languages like C with text-based preprocessors might cause some headaches but there are others that probably wouldn't be too bad. There's pretty far you can go with text-based formats since you aren't obligated to display the file exactly as it's stored on disk (and many current IDEs do minor code folding things). For example embedded images can be displayed inline in the source code, but be stored as some loadImage() function on disk. You could even have some comments with base64 binary data if you really needed - at that point binary vs text is mostly a performance issue, but parsing is usually pretty fast so being text-based might still have an advantage because of better interoperability with source control etc.

junon 5y ago

I'd much rather they do this with text and then use something like tree sitter to do the zooming and separation thing because I've always thought that would be useful, too. Looks neat, not sure how practical it'll be.

zokier 5y ago

Sounds a lot like sexprs(/LISP), something they seem to avoid mentioning. Overall I'm not convinced about the need for new file format, I don't really believe that parsing is a bottleneck in IDEs (unless we are talking about C++...), especially with tools like tree-sitter or rust-analyzer. Furthermore parsing should be easily cacheable if performance is a concern. Going the other way, tools like gofmt etc already pretty much standardize a canonical serialization format for any particular AST. That being said, I'm fan of the concept of structure editing etc, so I'm still interested to see what they come up with.

pizza 5y ago

What I need for a metaprogramming tool is a code generator that uses CodeQL as input, then allows me to specify very terse AST manipulation/transformation functions, then generate my output code for me. The one real flaw about CodeQL imo is that they don’t let me generate code.. Like suppose I found a very nice library whose only fatal flaw was that it only wrote its output to files as opposed to general buffers/streams. And I had another library that was godawful but it had a very nice, modular, network streaming routine. I want this: from nice_library, awful_library select concat(nice_library_routine_upto_filwriting_entrypoint, awful_library_routine_networkIO_entrypoint) It’s likely gonna need more information to make it possible but this is the sketch of my intent In fact a gui like Scratch may be just the type of graphical user interface that would make this nice.. think colorful affordances that prevent you from writing a buggy codeql query and iterating pointlessly when everything could be visually typechecked. You could even call it Frankenstein :) Would that become closer to reality with this?

duped 5y ago

I like seeing raw ambition in these projects, my reading is that this is not "let's make a better IDE" but "let's fundamentally change how software tools interact". There are numerous projects that I have worked on that would benefit from semantic code storage (and have used LSIF[1], which only goes so far). I think the demos are neat but really don't hit on the massive productivity increases, this is the stuff that people would throw money at : - Language-aware-yet-independent and polyglot build systems that don't need new frontends for new languages. - A CI platform that doesn't need configuration - Semantic fuzzing [1] https://lsif.dev/ https://lsif.dev/

abnercoimbre 5y ago

If you want to see their demo that made a pretty big splash it's available on Handmade Seattle. [0] Usual disclosure that I am the organizer for these things. [0] https://media.handmade-seattle.com/dion-systems https://media.handmade-seattle.com/dion-systems

Cilvic 5y ago

I think this is pretty cool. I've been thinking about this "tree" in context of presentations which are structure + content/facts + layout/visualization. Basically all presentation content is just a big tree/hierarchy of facts (text and numbers) and some applied layout. Many of the problems are similar: - you want to look at the problem on different levels "high level" story line, low level drilling into section 4.3.2, seamless zoom - you'd want to have different representations of the same information. During problem solving the information you are collecting is constantly growing and it's a back and forth between adding data, shaping it differently (lists become tables etc.) - easy entity/renaming, search and replace for facts + audit trails "where does the number come from" etc. - CSS / re-styling (why does it matter if important words are blue or red, that should just be a CSS theme you switch out) - there also exists "structs" which are structured entities like say "these are the 40 offices" the client has, and we want to represent them in various forms - ... I wonder if notion could be turned into a modern problem solving / result presentation maker. It's very flexible in surfacing the same content in different formats. More related to software development and their demo: Is any additional information in the tree, that is not /can not be in the text rendering. I get that the text rendering can be "loss-y" meaning that if it's badly rendered, information can be lost / the text has errors. But if you have the information, and you render it into a certain text representation that is error free. Then the program should also be able to parse it back into a tree? What are the situations were storing it in "tree form" are actually superior? Isn't the "tree" just a more compressed, less error prone and stricter text?