4 ms·

There's something I think could be game-changing on the front-end of programming languages: Just (as in 'only') change the (internal) representation of source

There's something I think could be game-changing on the front-end of programming languages: Just (as in 'only') change the (internal) representation of source code from character sequences to something that doesn't need a complex parsing process. In other words, eliminating the concept of syntax from language design, while keeping other language properties intact (could even use e.g. an LLVM back-end), allowing whoever desires it to write their own editors etc. as always. (This is different from eliminating text: IMO, the visualization should remain primarily text. I say to eliminate syntax in the sense that the text 'visualization' no longer has a connection with semantics, nor any grammatical constraints.) The reason I say 'just' change that: there are other systems structured to render AST-like structures (rather than parsing), but they are integrated with particular software systems. We need a general purpose alternate format with a similar independence of any project that 'character sequences' have for current source code. ------ I have a proposal for one that's very simple and general: represent languages as graphs of language constructs, and represent particular programs as paths through a language graph. More details at [1]. Making this substitution we can: throw out parsing, allow easy customization of language appearance (including things that would traditionally fall under 'syntax'), have much easier access to language insight (source representation is directly in terms of a language definition), give far easier to access to experimenting with language UX ideas, and more. I'd love to hear thoughtful feedback on this. I will likely invest a solid amount of time/effort into building a prototype before long, so if someone can spot a potential weakness that I've missed, I'd be forever grateful. (That said, please attempt at least a small amount of charity in reading: I wrote this in 2015 and most all I've gotten is people replying, "visual programming doesn't work" —even though it's expressly not advocating visual programming. I believe I really am talking about something fairly new here: and I am more than willing to listen if I'm mistaken. I have too many projects already; I have no need for this to work out. I just haven't been able to disprove it as... not being what it appears to be, and so I've felt an obligation to see it through for a long time.) [1] http://westoncb.blogspot.com/2015/06/how-to-make-view-independent-program.html http://westoncb.blogspot.com/2015/06/how-to-make-view-indepe...

chubot 7y ago
I skimmed your blog post, and I'm not quite sure I understand how it will make editing easier. It seems more complex to me, although that might be because it's unfamiliar. If you want some intelligent counterpoint, here is a post by the designer of Rust on why text is a good representation for programs: Always Bet On Text https://graydon2.dreamwidth.org/193447.html https://graydon2.dreamwidth.org/193447.html As someone who's working on a programming language, and has implemented a few DSLs and many parsers, I tend to agree with him. Parsing is a pain in the ass, but I think it can made easier. The difference between text and structured data is "just" parsing, so I don't feel that switching to structured data can be a real paradigm shift. If there was a paradigm shift to be found, then it would have been implemented already by converting text to structured data, and operating on that data. And this of course already happens in dozens of IDEs, and that's great, but it already happened. And while I have seen people be fantastically productive in IDEs, a lot of the best programmers I've seen also use emacs/vim. But your ideas do seem more fleshed out than most, and I think that developing a prototype is the only way to make them more concrete, and more clearly see the benefits and costs. I do think there are benefits to making structured data the primary format -- I just think they are outweighed by the costs. Graydon's post outlined a bunch of things you lose if you abandon text.
- westoncb 7y ago
  I can’t give a full reply atm, but briefly I’d say: yes, it’s more complex than character sequences but that complexity (which isn’t much) could be shifted to a lib once for any number of languages, as opposed to needing to parse every language uniquely.
  - chubot 7y ago
    That's true, but what I'm saying is that parsing is not a bottleneck in programming tools. Parsing every language uniquely is a lot of work. But it is doable and more or less "done". After that you have a whole bunch of other problems to solve (i.e. all the other things an IDE or debugger does), and that's where the real work is. You can get rid of parsing by making structured data the primary format rather than the secondary format. But then you introduce a whole host of other problems. For example, how do you 'git merge'? You can't even collaborate anymore because merging is a textual algorithm. You would have to write a new merge algorithm for every single programming language, because each one has its own structured data format. So basically you get a slight benefit by removing text, in exchange for a huge cost. BTW here are some posts I've written about parsing: http://www.oilshell.org/blog/tags.html?tag=parsing#parsing http://www.oilshell.org/blog/tags.html?tag=parsing#parsing Again, parsing is a pain in the ass, but I believe it can be made easier, and it's not a bottleneck in any case. In my mind, the biggest bottlenecks in programming are: 1) Understanding the problem well enough to formulate the solution, whether it's in a programming language, math, or a visual syntax. 2) "Size is code's worst enemy". Many programming languages are inappropriate to express the problems they are solving. The code becomes unmanageably large pretty quickly. A million lines of C++ code is not a great way to express a solution to anything, yet it's the state of the art for vital projects like LLVM and Clang, Word, Photoshop, etc. Both of these are 1000x bigger problems than parsing IMO. I'm not trying to criticize your specific idea, but I have heard the general idea to "get rid of text" several times, including more than once on this thread ... so that's my general rebuttal, if you're interested in a reasoned critique.

chubot 7y ago

I skimmed your blog post, and I'm not quite sure I understand how it will make editing easier. It seems more complex to me, although that might be because it's unfamiliar. If you want some intelligent counterpoint, here is a post by the designer of Rust on why text is a good representation for programs: Always Bet On Text https://graydon2.dreamwidth.org/193447.html https://graydon2.dreamwidth.org/193447.html As someone who's working on a programming language, and has implemented a few DSLs and many parsers, I tend to agree with him. Parsing is a pain in the ass, but I think it can made easier. The difference between text and structured data is "just" parsing, so I don't feel that switching to structured data can be a real paradigm shift. If there was a paradigm shift to be found, then it would have been implemented already by converting text to structured data, and operating on that data. And this of course already happens in dozens of IDEs, and that's great, but it already happened. And while I have seen people be fantastically productive in IDEs, a lot of the best programmers I've seen also use emacs/vim. But your ideas do seem more fleshed out than most, and I think that developing a prototype is the only way to make them more concrete, and more clearly see the benefits and costs. I do think there are benefits to making structured data the primary format -- I just think they are outweighed by the costs. Graydon's post outlined a bunch of things you lose if you abandon text.

westoncb 7y ago
I can’t give a full reply atm, but briefly I’d say: yes, it’s more complex than character sequences but that complexity (which isn’t much) could be shifted to a lib once for any number of languages, as opposed to needing to parse every language uniquely.
- chubot 7y ago
  That's true, but what I'm saying is that parsing is not a bottleneck in programming tools. Parsing every language uniquely is a lot of work. But it is doable and more or less "done". After that you have a whole bunch of other problems to solve (i.e. all the other things an IDE or debugger does), and that's where the real work is. You can get rid of parsing by making structured data the primary format rather than the secondary format. But then you introduce a whole host of other problems. For example, how do you 'git merge'? You can't even collaborate anymore because merging is a textual algorithm. You would have to write a new merge algorithm for every single programming language, because each one has its own structured data format. So basically you get a slight benefit by removing text, in exchange for a huge cost. BTW here are some posts I've written about parsing: http://www.oilshell.org/blog/tags.html?tag=parsing#parsing http://www.oilshell.org/blog/tags.html?tag=parsing#parsing Again, parsing is a pain in the ass, but I believe it can be made easier, and it's not a bottleneck in any case. In my mind, the biggest bottlenecks in programming are: 1) Understanding the problem well enough to formulate the solution, whether it's in a programming language, math, or a visual syntax. 2) "Size is code's worst enemy". Many programming languages are inappropriate to express the problems they are solving. The code becomes unmanageably large pretty quickly. A million lines of C++ code is not a great way to express a solution to anything, yet it's the state of the art for vital projects like LLVM and Clang, Word, Photoshop, etc. Both of these are 1000x bigger problems than parsing IMO. I'm not trying to criticize your specific idea, but I have heard the general idea to "get rid of text" several times, including more than once on this thread ... so that's my general rebuttal, if you're interested in a reasoned critique.