5 ms·

Yeah it's definitely crazy! A few people are working on this from different angles. There is research https://harc.ycr.org/project/realtalk/ https://harc.ycr.or

Yeah it's definitely crazy! A few people are working on this from different angles. There is research https://harc.ycr.org/project/realtalk/ https://harc.ycr.org/project/realtalk/, side projects that explore Jupyter notebook like programming environments: https://www.maria.cloud/intro https://www.maria.cloud/intro, or https://www.runkit.com; https://www.runkit.com; my team and I are working on https://www.clay.run https://www.clay.run: making it easier to prototype by allowing developers to write code that is instantly running without setup or configuration and 'fork' other people's code to have your own running copy.

shalabhc 9y ago
Maria, clay and runkit look interesting - I like the quick iterations and incremental programming they allow. But you're still writing 'text' to manipulate 'data'. An example of something fundamentally different would be if you can make programs without text. I intentionally said make not write to avoid framing the discussion. For instance why write `(circle 10)` when you can instead just draw a circle and then perhaps define processes to manipulate it? I don't mean code generation - why even have text as the canonical form that describes computation processes?
- kareemamin 9y ago
  yes absolutely! I think things along the idea of programming by example like Flash Fill https://www.microsoft.com/en-us/research/video/flash-fill-flashextract/ https://www.microsoft.com/en-us/research/video/flash-fill-fl... or direct manipulation of graphics or other objects would be very interesting. I think in order to move away from text it would be important to find use-cases where other forms of describing computational processes would give the person an order of magnitude more leverage over doing the same thing in text. The idea of programming by example would make it so that testing skills are more valuable than breaking down a process into logical steps.
- muxator 9y ago
  Maybe I am old school, but the fact that text is easily version able and comparable means a lot to me. I use to code with two side-by-side windows, comparing the old and the new version of my program. It's a sort of fluid evolution where I can always be in control. It would be a hard time doing that via visual programming.
  - kareemamin 9y ago
    I think you are right. It's much easier to compare old and new versions of a program if they are text. But we also should remember that we developed tools like diff to work with programming in text. If instead we were using a different paradigm we could explore the equivalent of diff in that model. Merge conflicts in text are often not real logic conflicts but just re-ordering things in a text file, or formatting changes. If we used a projectional editor, one that treats code as an AST instead of just flat text like lamdu - https://github.com/lamdu/lamdu https://github.com/lamdu/lamdu we would need to create a new way to look at the evolution of changes in the code but it could be more meaningful than just looking at text. Also you could have a timeline that shows you what got changed instead of comparing two text files. There can be lots of ways to achieve the same goal, understanding what changed and how, than just diffing two text files.
  - shalabhc 9y ago
    > text is easily version able and comparable means a lot to me. Don't you really want to do more meaningful comparisons though? I think text is a poor form for versioning and comparison because you always have to mentally extrapolate the textual diff (lines added/deleted) to the syntactic diff (functions added/deleted/modified) to the diff in effect (which logic in which systems is modified). There is no reason other forms of programs cannot also do diffs, that could be more useful. Visual does't have to mean flowchart boxes. Even a tree structure might be incrementally better than plain text.
    - dragonwriter 9y ago
      Syntax-aware diff for text isn't hard, but it's just not usually worth the effort because line-oriented text diff is good enough and more generally applicable.
  - nikofeyn 9y ago
    i think that's a little off. there is no reason why visual programming cannot be version-able and comparable, and in some cases it is. labview has some facilities for this, however, i will concede they could be better. but there is no inherent reason why text is better at this than visual programs. it's just that people haven't done it, which is the issue. everyone just assumes text is the natural way to interact with a computer, but yet most people forget they went through a lot of training, formal or informal, to get to that perspective. and one could argue that comparing text is not as good as comparing the differences between visual programs because the latter actually contains structure and relationships of how things are called more readily. what you describe with two side-by-side windows of the same VI (i.e., function or code block or whatever) is exactly something that i have done with labview. a better visual comparator is not beyond reach.
- dragandj 9y ago
  It is probably the same reason as why books (text) are still the main medium. Movies may be more popular entertainment, and comics might be preferred by some demographics, but everybody write something, and everybody read something, being it a facebook post, a tweet, a novel or a 10-tome encyclopaedia. Nobody says that books kinda suck and should be replaced by pictographs on papyri, but somehow many programmers at least have a phase when they think that having to program by drawing shapes could somehow be easier than writing textual code...
  - shalabhc 9y ago
    > books (text) are still the main medium Geometry books don't use text as the only form of communication. Neither do books about architecture, mechanical engineering, anatomy, circuit design, etc. Imagine a world where you used only text to convey all these ideas and their relationships. Computer programming today a bit like that. Why should we assume that pure text is the best and only suitable form for representing automatic computation?
- dragonwriter 9y ago
  > why even have text as the canonical form that describes computation processes? Because before you get to machine code, you’ll want to transform to an expression tree, and text is an excellent accessible way to represent that. The text form might be an IR with various graphical or other non-text (and possibly some text) source languages, though.
- empath75 9y ago
  https://vcvrack.com https://vcvrack.com Here’s an example of visual programming. All you’re really doing there is chaining together functions and tuning parameters, but instead of typing, you’re dragging wires around and twiddling knobs.
- ianbicking 9y ago
  I've yet to encounter a more precise and usable input mechanism than the keyboard. I've played a bit with drag-and-drop programming environments, and they've gotten a lot better, but it's hard to be as expressive and it's hard to be as efficient. That said, the input in The Mother Of All Demos (https://en.wikipedia.org/wiki/The_Mother_of_All_Demos https://en.wikipedia.org/wiki/The_Mother_of_All_Demos) maybe did improve over the keyboard, what with multi-button mice and single-hand cording keyboards. But it's not a setup people have engaged with much since then. You can also imagine more effort into displaying code in a less textual manner, even if the keyboard remains the primary input. There are even fonts that do this in a small way (turning >= into ≥, for instance). But plain text has a nice advantage over these: it's always clear how to reproduce what you see. There's no hidden information. There's no display bug that renders structure invisible (except perhaps bad indentation). Text can be trusted. The one visual overlay we use – syntax highlighting – is more like a checksum than an augmentation. Another possibility is to have a richer set of literals, like you suggest with a drawn circle. The drawn circle is not unlike any other object, it would have parameters, an in-memory and serialized representation. And it would have its own mini-editor of sorts. But how many constants do you need in a program? Not many in my experience. At first this made me think about JSX – which is modestly a more expressive form of JavaScript – but then I realized: what makes JSX special is that the code and HTML are mixed together. JSX and React are a kind of refutation of limited template systems. How do you draw a circle with size S and color C? I can express (circle 10) in concretely, but how can I express (circle (/ height 2))? The second expression is where programming gets interesting. Sketchpad (https://en.wikipedia.org/wiki/Sketchpad https://en.wikipedia.org/wiki/Sketchpad) is kind of interesting here, as my impression was that what made it special wasn't just the interactive graphics but that it was a constraint-based system, where the drawn figures could be interpreted and adapt to other environments, as they had both fixed and flexible sides. In a sense you were creating models, and those models were turned into concrete drawings. I think there's probably a better expression of programs than text, but it's going to require something more different than what we've thought of so far. My own inkling is that the Wolfram Language might have something in it. Maybe Symbolic Calculation (https://en.wikipedia.org/wiki/Symbolic_computation https://en.wikipedia.org/wiki/Symbolic_computation) is an aspect of it. Symbolic computation has the possibility of doing something and then telling the programmer why it did what it did. Like, imagine I have some program that makes a picture: (repeat (i 50) (transpose (color (circle (* i 0.5)) (- 1 (/ i 50)) 0 i) (* i 10) 0) I.e., it lays out a line of circles that grow in size and change from red to blue. Except I messed it up, I didn't get the blue argument to (color) correct. You can imagine a language where the circles weren't bitmaps, you were really making objects, and those objects know how and when they were made, they are just expansions of expressions. So I could inspect one of those overly-blue circles, and I can see the expression that made it, the value of i at the time it was made, and how each part of the expression evaluated as it formed the circle. That would be pretty nice! A graphical object is one example, but what if we could do that for all kinds of output? Maybe textual output is problem, not textual programs?

shalabhc 9y ago

Maria, clay and runkit look interesting - I like the quick iterations and incremental programming they allow. But you're still writing 'text' to manipulate 'data'. An example of something fundamentally different would be if you can make programs without text. I intentionally said make not write to avoid framing the discussion. For instance why write `(circle 10)` when you can instead just draw a circle and then perhaps define processes to manipulate it? I don't mean code generation - why even have text as the canonical form that describes computation processes?

kareemamin 9y ago
yes absolutely! I think things along the idea of programming by example like Flash Fill https://www.microsoft.com/en-us/research/video/flash-fill-flashextract/ https://www.microsoft.com/en-us/research/video/flash-fill-fl... or direct manipulation of graphics or other objects would be very interesting. I think in order to move away from text it would be important to find use-cases where other forms of describing computational processes would give the person an order of magnitude more leverage over doing the same thing in text. The idea of programming by example would make it so that testing skills are more valuable than breaking down a process into logical steps.
muxator 9y ago
Maybe I am old school, but the fact that text is easily version able and comparable means a lot to me. I use to code with two side-by-side windows, comparing the old and the new version of my program. It's a sort of fluid evolution where I can always be in control. It would be a hard time doing that via visual programming.
- kareemamin 9y ago
  I think you are right. It's much easier to compare old and new versions of a program if they are text. But we also should remember that we developed tools like diff to work with programming in text. If instead we were using a different paradigm we could explore the equivalent of diff in that model. Merge conflicts in text are often not real logic conflicts but just re-ordering things in a text file, or formatting changes. If we used a projectional editor, one that treats code as an AST instead of just flat text like lamdu - https://github.com/lamdu/lamdu https://github.com/lamdu/lamdu we would need to create a new way to look at the evolution of changes in the code but it could be more meaningful than just looking at text. Also you could have a timeline that shows you what got changed instead of comparing two text files. There can be lots of ways to achieve the same goal, understanding what changed and how, than just diffing two text files.
- shalabhc 9y ago
  > text is easily version able and comparable means a lot to me. Don't you really want to do more meaningful comparisons though? I think text is a poor form for versioning and comparison because you always have to mentally extrapolate the textual diff (lines added/deleted) to the syntactic diff (functions added/deleted/modified) to the diff in effect (which logic in which systems is modified). There is no reason other forms of programs cannot also do diffs, that could be more useful. Visual does't have to mean flowchart boxes. Even a tree structure might be incrementally better than plain text.
  - dragonwriter 9y ago
    Syntax-aware diff for text isn't hard, but it's just not usually worth the effort because line-oriented text diff is good enough and more generally applicable.
- nikofeyn 9y ago
  i think that's a little off. there is no reason why visual programming cannot be version-able and comparable, and in some cases it is. labview has some facilities for this, however, i will concede they could be better. but there is no inherent reason why text is better at this than visual programs. it's just that people haven't done it, which is the issue. everyone just assumes text is the natural way to interact with a computer, but yet most people forget they went through a lot of training, formal or informal, to get to that perspective. and one could argue that comparing text is not as good as comparing the differences between visual programs because the latter actually contains structure and relationships of how things are called more readily. what you describe with two side-by-side windows of the same VI (i.e., function or code block or whatever) is exactly something that i have done with labview. a better visual comparator is not beyond reach.
dragandj 9y ago
It is probably the same reason as why books (text) are still the main medium. Movies may be more popular entertainment, and comics might be preferred by some demographics, but everybody write something, and everybody read something, being it a facebook post, a tweet, a novel or a 10-tome encyclopaedia. Nobody says that books kinda suck and should be replaced by pictographs on papyri, but somehow many programmers at least have a phase when they think that having to program by drawing shapes could somehow be easier than writing textual code...
- shalabhc 9y ago
  > books (text) are still the main medium Geometry books don't use text as the only form of communication. Neither do books about architecture, mechanical engineering, anatomy, circuit design, etc. Imagine a world where you used only text to convey all these ideas and their relationships. Computer programming today a bit like that. Why should we assume that pure text is the best and only suitable form for representing automatic computation?
dragonwriter 9y ago
> why even have text as the canonical form that describes computation processes? Because before you get to machine code, you’ll want to transform to an expression tree, and text is an excellent accessible way to represent that. The text form might be an IR with various graphical or other non-text (and possibly some text) source languages, though.
empath75 9y ago
https://vcvrack.com https://vcvrack.com Here’s an example of visual programming. All you’re really doing there is chaining together functions and tuning parameters, but instead of typing, you’re dragging wires around and twiddling knobs.
ianbicking 9y ago
I've yet to encounter a more precise and usable input mechanism than the keyboard. I've played a bit with drag-and-drop programming environments, and they've gotten a lot better, but it's hard to be as expressive and it's hard to be as efficient. That said, the input in The Mother Of All Demos (https://en.wikipedia.org/wiki/The_Mother_of_All_Demos https://en.wikipedia.org/wiki/The_Mother_of_All_Demos) maybe did improve over the keyboard, what with multi-button mice and single-hand cording keyboards. But it's not a setup people have engaged with much since then. You can also imagine more effort into displaying code in a less textual manner, even if the keyboard remains the primary input. There are even fonts that do this in a small way (turning >= into ≥, for instance). But plain text has a nice advantage over these: it's always clear how to reproduce what you see. There's no hidden information. There's no display bug that renders structure invisible (except perhaps bad indentation). Text can be trusted. The one visual overlay we use – syntax highlighting – is more like a checksum than an augmentation. Another possibility is to have a richer set of literals, like you suggest with a drawn circle. The drawn circle is not unlike any other object, it would have parameters, an in-memory and serialized representation. And it would have its own mini-editor of sorts. But how many constants do you need in a program? Not many in my experience. At first this made me think about JSX – which is modestly a more expressive form of JavaScript – but then I realized: what makes JSX special is that the code and HTML are mixed together. JSX and React are a kind of refutation of limited template systems. How do you draw a circle with size S and color C? I can express (circle 10) in concretely, but how can I express (circle (/ height 2))? The second expression is where programming gets interesting. Sketchpad (https://en.wikipedia.org/wiki/Sketchpad https://en.wikipedia.org/wiki/Sketchpad) is kind of interesting here, as my impression was that what made it special wasn't just the interactive graphics but that it was a constraint-based system, where the drawn figures could be interpreted and adapt to other environments, as they had both fixed and flexible sides. In a sense you were creating models, and those models were turned into concrete drawings. I think there's probably a better expression of programs than text, but it's going to require something more different than what we've thought of so far. My own inkling is that the Wolfram Language might have something in it. Maybe Symbolic Calculation (https://en.wikipedia.org/wiki/Symbolic_computation https://en.wikipedia.org/wiki/Symbolic_computation) is an aspect of it. Symbolic computation has the possibility of doing something and then telling the programmer why it did what it did. Like, imagine I have some program that makes a picture: (repeat (i 50) (transpose (color (circle (* i 0.5)) (- 1 (/ i 50)) 0 i) (* i 10) 0) I.e., it lays out a line of circles that grow in size and change from red to blue. Except I messed it up, I didn't get the blue argument to (color) correct. You can imagine a language where the circles weren't bitmaps, you were really making objects, and those objects know how and when they were made, they are just expansions of expressions. So I could inspect one of those overly-blue circles, and I can see the expression that made it, the value of i at the time it was made, and how each part of the expression evaluated as it formed the circle. That would be pretty nice! A graphical object is one example, but what if we could do that for all kinds of output? Maybe textual output is problem, not textual programs?