7 ms·

All programming is visual, think about it. Text is visual. I'm not just pointing out a semantic discrepancy here. Text is a specific method of encoding meaning

All programming is visual, think about it. Text is visual. I'm not just pointing out a semantic discrepancy here. Text is a specific method of encoding meaning and processes in a visual way. In short, textual programming is just a subset of visual. People think of the dichotomy as a duality of text vs. visual, it is not. Text just has features people tend to like: A Fixed line based structure. Isomorphism with the english vocabulary... Who says other visual programs can't have this either? The issue with a lot of these "visual programs" is that they also represent a subset of Visual Programming similar to how text is just a tiny subset of visual programming. What you are in fact seeing with most "visual programs" is mostly programs represented as graphs. The real dichotomy people are discussing is not text vs. visual but text vs. graph. Whenever I see something like labView or something labeled as a "visual program" it's always a program represented with a series of nodes and lines, aka a graph. Hopefully by illustrating the reality of what's going on, people can see beyond representing programs as just text or just graphs. There are other ways to represent modules and processes.

DonHopkins 6y ago
Well put! Text is graphics, but technology has historically limited how it looks and what you can do with it. The idea of text with control characters like "carriage return" and "newline", fixed width text, limited length lines, and text editors that wrap words when the cursor gets to the edge of the screen, and terminals with multiple lines that scroll, all incrementally evolved out of emulating punched card machines. And now we emulate VT100s emulating punched card machines. But it doesn't have to be that way. Here's a great piece of exploratory research that imagines how things could be different, by an amazing visionary visual thinker and artist, Scott Kim: "Viewpoint" is part of Scott Kim's PhD dissertation that he did at Stanford in 1988 (with Donald Knuth as principle advisor). It was programmed in Cedar, and ran on a color Dorado workstation at Xerox PARC. https://www.youtube.com/watch?v=9G0r7jL3xl8 https://www.youtube.com/watch?v=9G0r7jL3xl8 >Demo and explanation of Viewpoint, a computer system that imagines how computers might be different had they been designed by visual thinkers instead of mathematicians. Caution: this is basic research, not a proposal for a practical piece of software. Part of my PhD Dissertation at Stanford University in 1988. At 12:22, he starts with a blank screen, and performs a "visual boot", to use Viewpoint to build itself. http://www.scottkim.com/ http://www.scottkim.com/ https://en.wikipedia.org/wiki/Scott_Kim https://en.wikipedia.org/wiki/Scott_Kim https://visualraccoon.wordpress.com/2013/08/14/the-viewpoint-thesis/ https://visualraccoon.wordpress.com/2013/08/14/the-viewpoint... This is a visualraccoon Perspective on Scott Kim‘s very graphic and revolutionary exploration, Viewpoint: Toward a Computer for Visual Thinker. What would it be like to go back to visual first principles and take a fresh look at graphic user interfaces? The Viewpoint Thesis is that a small number of pixel manipulation primitives can be defined such that if they are bound to keyboard and mouse actions it is then possible to build a simple text-graphic editor by drawing it, and that that editor can be used to draw-build itself.* The Viewpoint Thesis & Editor is part of a larger project founded on the hypothesis that: “Only by treating the screen itself as a first class citizen will we be able to build computers that are truly for visual thinkers.” Scott, 1987. This project includes building visual programming languages for such thinkers. Viewpoint: Toward a Computer for Visual Thinkers http://www.scottkim.com.previewc40.carrierzone.com/viewpoint/viewpoint-dissertation.pdf http://www.scottkim.com.previewc40.carrierzone.com/viewpoint... http://www.scottkim.com.previewc40.carrierzone.com/viewpoint/index.html http://www.scottkim.com.previewc40.carrierzone.com/viewpoint... When I started my PhD project in 1981, the IBM PC was brand new, and it would be years before the Mac and Microsoft Windows would appear. Nonetheless I was familiar with graphical user interfaces because of my internship at nearby Xerox PARC, the visionary research center that spawned, among other things, the laser printer and the bitmapped display. I was well-acquainted with Alan Kay's vision of a dynabook (he envisioned the notebook computer in 1975), and understood the power of graphic user interfaces and graphic tools like paint programs to bring the power of computers to artists and visual thinkers. At the time I was part of the digital typography program within the Stanford computer science department, which built computer programs for digital typeface designers. I was also enthralled by the Visual Thinking course at Stanford, which taught engineers in the product design program how to think visually. It struck me as odd, and deeply wrong, that we were building tools for visual artists in a programming language that was utterly symbolic, and lacking in visual sophistication. I yearned for a programming language that had the same visual clarity that graphic user interfaces had. So I set about wondering what a visual programming language might look like. If computers had been invented by artists and visually oriented people, instead of by mathematicians and engineers, how might they write programs? It seemed to me an important question, but one that hardly bothered most computer scientists. I read about a few attempts to build visual programming languages, and decided there was something fundamental I needed to understand. My journey took me deep into the foundations of computer science, where I asked fundamental questions like “what is programming” and “what is a user interaction” — questions that often get passed over in computer science (any definition of “programming” that starts with “a sequence of symbols that…” is not deep enough to encompass visual programming languages). I never did build a visual programming language, but I did figure out a fundamental idea (things get interesting when the user and the computer share the same mental model of data), and built a rudimentary visual editor that demonstrated my ideas. I’m posting my dissertation and the accompanying video demo to re-open the conversation. What in my dissertation is interesting to you? What other work do you know of along these lines? Do you know of foundational research in interaction design and visual programming? And most importantly, what would be a good next step to push the work forward? (My research stalled for lack of a juicy application domain.) Please email me your thoughts. I'd love to hear them.
- crimsonalucard 6y ago
  I do not agree with the freeform nature of viewpoint. The possibilities are boundless but you have to temper the boundlessness with the fact that our brains are biased and specialized to work in certain ways. Case in point can you communicate your entire post in terms of pictures? Is there any form of non linear diagramming you can do on a 2D canvas that can deliver your point better than text? If I give you the freedom of a 2D canvas to convey your point can you convey it better than the restricted form of text? No. In fact text was the best way to communicate your point. Restriction and lack of freedom are often more effective tools than ones that are completely unrestricted and boundless. It doesn't matter whether you're a visual thinker. When it comes to parsing and conveying complex information our brains have a dedicated module for parsing information that comes in as a linear and serial string of symbols. If you can talk, listen read or write than you can be exceptional at writing programs that do this. Your brain is biologically biased to operate this way. That being said if I show you a picture of a dog, you will learn more about what that dog looks like than any amount of textual description. There is no denying that certain things are communicated better visually and other things are communicated serially. It's hard to delineate axioms that communicate exactly what is better communicated visually and what is better communicated textually. I can only talk in terms of examples: Your ideas on visual programming is better communicated through text, what your car or face looks like is better communicated through pictures. One thing that has worked throughout history is a combination of text and pictures. Powerpoint. A linear slide show where each slide delivers serial text and pictures in parallel to the human brain. The future of visual programming, if it occurs at all, will ultimately be some form of a combination of diagrams and restricted serial strings of symbols following the same principles as powerpoint. I don't believe it has anything to do with foundational computer science but rather the language must be built around the biases and limitations of the human brain. We build biased textual languages because our brains are partly biased towards text. I have an idea about what that would look like and how that would work in terms of maintaining programmer productivity. Also I didn't see your dissertation (assuming your name is Don Hopkins) in your post.
  - DonHopkins 6y ago
    >Is there any form of non linear diagramming you can do on a 2D canvas that can deliver your point better than text? I could write a JavaScript program just for you, that drew blinking animated sparkly text in a canvas which said "WATCH THE VIDEO I POSTED", but there would still be no point to that, since you can lead a horse to water, but you can't make him drink. Because if you actually watched the video, you would realize that Viewpoint is all about text. Nobody claimed it was "better than text", or that it was trying to get rid of text. You'd know that if you had watched the video. But you don't know that, so I can conclude that me simply writing text was not enough to get you to watch the video. >I have an idea about what that would look like and how that would work in terms of maintaining programmer productivity. That's very nice. And do you know how much that's worth? Nothing. Not only are ideas worthless without an implementation, but also ideas that you haven't even bothered to write down without implementing are worth even less, because they're a waste of everyone's time, and not tempered by reality or experience or empirical testing or iterative design and development or stakeholder requirements or user feedback. What point is there to telling me that you have an idea about something you haven't implemented, or even written down, or drawn a picture of? Or did you forget to provide a link to your own dissertation, a video, and a runnable demo? It's much better for you to spend your precious time doing difficult tedious things like writing down and implementing and testing your brilliant ideas yourself, instead of having so much fun talking about how great it would be if only you wrote it down for other people to read, payed somebody on Fiverr to draw a picture of your idea for you, or found some out-of-work but highly skilled programming with nothing better to do, who was willing to pair up with a starry-eyed "idea guy", and work night and day for free, on your sincere verbal promise of equity and exposure, implementing your grandiose but unwritten ideas, in the hopes that you don't get angry at them for not precisely interpreting and efficiently engineering your brilliant vision correctly, or suddenly change your mind once you see what they actually implemented that did not perfectly live up to your idealistic expectations and nebulous handwaving. Good luck with that! Let me know when it ships.

DonHopkins 6y ago

Well put! Text is graphics, but technology has historically limited how it looks and what you can do with it. The idea of text with control characters like "carriage return" and "newline", fixed width text, limited length lines, and text editors that wrap words when the cursor gets to the edge of the screen, and terminals with multiple lines that scroll, all incrementally evolved out of emulating punched card machines. And now we emulate VT100s emulating punched card machines. But it doesn't have to be that way. Here's a great piece of exploratory research that imagines how things could be different, by an amazing visionary visual thinker and artist, Scott Kim: "Viewpoint" is part of Scott Kim's PhD dissertation that he did at Stanford in 1988 (with Donald Knuth as principle advisor). It was programmed in Cedar, and ran on a color Dorado workstation at Xerox PARC. https://www.youtube.com/watch?v=9G0r7jL3xl8 https://www.youtube.com/watch?v=9G0r7jL3xl8 >Demo and explanation of Viewpoint, a computer system that imagines how computers might be different had they been designed by visual thinkers instead of mathematicians. Caution: this is basic research, not a proposal for a practical piece of software. Part of my PhD Dissertation at Stanford University in 1988. At 12:22, he starts with a blank screen, and performs a "visual boot", to use Viewpoint to build itself. http://www.scottkim.com/ http://www.scottkim.com/ https://en.wikipedia.org/wiki/Scott_Kim https://en.wikipedia.org/wiki/Scott_Kim https://visualraccoon.wordpress.com/2013/08/14/the-viewpoint-thesis/ https://visualraccoon.wordpress.com/2013/08/14/the-viewpoint... This is a visualraccoon Perspective on Scott Kim‘s very graphic and revolutionary exploration, Viewpoint: Toward a Computer for Visual Thinker. What would it be like to go back to visual first principles and take a fresh look at graphic user interfaces? The Viewpoint Thesis is that a small number of pixel manipulation primitives can be defined such that if they are bound to keyboard and mouse actions it is then possible to build a simple text-graphic editor by drawing it, and that that editor can be used to draw-build itself.* The Viewpoint Thesis & Editor is part of a larger project founded on the hypothesis that: “Only by treating the screen itself as a first class citizen will we be able to build computers that are truly for visual thinkers.” Scott, 1987. This project includes building visual programming languages for such thinkers. Viewpoint: Toward a Computer for Visual Thinkers http://www.scottkim.com.previewc40.carrierzone.com/viewpoint/viewpoint-dissertation.pdf http://www.scottkim.com.previewc40.carrierzone.com/viewpoint... http://www.scottkim.com.previewc40.carrierzone.com/viewpoint/index.html http://www.scottkim.com.previewc40.carrierzone.com/viewpoint... When I started my PhD project in 1981, the IBM PC was brand new, and it would be years before the Mac and Microsoft Windows would appear. Nonetheless I was familiar with graphical user interfaces because of my internship at nearby Xerox PARC, the visionary research center that spawned, among other things, the laser printer and the bitmapped display. I was well-acquainted with Alan Kay's vision of a dynabook (he envisioned the notebook computer in 1975), and understood the power of graphic user interfaces and graphic tools like paint programs to bring the power of computers to artists and visual thinkers. At the time I was part of the digital typography program within the Stanford computer science department, which built computer programs for digital typeface designers. I was also enthralled by the Visual Thinking course at Stanford, which taught engineers in the product design program how to think visually. It struck me as odd, and deeply wrong, that we were building tools for visual artists in a programming language that was utterly symbolic, and lacking in visual sophistication. I yearned for a programming language that had the same visual clarity that graphic user interfaces had. So I set about wondering what a visual programming language might look like. If computers had been invented by artists and visually oriented people, instead of by mathematicians and engineers, how might they write programs? It seemed to me an important question, but one that hardly bothered most computer scientists. I read about a few attempts to build visual programming languages, and decided there was something fundamental I needed to understand. My journey took me deep into the foundations of computer science, where I asked fundamental questions like “what is programming” and “what is a user interaction” — questions that often get passed over in computer science (any definition of “programming” that starts with “a sequence of symbols that…” is not deep enough to encompass visual programming languages). I never did build a visual programming language, but I did figure out a fundamental idea (things get interesting when the user and the computer share the same mental model of data), and built a rudimentary visual editor that demonstrated my ideas. I’m posting my dissertation and the accompanying video demo to re-open the conversation. What in my dissertation is interesting to you? What other work do you know of along these lines? Do you know of foundational research in interaction design and visual programming? And most importantly, what would be a good next step to push the work forward? (My research stalled for lack of a juicy application domain.) Please email me your thoughts. I'd love to hear them.

crimsonalucard 6y ago
I do not agree with the freeform nature of viewpoint. The possibilities are boundless but you have to temper the boundlessness with the fact that our brains are biased and specialized to work in certain ways. Case in point can you communicate your entire post in terms of pictures? Is there any form of non linear diagramming you can do on a 2D canvas that can deliver your point better than text? If I give you the freedom of a 2D canvas to convey your point can you convey it better than the restricted form of text? No. In fact text was the best way to communicate your point. Restriction and lack of freedom are often more effective tools than ones that are completely unrestricted and boundless. It doesn't matter whether you're a visual thinker. When it comes to parsing and conveying complex information our brains have a dedicated module for parsing information that comes in as a linear and serial string of symbols. If you can talk, listen read or write than you can be exceptional at writing programs that do this. Your brain is biologically biased to operate this way. That being said if I show you a picture of a dog, you will learn more about what that dog looks like than any amount of textual description. There is no denying that certain things are communicated better visually and other things are communicated serially. It's hard to delineate axioms that communicate exactly what is better communicated visually and what is better communicated textually. I can only talk in terms of examples: Your ideas on visual programming is better communicated through text, what your car or face looks like is better communicated through pictures. One thing that has worked throughout history is a combination of text and pictures. Powerpoint. A linear slide show where each slide delivers serial text and pictures in parallel to the human brain. The future of visual programming, if it occurs at all, will ultimately be some form of a combination of diagrams and restricted serial strings of symbols following the same principles as powerpoint. I don't believe it has anything to do with foundational computer science but rather the language must be built around the biases and limitations of the human brain. We build biased textual languages because our brains are partly biased towards text. I have an idea about what that would look like and how that would work in terms of maintaining programmer productivity. Also I didn't see your dissertation (assuming your name is Don Hopkins) in your post.
- DonHopkins 6y ago
  >Is there any form of non linear diagramming you can do on a 2D canvas that can deliver your point better than text? I could write a JavaScript program just for you, that drew blinking animated sparkly text in a canvas which said "WATCH THE VIDEO I POSTED", but there would still be no point to that, since you can lead a horse to water, but you can't make him drink. Because if you actually watched the video, you would realize that Viewpoint is all about text. Nobody claimed it was "better than text", or that it was trying to get rid of text. You'd know that if you had watched the video. But you don't know that, so I can conclude that me simply writing text was not enough to get you to watch the video. >I have an idea about what that would look like and how that would work in terms of maintaining programmer productivity. That's very nice. And do you know how much that's worth? Nothing. Not only are ideas worthless without an implementation, but also ideas that you haven't even bothered to write down without implementing are worth even less, because they're a waste of everyone's time, and not tempered by reality or experience or empirical testing or iterative design and development or stakeholder requirements or user feedback. What point is there to telling me that you have an idea about something you haven't implemented, or even written down, or drawn a picture of? Or did you forget to provide a link to your own dissertation, a video, and a runnable demo? It's much better for you to spend your precious time doing difficult tedious things like writing down and implementing and testing your brilliant ideas yourself, instead of having so much fun talking about how great it would be if only you wrote it down for other people to read, payed somebody on Fiverr to draw a picture of your idea for you, or found some out-of-work but highly skilled programming with nothing better to do, who was willing to pair up with a starry-eyed "idea guy", and work night and day for free, on your sincere verbal promise of equity and exposure, implementing your grandiose but unwritten ideas, in the hopes that you don't get angry at them for not precisely interpreting and efficiently engineering your brilliant vision correctly, or suddenly change your mind once you see what they actually implemented that did not perfectly live up to your idealistic expectations and nebulous handwaving. Good luck with that! Let me know when it ships.