16 ms·

Maybe visual programming is the answer, maybe not

136 points by mpweiher 7y ago

fermienrico 7y ago
I think it would be great to do the reverse - take existing programs and map out the calls in a graph structure that can be visually displayed. Hardest part of looking at existing code base is to "mentally map" the call stacks and overtime, atleast for the way my brain operates, I tend to build a 2d map in my mind. It doesn't have concrete positions for blocks (functions), but there is a sense of structure in the brain that persists. I can come back to the code base and it is evoked automatically. We need visual tools for examining codebases just like we have tools for examining database schemas. Perhaps its not that simple though.
- atupis 7y ago
  It would be pretty cool when you zoom out first you see methods of how they are linked. Then classes and how they linked together.
- TsiCClawOfLight 7y ago
  Have you tried sourcetrail?
- wvenable 7y ago
  Having worked in actual visual programming environments I'd argue that even the most trivial applications are too complicated to mapped out visually. There just isn't enough visual resolution in a screen, or for that matter, in your eye to render it sufficiently. Zoom out and lose detail. Zoom in and use lose context. I've seen programs printed out into huge graphs covering entire boardroom tables and I can assure you nobody benefited from the display. Ironically the convoluted programs that people think this would be a solution for make for the most disgusting and hard to follow diagrams.
  - som33 7y ago
    It's just a matter of a lot of research. I've been thinking for a long time that now that we have tonnes of memory and cpu power, its time to add hardware that keeps track of stuff and make things easier to understand. For instance when you look at hex in a file, you see a bunch of numbers, but it would be nice if there was some inference and cloud database of commonly used tropes/functions and could infer concepts, ideas of what the programmer was thinking. Currently with code, you get someones thoughts that are then translated into a language, then are then translated into machine code. It would be nice not to lose information regarding what does what in human readable language, I understand for performance reasons why programming languages as they currently are do what they do. But c and c++ were really designed when hardware and memory was expensive. So they are "too the metal" in terms of their model. I think reimplementing a basic machine, aka actual hardware design has to be done at the same time as a compiler is made, and work out the theory concepts behind it. The fragility of von neuman machines and their code has always bugged me, the machine can only blindly do what the hardware is designed to do, since hardware is all about speed, you want to minimize information, but for coding, it would make sense to have machines developed just for development and have "light code" for when you release it out into the wild. Either way I think there is plenty of innovation left, it's just a lot of research at the hardware and compiler level so that you have base foundations and abstractions to make real innovations in understanding how programs behave. One of the massive issues is not being able to understand cause and effect in programming which leads to bugs, because the brain can't make easy models of how code or algorithms will behave.
  - vlovich123 7y ago
    I found the reverse. I had to do a tech debt cleanup of the code and found that doing an abstract diagram made it much easier to map out the interrelation between all the classes which was hard to keep in my head all at once. Also made it easier to communicate to others visually why it was spaghetti & easy to create a diagram about how I'd like to clean it up. I used PlantUML because what a lot of text -> visual things miss is that they don't provide the necessary knobs to tune the level of detail so doing the diagram by hand was a necessity. There may be multiple ways to look at something - sequence graphs, data flows - and additionally you don't want to capture all the details in the visualization, you want to be able to cut-off the diagrams at choke points where they're not helpful (e.g. I don't care about package X in the code) which is what all visualization packages try to do. This is directly analogous to geographic maps. Imagine if you viewed the map for a state/province whatever & had the local road network detailed. That map would be unusable & discarded quickly. Diagramming tools need to be able to take text & generate a visualization that you can zoom into/out of & lose/gain the appropriate level of detail necessary, be able to switch between different aspects (like you can with street view vs regular vs satellite, vs topography). I think the reason we don't have equivalent tools for programming is three-fold: 1. There's a lot of popular languages. Maintaining support for parsing & mapping that to the visualization abstraction is going to be a lot of work. Probably manageable though. 2. As engineers we tend to think in either solving the problem or not. No visualization is going to be good as a hand-drawn diagram in the moment nor is it probably good enough (at least the first version) as a vehicle for making the modification. 3. There's lot of type erasure/dynamism that's impossible to capture in code analysis (e.g. type-erased callbacks like `std::function`, `std::any`). That makes the visualization incorrect or cause it to fail prematurely.
- fragmede 7y ago
  There is an absolute dearth of tools, especially modern ones, in this space, but the tools to generate call graphs have existed for quite a while. Egypt will parse output from gcc and output a dot file for gnuplot/Graphviz. There may also be D3.js support in places. I agree we need visual tools, especially when trying to maintain a large codebase, but only the largest companies can sponsor development of an in-house tool for their codebase, and many of them won't. Maintaining code isn't sexy and doesn't make for new features to sell.
- solidasparagus 7y ago
  We look at compute graphs in deep learning, but it isn't very valuable because the graphs are so complicated. Among other issues, it's difficult to create a visualization of a compute graph that represents the time dimension well.
marktangotango 7y ago
This article provides a good overview of the problems and pitfalls of visual programming languages and environments and why their perceived benefits have not been realized up to now. I'd add one more. Cobol has gotten a lot of attention lately and although not "visual" in any sense whatsoever, had the goal of making programming accessible to managers and business users by adopting paragraphs and sentences to add structure to programs. Which is great and fine for trivial examples, but in practice the 25,000+ LOC real programs (and thousands of these in typical systems) were/are inscrutable to everyone but the specialist. The inherent problem is the lack of primitives to build abstractions to "move the langauge toward the problem domain". These primitives can be first class functions, objects, macros, etc [1]. Visual programming environments I've seen also fail to provide any such mechanisms as this article supports. [1] http://www.paulgraham.com/avg.html http://www.paulgraham.com/avg.html
kens 7y ago
The model of a program as a linear file of 80-column text is a historical accident based on IBM mainframes in the 1950s, and the decks of punch cards they read. It seems like there must be a way to treat text in a more flexible way without going to a full visual programming language. For example, consider all the things that have a physical order and position in a file that is actually irrelevant. Why can't methods (for instance) get displayed as convenient, without worrying about where they are in the file? Limiting programs to fixed-width, single-font text also seems like a historical artifact. (The Xerox Alto allowed full WYSIWYG text for program code.) Having to draw diagrams with ASCII graphics just seems wrong. Admittedly, these examples are somewhat trivial, but my point is that program structure is probably a local maximum, not the best possible solution.
- mgummelt 7y ago
  > Why can't methods (for instance) get displayed as convenient, without worrying about where they are in the file? Get displayed as convenient what? > Limiting programs to fixed-width, single-font text also seems like a historical artifact. (The Xerox Alto allowed full WYSIWYG text for program code.) Having to draw diagrams with ASCII graphics just seems wrong. I can imagine benefits of better diagrams, for documentation, but how would more fonts in the code help?
  - Noumenon72 7y ago
    Comments could use native italics and links instead of HTML markup. More comment would fit on a line with a proportional font. Headings would be great for breaking up sections of code, and could be collapsible. Maybe linkable as well. Things that are arrays of arrays or tuples, like enums, could be tables. When you pass three functions to create an observer, there could be a table with columns "Parameter name, signature, implementation". Async code like observables and a thread's run method could be in a different font than code that executes right when you interpret it.
- jack_h 7y ago
  > It seems like there must be a way to treat text in a more flexible way without going to a full visual programming language. I've mentioned this a few times on past articles on HN, but I think the issue is we're marrying two different concepts with text based programming languages, namely presentation and representation. Consider - in an idealized context - HTML and CSS. One being the representation, the other being the presentation or more accurately how to present the representation. A user can change how the HTML is presented to them without changing the HTML directly. I think something like this would be useful for programming source code. So as a thought experiment for this in terms of programming, the source code would be some backing representation that you edit which can be rendered in any way a user desires, including back to regular text as we have now. Tooling (compilers/interpreters, version control, editors, etc) would deal primarily with the representation rather than the presentation. I think this would also solve a lot of problems that tooling ends up dealing with. Consider version control not having to deal with formatting changes because that's just an artifact of how a programmer chooses to render the representation rather than the representation itself. Autoformatters (gofmt, clang-format, etc) would be less about putting source code in a canonical format for the sake of tooling and more about rendering source code as text in a particular chosen way for individual programmers. Similar for tooling such as IDE code folding or refactoring, document generation, etc. This is actually something I have been pondering for a decade at this point. I have no idea how a system such as this could be implemented such that it is general and sound enough to be usable. But I still think it would be a neat way to program.
fatso784 7y ago
The issue is this dichotomy people keep fabricating between the “textual” and the “visual.” This makes it seem like one is “better” than the other for all time and we just need to get to the bottom of it. That distinction is made up and is itself part of the problem, and what we should be focusing attention to: https://dl.acm.org/doi/abs/10.1145/3313831.3376731 https://dl.acm.org/doi/abs/10.1145/3313831.3376731 A related point this work makes is this association people make between “visual” and “flow diagram” coding. Not all “visual” coding is flow-based.
jcelerier 7y ago
> Lack of evidence of their worth. There are not many Visual Languages that would be generally agreed are ‘‘successful,’’ and there is little in the way of formal experiments or informal experience that shows that Visual Languages are good. This is bullshit. A lot of musicians & visual artists are super successful with tools like Max/MSP, PureData, vvvv, or the ton of graph-based shader editors out there. Max/MSP is taught in a lot of place to music students, e.g. in conservatories. I promise that by the time you get them to understand for() and if() they've already made some music in Max, and the better ones have already started toying with sensors and controllers. Disclaimer: I worked 3.something years in a company where the owner routinely shipped software entirely built in Max. People without any programming abilities are able to use Blender's node editor to make super nice visuals : https://www.youtube.com/watch?v=JhLVzcCl1ug https://www.youtube.com/watch?v=JhLVzcCl1ug and at the other end of the spectrum, code that runs in airplanes is routinely designed using SCADE (https://www.ansys.com/products/embedded-software/ansys-scade-suite https://www.ansys.com/products/embedded-software/ansys-scade... - see e.g. these slides which talk about its use in the A380: http://www.artist-embedded.org/docs/Events/2008/Autrans/SLIDES/ArtistSummerSchool2008_Berry.pdf http://www.artist-embedded.org/docs/Events/2008/Autrans/SLID...) and other tools based in the synchronous reactive model.
- barbegal 7y ago
  I can write programs in LabVIEW in a few hours that would take me weeks in another language. Not all of that is due to the graphical nature, most probably due to the domain specificity and IDE integration. LabVIEW makes it super easy to create GUIs, visualise data and debug whilst running.
  - throwaway894345 7y ago
    I've used a bit of LabVIEW; my hunch is that LabVIEW's standard library has high level components, which it can provide because its niche is so narrow. You could make a similar framework or library in a general purpose language. The other thing LabVIEW provides is a comprehensive solution for project management--many general purpose programming languages give you half-assed package and dependency management (less than half-assed if you're dealing with C or C++) and some make you learn an entirely different programming or configuration language or just to list your dependencies or make your tests runnable. Those are the big rocks that make LabVIEW easier than most general purpose programming languages; however, if you're comparing LabVIEW to C, there are many other advantages, notably garbage collection and abstraction--things that are common in high level programming languages (anyone who says C is a HLL can see themselves out). The visual language is gravy compared to the standard library and the integrated programming environment.
tbabb 7y ago
> Difficulty with large programs or large data. [...] too little will fit on the screen Solved by the ability to pan and zoom. Node graph tools in professional visual effects environments can grow to >20,000 nodes in a document, especially given that nodes can be nested in groups or cross-included from other documents. > Need for automatic layout. [...] For example, generating an optimal layout of graphs and trees is NP-Complete [95]. Doesn't need to be absolutely optimal to make the problem go away; making something that helps the user enough that they don't have to think about it is completely tractable. Many professional node graph tools don't do any automatic layout at all and let the user handle it; which while less than perfect, doesn't stop the tools from being incredibly powerful. > Lack of formal specification. Currently, there is no formal way to describe a Visual Language. First, a formal specification is not necessary in order to build something useful. Second, if you really need a formal specification, then write/invent one. There is nothing about visual systems that makes them less possible to specify than any other software system. > Tremendous difficulty in building editors and environments. [...] These editors are hard to create [...] the language designer must create a system for display [...] which usually requires low-level graphics programming. So... do that stuff? What about that work is more difficult than the other parts of writing a compiler; or any other piece of complex software? Maybe this was more of a big deal in 1989, but basically all software today is based on a visual UI, so the complaint that the need for a UI makes visual languages intractable is... pretty ridiculous now. None of this would be harder than, say, writing a simple video game. > Lack of evidence of their worth. There are not many Visual Languages that would be generally agreed are "successful" "Nothing good exists yet, so nothing good is possible" doesn't hold water for me. > Metrics might include learning time, execution speed, retention, etc. From my experience in visual effects, non-coders can pick up a node graph tool very quickly and get very creative/inventive with nodes— but the moment they have to fall back to writing a script, they struggle or don't even try. It seems abundantly clear to me that node graphs are far easier to learn (but can be just as powerful) as writing with a machine grammar. I also did an image processing development project in a node graph tool that took about 4 days of experimentation to get to patentable IP. If I had to do the same project in Python or C++, it would have taken weeks or months, if I was even able to solve the problem at all without the fluidity and responsiveness of the node based editor. So even for "serious developers", a good node-based editor can multiply speed by a factor of 10. > Poor representations. Many visual representations are simply not very good. Can't argue there— but the solution is to make one that is good. :) > Lack of Portability of Programs. [...] Graphical languages require special software to view and edit ...Like basically all other file formats in the universe. :) It is not 1989 anymore, and it's time to fix this stuff.
hermitcrab 7y ago
Visual programming is great for some application areas. I recently wrote software that allows you to transform data step-by-step visually (https://www.easydatatransform.com https://www.easydatatransform.com). I think the visual, flow-based approach here is a real win, especially for non-programmers. However I wrote the software in a primarily text-based programming environment (C++ and QtCreator). I think it would have been a nightmare to do in a purely visual programming environment. Horses for courses.
- ken 7y ago
  "Flow-based" is only one kind of visual programming. UML alone has over a dozen different kinds of standardized diagrams. I can write a state machine with a big switch statement, for example, but I still find them easier to read when presented visually. Maybe we just haven't figured out good visualization types yet for some kinds of programs we write. Interestingly, the opposite is not true. I have yet to see a case where we had a half-decent visualization, and people still said "This would be easier to understand if it were 1000 lines of C++".
  - hermitcrab 7y ago
    I think a visual approach is great for representing the overall structure of a C++ program (e.g. inheritance hierarchy), but I much prefer code for the lower level logic. People have been using visual representations of programs for ~70 years. I'm not sure how much room there is for major improvements to visualization.
    - ken 7y ago
      Back in the 1980's, I heard virtually the same argument -- just replace "C++" and "visualization" with "assembly language" and "compilers". There turned out to be lots of room for improvements at the top! We could put almost all of the old housekeeping and busywork into our tools, and use a much simpler language for describing the unique parts of our program that we actually care about. I see no evidence that our current style of object-oriented programming is going to turn out to be the pinnacle of computing abstraction. The median language is (slowly but steadily) climbing the abstraction ladder, and each step brings it closer to an abstract visualization.
rezmason 7y ago
During times when the world isn't falling apart, I teach Scratch to kids on weekends: https://scratch.mit.edu/ https://scratch.mit.edu/ I don't pretend that Scratch's drag-and-drop block paradigm is a viable replacement for text-based programming, but I do think it demonstrates that there might be a text-like system we could create that would free us from keyboards, without reducing our source code to piles of spaghetti. Also, is declarative programming typically mappable to a visual paradigm without severe compromise? Think of all the applications that produce non-executable documents; don't they factor into the visual programming discussion?
- Noumenon72 7y ago
  Scratch is really fun and with a big budget I bet a lot of improvements could be made to it. GoAnywhere MFT is drag-and-drop and available for commercial use. I enjoyed it. My programs were pretty and powerful, if not well architected. Tasker programming is visual; it lets you program on a mobile phone with taps and not typing.
ken 7y ago
These issues are old. - Large programs / graphs? I can browse maps and photos of the entire planet from my web browser. ZUIs are well understood now. Also, computers got really big and fast. My graphics card has 8 GB (!) of RAM. Find a VPL that was designed for the era where graphics are cheap. - Automatic layout is hard? Yes, an optimal solution to graph layout is NP-complete, but so is register allocation, and my compiler still works (and that isn't even its bottleneck). There's plenty of cheap approximations that are 99% as good. - No formal spec? They sound desperate to come up with excuses why something can't work. How many (textual) programming languages had a formal spec before they got popular? I tried to look up when Ruby got a formal BNF grammar, for comparison, and it seems the answer in 2020 is still "nope, whatever parse.y does (13,000+ lines of C), that's Ruby". - Difficulty to create editors? This is a legitimate concern, as it's true "there are no general purpose Visual Language editors", but that's an economic issue, not a technical one. Plain text is one of the very few formats where we have general-purpose editors. In most areas of computing, profitability has taken over, and companies refuse to cooperate. There's no one word processing or spreadsheet or database format, either. - Lack of evidence? Again, they're holding VPLs to a higher standard. When a language like Ruby appeared, nobody asked for proof. Simply creating something that some programmers liked was sufficient. - Poor representation? That's an issue with all kinds of programs! If I had a nickel for every horribly written program I've had to deal with in my life... - Lack of portability? This is another symptom of the editor problem, above. Make a standard file format, and actually figure out how to get people to use it, and there's no problem. These issues are from an article written in the 1980's. The same day as its publication, Apple released the Macintosh IIci, with a 40 or 80 MB hard disk, 1 or 4 MB of RAM, and a 640x480 pixel display. (For all that computing power, the starting price was $6269 -- only slightly less than the $6615 starting price for a Honda Civic that year.) Clearly, visual programming was a non-starter in that day and age. So were 90% of the other things I do on my computer every day. That doesn't mean they're bad ideas. If you'd have asked me to write down a list of issues with C++ in 1989, I'd have a list that was similarly useless in 2020.
- sleepygardener 7y ago
  One more general-purpose thing I miss very much is filesystem. Really what is a good option for a usb disk filesystem which you would like to use on Mac, Win and Linux? Seems to me that without using Paragon/Tuxera/etc. you're screwed.
bori5 7y ago
Simulink is a very successful and powerful visual language for doing all sorts of wonderful things with. In the right hands of course.
dfilppi 7y ago
I visually program using vi and have for decades. Graphical programming is cumbersome for anything but trivial applications.
outlace 7y ago
Whatever happened to Luna Lang? https://luna-lang.org/ https://luna-lang.org/
- magicmouse 7y ago
  Luna project received an additional million in funding recently. still cranking away in Poland on that project. It is a very challenging project, they are trying to make a hybrid text and graphical form language, where you can switch between them. That's a very complicated thing to build. Wish them luck, they will need it.
anigbrowl 7y ago
This is a good and thoughtful essay, but in some respects reads like a rationalization of the author's preferences rather than an attempt to engage with the problems. He makes good points about the lack of formal specification, portability, and specialty software; this is indeed a problem, as all the best visual programming platforms are proprietary. On the other hand, there's no fundamental reason that visual editors shouldn't be able to load and save source code; it's just a different way of thinking about program structure and user interface. There's already a lot of tools out there that generate dependency graphs within or alongside an IDE; the main shortcoming i see is a lack of nesting. There are few instances in which i want to see the entire structure of a program all at once, any more than I want to read all the source code in a single giant text buffer One might equally ask if textual languages are so great, why are there so many of them? Couldn't everything be written in a nice capable high-level language like C? And of course it could, but lots of people don't like C's minimalism and lack of guardrails. It seems to me that most new general-purpose languages originate in a desire to have the computer do some of the tedious housekeeping (like casting variables) and frustration with awkward syntax. I'm old, so when I was learning to write code it was just flat text files and every build necessitated a round of swearing over forgotten semicolons, mismatched brackets, or incorrectly spelled variables. Syntax highlighting, code folding, autocompletion, and bracket matchings which do so much to make development in a modern IDE pleasant and productive were once viewed as undesirable crutches that would lead to poor practices, lack of portability, and inaccurate second-guessing of developers' intention by the IDE, subjecting hapless coders to the tyrannical will of the interface illuminati. I suspect that behind the aversion to visual programming lies a fear of standardization and automation that will result in widespread developer redundancy as it becomes easier for domain experts to build out their tools without code specialists, somewhat parallel to the development of electronics from hobby clubs where people helped each other with soldering irons, multimeters, and resistor identification charts to today's electronic factories populated only by pick-and-place machines assembling components too small for adult hands to manipulate at dizzying speeds. The more automation and and ML that finds its way into IDEs, the more standardized the underlying libraries are likely to become. To the extent that programmers are freed from syntax production and housekeeping duties and processes can more easily be read (from visual graphs/charts) with domain knowledge alone, the more likely it is that the most common functions become wholly standardized, in the same way that chips are. 555 timer ICs have been around since 1971 and the most popular derivatives are dual and quad versions. Nobody is writing new and improved 555 timers and there's no real demand for them.
paulhodge 7y ago
When a programmer says "I want visual programming" I think it's often one of those situations where 1) the asker has correctly identified a pain point, but 2) the thing they are asking for isn't necessarily the right way to do it. Instead of a box-and-lines diagram, I think the stuff that people really want is: - Structured editing. Valid change operations are obvious (maybe listed in a menu). No invalid states. - Cross-cutting connections are explicitly displayed (with lines or arrows or etc) instead of being implicit. - UI does a better job of highlighting just the relevant information and hiding everything else. - Overall more simplicity. As in, I think some people imagine that their thousand-lines-of-code project could be fully represented as a few boxes and lines with labels. There's some unrealistic expectations tied up in this area. But simplicity is still a good goal. Those are all good goals and there's different approaches to solve them. A lot of those asks can be solved by keeping the text document interface and adding enough cleverness on top of that (like EVE / LightTable). And there's other visual systems other than boxes-and-lines. People just need to think outside the box. (ha ha, sorry)
- ken 7y ago
  But why? In every case other than compilers, when we say we want these types of features, we never mean "let's stick with plain text, like the previous generation did, and not even consider a different type of solution".
- ilaksh 7y ago
  Visual programming doesn't just mean boxes and lines. It could mean anything other than plain text.
  - DonHopkins 7y ago
    That's right. It has to do with the dimensionality and topology of the language syntax, not just how it looks. C++ is not a visual programming language, no matter how much you colorize and pretty-print and fold it, because it's represented as a one-dimensional stream of characters, one after the other. Excel is a visual programming language (and one of the most widely used programming languages in the world of any kind, visual or not), because it's represented as a two-dimensional grid of numbers, expressions, strings, relative and absolute two-dimensional references, and it also (though no not necessarily for the definition of a VPL) supports all kinds of graphical attributes, colored text and background, fonts, text styles, formatting, boxes and lines, freezing and grouping and outlining rows and columns, embedded graphs and graphics, etc.
BigglesB 7y ago
I do a fair bit of UE4 coding in both Blueprint & C++. Each has benefits and drawbacks, many of which have been mentioned already. The biggest unmentioned one from a learning perspective though, is that with node-based editors, while you can still make logic errors, it’s difficult to actually make syntax errors. When you get used to it, you might not mind forgetting occasionally to match brackets, remember semicolons, use the right keywords in the right places etc, but as a beginner, all of these things can be quite frustrating and opaque, with often cryptic error messages. With visual node-graphs though, many of these things are just non-issues, which I think helps beginners gain confidence way faster without bouncing off and giving up in frustration. You can’t even connect pins of the wrong type most of the time, so you get some pretty well communicated type safety to boot. I’d actually love to see an evolution of a fairly advanced visual programming tool like Blueprint turn into a more general-use language, although preferably with files that are still stored as plain text (so at least play nicely with git etc) rather than the binary black boxes that are currently chewing up my LFS...
- boarnoah 7y ago
  I really don't get why they didn't want to serialize blueprints into XML or something, things have improved a bit now with the blueprint diffing feature in terms of merging, but having to put BPs into LFS is always going to be a pain point.
- joreg 7y ago
  This is exactly what we're building vvvv for, please have a look: http://visualprogramming.net/ http://visualprogramming.net/
Roboprog 7y ago
I would love to have a tool that would display code as some kind of annotated AST, rather than the layout imposed by most IDEs or tools such as gofmt. Put a bag over Java’s ugly face, that kind of thing. I don’t want to look at IntelliJ’s “staircase of doom” formatting, I want to see one parameter per line when there is more than one parameter. I don’t want to trace down data “COME FROM”s (local variables that are only used once and have to be tracked down from whence they came). Give me an alternate layout which directly tracks between the text and the diagram, similar to a structure layout, but showing details within functions/methods as a tree, as well. Show me clearly how values/expressions funnel or pipeline into each other, but filter out some of the noise. And auto-display fuglyCaps names as kebab-case, while we’re at it :-). Cuz, science.
- mjul 7y ago
  What you are describing is closely related to some of the ideas of Intentional Programming. The core idea is that the code is just a model (AST, graph...) and your editor projects it to whatever format or syntax you like. Development then is coding in these representations of the model or building new smart abstractions and model transformations (DSLs) for going from your screen representation to lower-level building blocks and eventually compiling to e.g. something that runs on the JVM. It quickly becomes just as complicated as it sounds. The best implementation in my experience is that of JetBrains’ free tool MPS (Meta Programming System). They start with the model (your AST) rather than the code. The learning curve is steep and it is difficult to adopt it in teams where people are more focused on shipping features than building exotic tooling to help them do it.
  - Roboprog 7y ago
    Thanks. That goes on my to-do list, then.
airstrike 7y ago
I just want a modern spreadsheet-like interface that isn't limited to the concept of sheets. A visually-represented functional language that lets a user with no programming knowledge run a pipeline of data transformation and ultimately output the data in a management/client-ready format. I want to build it myself, but I have neither the time nor the expertise.
- thanksforfish 7y ago
  What limitations are you looking to remove? 2d to multi-dimensional? Tables that float instead of living on sheets?
  - airstrike 7y ago
    EDIT: I think users can only really easily operate with 2D data, but to borrow someone else's acronym, a ZUI with multiple layers / levels of data could be more ergonomic than a vanilla GUI, so in a way I am indeed arguing for 3D. And yes to floating tables. Probably allowing users to also reference other sheets/tables semantically rather than just by row-column. Also creating explicitly defined (visual) pipelines of how the data is being transformed so users can audit and understand complex "spreadsheets". Auditing deeply linked calculations by following text formulas is too cumbersome for most users.
    - thewebcount 7y ago
      I've sat through a presentation done with a ZUI (Zoomable User Interface) and it was nauseating. I literally almost threw up due to the unexpected motion every few seconds.
      - airstrike 7y ago
        I believe you, but I think there's a difference between watching someone work and working – if you control the motions, they're not unexpected after all
- RedNifre 7y ago
crimsonalucard 7y ago
All programming is visual, think about it. Text is visual. I'm not just pointing out a semantic discrepancy here. Text is a specific method of encoding meaning and processes in a visual way. In short, textual programming is just a subset of visual. People think of the dichotomy as a duality of text vs. visual, it is not. Text just has features people tend to like: A Fixed line based structure. Isomorphism with the english vocabulary... Who says other visual programs can't have this either? The issue with a lot of these "visual programs" is that they also represent a subset of Visual Programming similar to how text is just a tiny subset of visual programming. What you are in fact seeing with most "visual programs" is mostly programs represented as graphs. The real dichotomy people are discussing is not text vs. visual but text vs. graph. Whenever I see something like labView or something labeled as a "visual program" it's always a program represented with a series of nodes and lines, aka a graph. Hopefully by illustrating the reality of what's going on, people can see beyond representing programs as just text or just graphs. There are other ways to represent modules and processes.
- DonHopkins 7y ago
  Well put! Text is graphics, but technology has historically limited how it looks and what you can do with it. The idea of text with control characters like "carriage return" and "newline", fixed width text, limited length lines, and text editors that wrap words when the cursor gets to the edge of the screen, and terminals with multiple lines that scroll, all incrementally evolved out of emulating punched card machines. And now we emulate VT100s emulating punched card machines. But it doesn't have to be that way. Here's a great piece of exploratory research that imagines how things could be different, by an amazing visionary visual thinker and artist, Scott Kim: "Viewpoint" is part of Scott Kim's PhD dissertation that he did at Stanford in 1988 (with Donald Knuth as principle advisor). It was programmed in Cedar, and ran on a color Dorado workstation at Xerox PARC. https://www.youtube.com/watch?v=9G0r7jL3xl8 https://www.youtube.com/watch?v=9G0r7jL3xl8 >Demo and explanation of Viewpoint, a computer system that imagines how computers might be different had they been designed by visual thinkers instead of mathematicians. Caution: this is basic research, not a proposal for a practical piece of software. Part of my PhD Dissertation at Stanford University in 1988. At 12:22, he starts with a blank screen, and performs a "visual boot", to use Viewpoint to build itself. http://www.scottkim.com/ http://www.scottkim.com/ https://en.wikipedia.org/wiki/Scott_Kim https://en.wikipedia.org/wiki/Scott_Kim https://visualraccoon.wordpress.com/2013/08/14/the-viewpoint-thesis/ https://visualraccoon.wordpress.com/2013/08/14/the-viewpoint... This is a visualraccoon Perspective on Scott Kim‘s very graphic and revolutionary exploration, Viewpoint: Toward a Computer for Visual Thinker. What would it be like to go back to visual first principles and take a fresh look at graphic user interfaces? The Viewpoint Thesis is that a small number of pixel manipulation primitives can be defined such that if they are bound to keyboard and mouse actions it is then possible to build a simple text-graphic editor by drawing it, and that that editor can be used to draw-build itself.* The Viewpoint Thesis & Editor is part of a larger project founded on the hypothesis that: “Only by treating the screen itself as a first class citizen will we be able to build computers that are truly for visual thinkers.” Scott, 1987. This project includes building visual programming languages for such thinkers. Viewpoint: Toward a Computer for Visual Thinkers http://www.scottkim.com.previewc40.carrierzone.com/viewpoint/viewpoint-dissertation.pdf http://www.scottkim.com.previewc40.carrierzone.com/viewpoint... http://www.scottkim.com.previewc40.carrierzone.com/viewpoint/index.html http://www.scottkim.com.previewc40.carrierzone.com/viewpoint... When I started my PhD project in 1981, the IBM PC was brand new, and it would be years before the Mac and Microsoft Windows would appear. Nonetheless I was familiar with graphical user interfaces because of my internship at nearby Xerox PARC, the visionary research center that spawned, among other things, the laser printer and the bitmapped display. I was well-acquainted with Alan Kay's vision of a dynabook (he envisioned the notebook computer in 1975), and understood the power of graphic user interfaces and graphic tools like paint programs to bring the power of computers to artists and visual thinkers. At the time I was part of the digital typography program within the Stanford computer science department, which built computer programs for digital typeface designers. I was also enthralled by the Visual Thinking course at Stanford, which taught engineers in the product design program how to think visually. It struck me as odd, and deeply wrong, that we were building tools for visual artists in a programming language that was utterly symbolic, and lacking in visual sophistication. I yearned for a programming language that had the same visual clarity that graphic user interfaces had. So I set about wondering what a visual programming language might look like. If computers had been invented by artists and visually oriented people, instead of by mathematicians and engineers, how might they write programs? It seemed to me an important question, but one that hardly bothered most computer scientists. I read about a few attempts to build visual programming languages, and decided there was something fundamental I needed to understand. My journey took me deep into the foundations of computer science, where I asked fundamental questions like “what is programming” and “what is a user interaction” — questions that often get passed over in computer science (any definition of “programming” that starts with “a sequence of symbols that…” is not deep enough to encompass visual programming languages). I never did build a visual programming language, but I did figure out a fundamental idea (things get interesting when the user and the computer share the same mental model of data), and built a rudimentary visual editor that demonstrated my ideas. I’m posting my dissertation and the accompanying video demo to re-open the conversation. What in my dissertation is interesting to you? What other work do you know of along these lines? Do you know of foundational research in interaction design and visual programming? And most importantly, what would be a good next step to push the work forward? (My research stalled for lack of a juicy application domain.) Please email me your thoughts. I'd love to hear them.
tartoran 7y ago
I think this is relevant to the discussion: Fructure: A Structured Editing Engine in Racket https://m.youtube.com/watch?v=CnbVCNIh1NA https://m.youtube.com/watch?v=CnbVCNIh1NA
indymike 7y ago
Every time I try a visual programming environment, I end up ditching it for code not because I can't to it in the visual language, but because it is very slow, tedious and boring.
wrnr 7y ago
Their might be an under-explored middle ground here, somewhere between representing a program as ASCII text and a full blown graph ZUI. Language is already a very efficient way to layout a graph structure, so lets keep that and borrow from maths the notation for summation, integration, ect, and from physics maybe Dirac's bracket notation for matrixes. I have thought of doing somethings like that in the browser with a WYSIWYG editor (document.designMode='on') and web components. The DOM is like the AST of the program and is fed into a runtime. Non-expression tags are basically just comments. Not for regular programming, but for a type of scientific notebook interface to do exploratory programming. Anyway one of this days.
- alricb 7y ago
  Maybe something like Fortress? [1] 1: https://web.archive.org/web/20060819201513/http://research.sun.com/projects/plrg/PLDITutorialSlides9Jun2006.pdf https://web.archive.org/web/20060819201513/http://research.s...
ibains 7y ago
Maybe the path is to merge visual and code programming. We're doing that for data engineering - you can draw in visual editor, and press a button and you have code. So as long as you constrain the domain, it seems doable. https://prophecy.io/ https://prophecy.io/, https://medium.com/prophecy-io/spark-deserves-a-better-ide-92d23175f3b4 https://medium.com/prophecy-io/spark-deserves-a-better-ide-9...
ilaksh 7y ago
https://en.m.wikipedia.org/wiki/Intentional_programming https://en.m.wikipedia.org/wiki/Intentional_programming
Too 7y ago
Here is a long list of reasons this is a bad idea: https://blueprintsfromhell.tumblr.com/ https://blueprintsfromhell.tumblr.com/ Having used my fair share of labview i know such examples are reality. Common mistake of the model is the sentiment that because it's visual everybody can do it! Biggest wtf i experienced was when system engineers wrote half completed "reference-implementations" in labiew, which coders then had to translate to Java for production, to sort out all the edge cases, because these visual tools (and the people that use them) are very bad at expressing edge cases. They might as well have given us graphs drawn on paper because the references were so far from reality you could never run them. Together with that, when the only tool you have is a hammer, everything looks like a nail. Take https://blueprintsfromhell.tumblr.com/image/162487395171 https://blueprintsfromhell.tumblr.com/image/162487395171 as an example, it's actually one of the simpler graphs from that page. But if you take a look at it you see it's just doing standard function calls in a sequence, like trunctate(), toText() setText(), what's then the point of making it visual?! For simple signal control loops with only mathematical operations the labview model works really well, but when you start doing things like subString, groupBy or indexOf and don't realize you should change domain, that's when you get the worst spaghetti. The benefit of a visual representation is that there are different ways to visualize things and you can choose which one fits the problem best. You have sequence, dependency, state, class, call-graphs, signals, lexical parsers, tables, charts, etc. Most visual environments only implement one of these and very soon you are using the wrong tool for the job.
- dkersten 7y ago
  And yet Blueprints enables a lot of non-programmers to write game logic. Actually, that’s why you have so many bad examples: they’re not programmers, they don’t necessarily know about software engineering principles like abstraction, or they don’t care because they consider it throwaway code. There are a LOT of similarly bad examples for textual languages (dailywtf anyone?), so its unfair to say that this is proof that visual bad textual good. I would very strongly argue that Media Molecules’ Dreams would not be as popular as it is if it used textual programming to implement game logic.
tomasGiden 7y ago
I’ve seen some projects succeed with (because of?) visual programming languages and others fail because of them. My general feeling is that a visual programming language excels as domain specific languages where the domain is quite visual. Ive seen two problems with visual programming languages which are quite severe in my opinion. 1. Merging sucks. Merging visuals are not straight forwards. I’ve seen companies do trunk based development before it is was hip just to get around it. 2. As with all high level languages, without an understanding about the overhead of different constructs, it is easy to create very inefficient solutions. On cloud systems it may be easy to throw more processing power at the problem. On embedded systems, less so.
wwarner 7y ago
I like the idea of visual programming for the same reason I like the idea of my compiler acting as a theorem prover. However, my sense is that both visual programming and formal systems run into the same problem, in that the space of possibilities, the space that needs to be expressed visually, explodes combinatorially, and so we find that the process of working in a visual environment is necessarily less expressive, always a bit behind what can be said by just writing it out in a program. That's why we do our best with compilers and type systems, but still have to write tests.
DonHopkins 7y ago
If you're going to criticize visual programming languages in general based on a 31-year-old paper written by Brad Myers in 1989, then you should at least look at some of the work he has done addressing those problems he set out in that paper, over the intervening three decades. http://www.cs.cmu.edu/~bam/ http://www.cs.cmu.edu/~bam/ >Brad A. Myers is a Professor in the Human-Computer Interaction Institute in the School of Computer Science at Carnegie Mellon University. He was chosen to receive the ACM SIGCHI Lifetime Achievement Award in Research in 2017, for outstanding fundamental and influential research contributions to the study of human-computer interaction. He is an IEEE Fellow, ACM Fellow, member of the CHI Academy, and winner of 15 Best Paper type awards and 5 Most Influential Paper Awards. He is the author or editor of over 500 publications, including the books "Creating User Interfaces by Demonstration" and "Languages for Developing User Interfaces," and he has been on the editorial board of six journals. He has been a consultant on user interface design and implementation to over 90 companies, and regularly teaches courses on user interface design and software. Myers received a PhD in computer science at the University of Toronto where he developed the Peridot user interface tool. He received the MS and BSc degrees from the Massachusetts Institute of Technology during which time he was a research intern at Xerox PARC. From 1980 until 1983, he worked at PERQ Systems Corporation. His research interests include user interfaces, programming environments, programming language design, end-user software engineering (EUSE), API usability, developer experience (DevX or DX), interaction techniques, programming by example, mobile computing, and visual programming. He belongs to ACM, SIGCHI, IEEE, and the IEEE Computer Society. For example, he wrote this one year later in 1990: Creating User Interfaces Using Programming by Example, Visual Programming, and Constraints. Brad A Myers. ACM Transactions on Programming Languages and Systems, Vol 12, No. 2, April 1990. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.460.2323&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.460... I really enjoyed this 1999 paper “A Taxonomy of Simulation Software: A work in progress” from Learning Technology Review by Kurt Schmucker at Apple. It covered many of my favorite visual programming languages. http://donhopkins.com/home/documents/taxonomy.pdf http://donhopkins.com/home/documents/taxonomy.pdf And it's also worth reading the more modern and comprehensive "Gadget Background Survey" that Chaim Gingold did at HARC, which includes Alan Kay's favorites, Rockey’s Boots and Robot Odyssey, and Chaim's amazing SimCity Reverse Diagrams and lots of great stuff I’d never seen before: http://chaim.io/download/Gingold%20(2017)%20Gadget%20(1)%20Survey.pdf http://chaim.io/download/Gingold%20(2017)%20Gadget%20(1)%20S... I've also been greatly inspired by the systems described in the classic books “Visual Programming” by Nan C Shu, and “Watch What I Do: Programming by Demonstration” edited by Alan Cypher. Visual Programming. Nan C Shu: https://archive.org/details/visualprogrammin00shu_2pf https://archive.org/details/visualprogrammin00shu_2pf Watch What I Do: Programming by Demonstration. Allen Cypher,. Daniel Conrad Halbert. https://archive.org/details/watchwhatido00alle https://archive.org/details/watchwhatido00alle
gampleman 7y ago
I think the lack of auto layout is actually a pretty significant part of the problem. It is an unsolved problem in data viz on how to layout complex graphs in a way that humans can parse. That also means that most humans are not very good at it. Personally my favourite VPL was Automator, which had the control flow primitives equivalent to the bash operators | and ;. This made layout and control flow trivial at the expense of expressiveness. However, as the goal of the thing was basically “terminal for the rest of us”, arguably those two operators are what most people end up using at the command line anyway. Of course Automator failed because of a number of other issues (lack of abstraction and bad package management I think are at the top of the list). But I think tools like that have a good amount of potential.
yummypaint 7y ago
Surprised to not see schematic design entry for FPGAs mentioned in the comments. It's still considered heresy by professionals, but i think there can be some benefits to doing the top level design schematically so you can see at a glance how the major components connect to each other and the external interfaces. Things do become quickly unmanageable if you try to define all the logic that way, but if you limit it to just wiring black boxes together i would argue it can improve readability.

136 points by mpweiher 7y ago

I think it would be great to do the reverse - take existing programs and map out the calls in a graph structure that can be visually displayed. Hardest part of looking at existing code base is to "mentally map" the call stacks and overtime, atleast for the way my brain operates, I tend to build a 2d map in my mind. It doesn't have concrete positions for blocks (functions), but there is a sense of structure in the brain that persists. I can come back to the code base and it is evoked automatically. We need visual tools for examining codebases just like we have tools for examining database schemas. Perhaps its not that simple though.

atupis 7y ago
It would be pretty cool when you zoom out first you see methods of how they are linked. Then classes and how they linked together.
TsiCClawOfLight 7y ago
Have you tried sourcetrail?
wvenable 7y ago
Having worked in actual visual programming environments I'd argue that even the most trivial applications are too complicated to mapped out visually. There just isn't enough visual resolution in a screen, or for that matter, in your eye to render it sufficiently. Zoom out and lose detail. Zoom in and use lose context. I've seen programs printed out into huge graphs covering entire boardroom tables and I can assure you nobody benefited from the display. Ironically the convoluted programs that people think this would be a solution for make for the most disgusting and hard to follow diagrams.
- som33 7y ago
  It's just a matter of a lot of research. I've been thinking for a long time that now that we have tonnes of memory and cpu power, its time to add hardware that keeps track of stuff and make things easier to understand. For instance when you look at hex in a file, you see a bunch of numbers, but it would be nice if there was some inference and cloud database of commonly used tropes/functions and could infer concepts, ideas of what the programmer was thinking. Currently with code, you get someones thoughts that are then translated into a language, then are then translated into machine code. It would be nice not to lose information regarding what does what in human readable language, I understand for performance reasons why programming languages as they currently are do what they do. But c and c++ were really designed when hardware and memory was expensive. So they are "too the metal" in terms of their model. I think reimplementing a basic machine, aka actual hardware design has to be done at the same time as a compiler is made, and work out the theory concepts behind it. The fragility of von neuman machines and their code has always bugged me, the machine can only blindly do what the hardware is designed to do, since hardware is all about speed, you want to minimize information, but for coding, it would make sense to have machines developed just for development and have "light code" for when you release it out into the wild. Either way I think there is plenty of innovation left, it's just a lot of research at the hardware and compiler level so that you have base foundations and abstractions to make real innovations in understanding how programs behave. One of the massive issues is not being able to understand cause and effect in programming which leads to bugs, because the brain can't make easy models of how code or algorithms will behave.
- vlovich123 7y ago
  I found the reverse. I had to do a tech debt cleanup of the code and found that doing an abstract diagram made it much easier to map out the interrelation between all the classes which was hard to keep in my head all at once. Also made it easier to communicate to others visually why it was spaghetti & easy to create a diagram about how I'd like to clean it up. I used PlantUML because what a lot of text -> visual things miss is that they don't provide the necessary knobs to tune the level of detail so doing the diagram by hand was a necessity. There may be multiple ways to look at something - sequence graphs, data flows - and additionally you don't want to capture all the details in the visualization, you want to be able to cut-off the diagrams at choke points where they're not helpful (e.g. I don't care about package X in the code) which is what all visualization packages try to do. This is directly analogous to geographic maps. Imagine if you viewed the map for a state/province whatever & had the local road network detailed. That map would be unusable & discarded quickly. Diagramming tools need to be able to take text & generate a visualization that you can zoom into/out of & lose/gain the appropriate level of detail necessary, be able to switch between different aspects (like you can with street view vs regular vs satellite, vs topography). I think the reason we don't have equivalent tools for programming is three-fold: 1. There's a lot of popular languages. Maintaining support for parsing & mapping that to the visualization abstraction is going to be a lot of work. Probably manageable though. 2. As engineers we tend to think in either solving the problem or not. No visualization is going to be good as a hand-drawn diagram in the moment nor is it probably good enough (at least the first version) as a vehicle for making the modification. 3. There's lot of type erasure/dynamism that's impossible to capture in code analysis (e.g. type-erased callbacks like `std::function`, `std::any`). That makes the visualization incorrect or cause it to fail prematurely.
fragmede 7y ago
There is an absolute dearth of tools, especially modern ones, in this space, but the tools to generate call graphs have existed for quite a while. Egypt will parse output from gcc and output a dot file for gnuplot/Graphviz. There may also be D3.js support in places. I agree we need visual tools, especially when trying to maintain a large codebase, but only the largest companies can sponsor development of an in-house tool for their codebase, and many of them won't. Maintaining code isn't sexy and doesn't make for new features to sell.
solidasparagus 7y ago
We look at compute graphs in deep learning, but it isn't very valuable because the graphs are so complicated. Among other issues, it's difficult to create a visualization of a compute graph that represents the time dimension well.

marktangotango 7y ago

This article provides a good overview of the problems and pitfalls of visual programming languages and environments and why their perceived benefits have not been realized up to now. I'd add one more. Cobol has gotten a lot of attention lately and although not "visual" in any sense whatsoever, had the goal of making programming accessible to managers and business users by adopting paragraphs and sentences to add structure to programs. Which is great and fine for trivial examples, but in practice the 25,000+ LOC real programs (and thousands of these in typical systems) were/are inscrutable to everyone but the specialist. The inherent problem is the lack of primitives to build abstractions to "move the langauge toward the problem domain". These primitives can be first class functions, objects, macros, etc [1]. Visual programming environments I've seen also fail to provide any such mechanisms as this article supports. [1] http://www.paulgraham.com/avg.html http://www.paulgraham.com/avg.html

kens 7y ago

The model of a program as a linear file of 80-column text is a historical accident based on IBM mainframes in the 1950s, and the decks of punch cards they read. It seems like there must be a way to treat text in a more flexible way without going to a full visual programming language. For example, consider all the things that have a physical order and position in a file that is actually irrelevant. Why can't methods (for instance) get displayed as convenient, without worrying about where they are in the file? Limiting programs to fixed-width, single-font text also seems like a historical artifact. (The Xerox Alto allowed full WYSIWYG text for program code.) Having to draw diagrams with ASCII graphics just seems wrong. Admittedly, these examples are somewhat trivial, but my point is that program structure is probably a local maximum, not the best possible solution.

mgummelt 7y ago
> Why can't methods (for instance) get displayed as convenient, without worrying about where they are in the file? Get displayed as convenient what? > Limiting programs to fixed-width, single-font text also seems like a historical artifact. (The Xerox Alto allowed full WYSIWYG text for program code.) Having to draw diagrams with ASCII graphics just seems wrong. I can imagine benefits of better diagrams, for documentation, but how would more fonts in the code help?
- Noumenon72 7y ago
  Comments could use native italics and links instead of HTML markup. More comment would fit on a line with a proportional font. Headings would be great for breaking up sections of code, and could be collapsible. Maybe linkable as well. Things that are arrays of arrays or tuples, like enums, could be tables. When you pass three functions to create an observer, there could be a table with columns "Parameter name, signature, implementation". Async code like observables and a thread's run method could be in a different font than code that executes right when you interpret it.
jack_h 7y ago
> It seems like there must be a way to treat text in a more flexible way without going to a full visual programming language. I've mentioned this a few times on past articles on HN, but I think the issue is we're marrying two different concepts with text based programming languages, namely presentation and representation. Consider - in an idealized context - HTML and CSS. One being the representation, the other being the presentation or more accurately how to present the representation. A user can change how the HTML is presented to them without changing the HTML directly. I think something like this would be useful for programming source code. So as a thought experiment for this in terms of programming, the source code would be some backing representation that you edit which can be rendered in any way a user desires, including back to regular text as we have now. Tooling (compilers/interpreters, version control, editors, etc) would deal primarily with the representation rather than the presentation. I think this would also solve a lot of problems that tooling ends up dealing with. Consider version control not having to deal with formatting changes because that's just an artifact of how a programmer chooses to render the representation rather than the representation itself. Autoformatters (gofmt, clang-format, etc) would be less about putting source code in a canonical format for the sake of tooling and more about rendering source code as text in a particular chosen way for individual programmers. Similar for tooling such as IDE code folding or refactoring, document generation, etc. This is actually something I have been pondering for a decade at this point. I have no idea how a system such as this could be implemented such that it is general and sound enough to be usable. But I still think it would be a neat way to program.

fatso784 7y ago

The issue is this dichotomy people keep fabricating between the “textual” and the “visual.” This makes it seem like one is “better” than the other for all time and we just need to get to the bottom of it. That distinction is made up and is itself part of the problem, and what we should be focusing attention to: https://dl.acm.org/doi/abs/10.1145/3313831.3376731 https://dl.acm.org/doi/abs/10.1145/3313831.3376731 A related point this work makes is this association people make between “visual” and “flow diagram” coding. Not all “visual” coding is flow-based.

jcelerier 7y ago

> Lack of evidence of their worth. There are not many Visual Languages that would be generally agreed are ‘‘successful,’’ and there is little in the way of formal experiments or informal experience that shows that Visual Languages are good. This is bullshit. A lot of musicians & visual artists are super successful with tools like Max/MSP, PureData, vvvv, or the ton of graph-based shader editors out there. Max/MSP is taught in a lot of place to music students, e.g. in conservatories. I promise that by the time you get them to understand for() and if() they've already made some music in Max, and the better ones have already started toying with sensors and controllers. Disclaimer: I worked 3.something years in a company where the owner routinely shipped software entirely built in Max. People without any programming abilities are able to use Blender's node editor to make super nice visuals : https://www.youtube.com/watch?v=JhLVzcCl1ug https://www.youtube.com/watch?v=JhLVzcCl1ug and at the other end of the spectrum, code that runs in airplanes is routinely designed using SCADE (https://www.ansys.com/products/embedded-software/ansys-scade-suite https://www.ansys.com/products/embedded-software/ansys-scade... - see e.g. these slides which talk about its use in the A380: http://www.artist-embedded.org/docs/Events/2008/Autrans/SLIDES/ArtistSummerSchool2008_Berry.pdf http://www.artist-embedded.org/docs/Events/2008/Autrans/SLID...) and other tools based in the synchronous reactive model.

barbegal 7y ago
I can write programs in LabVIEW in a few hours that would take me weeks in another language. Not all of that is due to the graphical nature, most probably due to the domain specificity and IDE integration. LabVIEW makes it super easy to create GUIs, visualise data and debug whilst running.
- throwaway894345 7y ago
  I've used a bit of LabVIEW; my hunch is that LabVIEW's standard library has high level components, which it can provide because its niche is so narrow. You could make a similar framework or library in a general purpose language. The other thing LabVIEW provides is a comprehensive solution for project management--many general purpose programming languages give you half-assed package and dependency management (less than half-assed if you're dealing with C or C++) and some make you learn an entirely different programming or configuration language or just to list your dependencies or make your tests runnable. Those are the big rocks that make LabVIEW easier than most general purpose programming languages; however, if you're comparing LabVIEW to C, there are many other advantages, notably garbage collection and abstraction--things that are common in high level programming languages (anyone who says C is a HLL can see themselves out). The visual language is gravy compared to the standard library and the integrated programming environment.

tbabb 7y ago

> Difficulty with large programs or large data. [...] too little will fit on the screen Solved by the ability to pan and zoom. Node graph tools in professional visual effects environments can grow to >20,000 nodes in a document, especially given that nodes can be nested in groups or cross-included from other documents. > Need for automatic layout. [...] For example, generating an optimal layout of graphs and trees is NP-Complete [95]. Doesn't need to be absolutely optimal to make the problem go away; making something that helps the user enough that they don't have to think about it is completely tractable. Many professional node graph tools don't do any automatic layout at all and let the user handle it; which while less than perfect, doesn't stop the tools from being incredibly powerful. > Lack of formal specification. Currently, there is no formal way to describe a Visual Language. First, a formal specification is not necessary in order to build something useful. Second, if you really need a formal specification, then write/invent one. There is nothing about visual systems that makes them less possible to specify than any other software system. > Tremendous difficulty in building editors and environments. [...] These editors are hard to create [...] the language designer must create a system for display [...] which usually requires low-level graphics programming. So... do that stuff? What about that work is more difficult than the other parts of writing a compiler; or any other piece of complex software? Maybe this was more of a big deal in 1989, but basically all software today is based on a visual UI, so the complaint that the need for a UI makes visual languages intractable is... pretty ridiculous now. None of this would be harder than, say, writing a simple video game. > Lack of evidence of their worth. There are not many Visual Languages that would be generally agreed are "successful" "Nothing good exists yet, so nothing good is possible" doesn't hold water for me. > Metrics might include learning time, execution speed, retention, etc. From my experience in visual effects, non-coders can pick up a node graph tool very quickly and get very creative/inventive with nodes— but the moment they have to fall back to writing a script, they struggle or don't even try. It seems abundantly clear to me that node graphs are far easier to learn (but can be just as powerful) as writing with a machine grammar. I also did an image processing development project in a node graph tool that took about 4 days of experimentation to get to patentable IP. If I had to do the same project in Python or C++, it would have taken weeks or months, if I was even able to solve the problem at all without the fluidity and responsiveness of the node based editor. So even for "serious developers", a good node-based editor can multiply speed by a factor of 10. > Poor representations. Many visual representations are simply not very good. Can't argue there— but the solution is to make one that is good. :) > Lack of Portability of Programs. [...] Graphical languages require special software to view and edit ...Like basically all other file formats in the universe. :) It is not 1989 anymore, and it's time to fix this stuff.

hermitcrab 7y ago

Visual programming is great for some application areas. I recently wrote software that allows you to transform data step-by-step visually (https://www.easydatatransform.com https://www.easydatatransform.com). I think the visual, flow-based approach here is a real win, especially for non-programmers. However I wrote the software in a primarily text-based programming environment (C++ and QtCreator). I think it would have been a nightmare to do in a purely visual programming environment. Horses for courses.

ken 7y ago
"Flow-based" is only one kind of visual programming. UML alone has over a dozen different kinds of standardized diagrams. I can write a state machine with a big switch statement, for example, but I still find them easier to read when presented visually. Maybe we just haven't figured out good visualization types yet for some kinds of programs we write. Interestingly, the opposite is not true. I have yet to see a case where we had a half-decent visualization, and people still said "This would be easier to understand if it were 1000 lines of C++".
- hermitcrab 7y ago
  I think a visual approach is great for representing the overall structure of a C++ program (e.g. inheritance hierarchy), but I much prefer code for the lower level logic. People have been using visual representations of programs for ~70 years. I'm not sure how much room there is for major improvements to visualization.
  - ken 7y ago
    Back in the 1980's, I heard virtually the same argument -- just replace "C++" and "visualization" with "assembly language" and "compilers". There turned out to be lots of room for improvements at the top! We could put almost all of the old housekeeping and busywork into our tools, and use a much simpler language for describing the unique parts of our program that we actually care about. I see no evidence that our current style of object-oriented programming is going to turn out to be the pinnacle of computing abstraction. The median language is (slowly but steadily) climbing the abstraction ladder, and each step brings it closer to an abstract visualization.

rezmason 7y ago

During times when the world isn't falling apart, I teach Scratch to kids on weekends: https://scratch.mit.edu/ https://scratch.mit.edu/ I don't pretend that Scratch's drag-and-drop block paradigm is a viable replacement for text-based programming, but I do think it demonstrates that there might be a text-like system we could create that would free us from keyboards, without reducing our source code to piles of spaghetti. Also, is declarative programming typically mappable to a visual paradigm without severe compromise? Think of all the applications that produce non-executable documents; don't they factor into the visual programming discussion?

Noumenon72 7y ago
Scratch is really fun and with a big budget I bet a lot of improvements could be made to it. GoAnywhere MFT is drag-and-drop and available for commercial use. I enjoyed it. My programs were pretty and powerful, if not well architected. Tasker programming is visual; it lets you program on a mobile phone with taps and not typing.

ken 7y ago

These issues are old. - Large programs / graphs? I can browse maps and photos of the entire planet from my web browser. ZUIs are well understood now. Also, computers got really big and fast. My graphics card has 8 GB (!) of RAM. Find a VPL that was designed for the era where graphics are cheap. - Automatic layout is hard? Yes, an optimal solution to graph layout is NP-complete, but so is register allocation, and my compiler still works (and that isn't even its bottleneck). There's plenty of cheap approximations that are 99% as good. - No formal spec? They sound desperate to come up with excuses why something can't work. How many (textual) programming languages had a formal spec before they got popular? I tried to look up when Ruby got a formal BNF grammar, for comparison, and it seems the answer in 2020 is still "nope, whatever parse.y does (13,000+ lines of C), that's Ruby". - Difficulty to create editors? This is a legitimate concern, as it's true "there are no general purpose Visual Language editors", but that's an economic issue, not a technical one. Plain text is one of the very few formats where we have general-purpose editors. In most areas of computing, profitability has taken over, and companies refuse to cooperate. There's no one word processing or spreadsheet or database format, either. - Lack of evidence? Again, they're holding VPLs to a higher standard. When a language like Ruby appeared, nobody asked for proof. Simply creating something that some programmers liked was sufficient. - Poor representation? That's an issue with all kinds of programs! If I had a nickel for every horribly written program I've had to deal with in my life... - Lack of portability? This is another symptom of the editor problem, above. Make a standard file format, and actually figure out how to get people to use it, and there's no problem. These issues are from an article written in the 1980's. The same day as its publication, Apple released the Macintosh IIci, with a 40 or 80 MB hard disk, 1 or 4 MB of RAM, and a 640x480 pixel display. (For all that computing power, the starting price was $6269 -- only slightly less than the $6615 starting price for a Honda Civic that year.) Clearly, visual programming was a non-starter in that day and age. So were 90% of the other things I do on my computer every day. That doesn't mean they're bad ideas. If you'd have asked me to write down a list of issues with C++ in 1989, I'd have a list that was similarly useless in 2020.

sleepygardener 7y ago
One more general-purpose thing I miss very much is filesystem. Really what is a good option for a usb disk filesystem which you would like to use on Mac, Win and Linux? Seems to me that without using Paragon/Tuxera/etc. you're screwed.

bori5 7y ago

Simulink is a very successful and powerful visual language for doing all sorts of wonderful things with. In the right hands of course.

dfilppi 7y ago

I visually program using vi and have for decades. Graphical programming is cumbersome for anything but trivial applications.

outlace 7y ago

Whatever happened to Luna Lang? https://luna-lang.org/ https://luna-lang.org/

magicmouse 7y ago
Luna project received an additional million in funding recently. still cranking away in Poland on that project. It is a very challenging project, they are trying to make a hybrid text and graphical form language, where you can switch between them. That's a very complicated thing to build. Wish them luck, they will need it.

anigbrowl 7y ago

This is a good and thoughtful essay, but in some respects reads like a rationalization of the author's preferences rather than an attempt to engage with the problems. He makes good points about the lack of formal specification, portability, and specialty software; this is indeed a problem, as all the best visual programming platforms are proprietary. On the other hand, there's no fundamental reason that visual editors shouldn't be able to load and save source code; it's just a different way of thinking about program structure and user interface. There's already a lot of tools out there that generate dependency graphs within or alongside an IDE; the main shortcoming i see is a lack of nesting. There are few instances in which i want to see the entire structure of a program all at once, any more than I want to read all the source code in a single giant text buffer One might equally ask if textual languages are so great, why are there so many of them? Couldn't everything be written in a nice capable high-level language like C? And of course it could, but lots of people don't like C's minimalism and lack of guardrails. It seems to me that most new general-purpose languages originate in a desire to have the computer do some of the tedious housekeeping (like casting variables) and frustration with awkward syntax. I'm old, so when I was learning to write code it was just flat text files and every build necessitated a round of swearing over forgotten semicolons, mismatched brackets, or incorrectly spelled variables. Syntax highlighting, code folding, autocompletion, and bracket matchings which do so much to make development in a modern IDE pleasant and productive were once viewed as undesirable crutches that would lead to poor practices, lack of portability, and inaccurate second-guessing of developers' intention by the IDE, subjecting hapless coders to the tyrannical will of the interface illuminati. I suspect that behind the aversion to visual programming lies a fear of standardization and automation that will result in widespread developer redundancy as it becomes easier for domain experts to build out their tools without code specialists, somewhat parallel to the development of electronics from hobby clubs where people helped each other with soldering irons, multimeters, and resistor identification charts to today's electronic factories populated only by pick-and-place machines assembling components too small for adult hands to manipulate at dizzying speeds. The more automation and and ML that finds its way into IDEs, the more standardized the underlying libraries are likely to become. To the extent that programmers are freed from syntax production and housekeeping duties and processes can more easily be read (from visual graphs/charts) with domain knowledge alone, the more likely it is that the most common functions become wholly standardized, in the same way that chips are. 555 timer ICs have been around since 1971 and the most popular derivatives are dual and quad versions. Nobody is writing new and improved 555 timers and there's no real demand for them.

paulhodge 7y ago

When a programmer says "I want visual programming" I think it's often one of those situations where 1) the asker has correctly identified a pain point, but 2) the thing they are asking for isn't necessarily the right way to do it. Instead of a box-and-lines diagram, I think the stuff that people really want is: - Structured editing. Valid change operations are obvious (maybe listed in a menu). No invalid states. - Cross-cutting connections are explicitly displayed (with lines or arrows or etc) instead of being implicit. - UI does a better job of highlighting just the relevant information and hiding everything else. - Overall more simplicity. As in, I think some people imagine that their thousand-lines-of-code project could be fully represented as a few boxes and lines with labels. There's some unrealistic expectations tied up in this area. But simplicity is still a good goal. Those are all good goals and there's different approaches to solve them. A lot of those asks can be solved by keeping the text document interface and adding enough cleverness on top of that (like EVE / LightTable). And there's other visual systems other than boxes-and-lines. People just need to think outside the box. (ha ha, sorry)

ken 7y ago
But why? In every case other than compilers, when we say we want these types of features, we never mean "let's stick with plain text, like the previous generation did, and not even consider a different type of solution".
ilaksh 7y ago
Visual programming doesn't just mean boxes and lines. It could mean anything other than plain text.
- DonHopkins 7y ago
  That's right. It has to do with the dimensionality and topology of the language syntax, not just how it looks. C++ is not a visual programming language, no matter how much you colorize and pretty-print and fold it, because it's represented as a one-dimensional stream of characters, one after the other. Excel is a visual programming language (and one of the most widely used programming languages in the world of any kind, visual or not), because it's represented as a two-dimensional grid of numbers, expressions, strings, relative and absolute two-dimensional references, and it also (though no not necessarily for the definition of a VPL) supports all kinds of graphical attributes, colored text and background, fonts, text styles, formatting, boxes and lines, freezing and grouping and outlining rows and columns, embedded graphs and graphics, etc.

BigglesB 7y ago

I do a fair bit of UE4 coding in both Blueprint & C++. Each has benefits and drawbacks, many of which have been mentioned already. The biggest unmentioned one from a learning perspective though, is that with node-based editors, while you can still make logic errors, it’s difficult to actually make syntax errors. When you get used to it, you might not mind forgetting occasionally to match brackets, remember semicolons, use the right keywords in the right places etc, but as a beginner, all of these things can be quite frustrating and opaque, with often cryptic error messages. With visual node-graphs though, many of these things are just non-issues, which I think helps beginners gain confidence way faster without bouncing off and giving up in frustration. You can’t even connect pins of the wrong type most of the time, so you get some pretty well communicated type safety to boot. I’d actually love to see an evolution of a fairly advanced visual programming tool like Blueprint turn into a more general-use language, although preferably with files that are still stored as plain text (so at least play nicely with git etc) rather than the binary black boxes that are currently chewing up my LFS...

boarnoah 7y ago
I really don't get why they didn't want to serialize blueprints into XML or something, things have improved a bit now with the blueprint diffing feature in terms of merging, but having to put BPs into LFS is always going to be a pain point.
joreg 7y ago
This is exactly what we're building vvvv for, please have a look: http://visualprogramming.net/ http://visualprogramming.net/

Roboprog 7y ago

I would love to have a tool that would display code as some kind of annotated AST, rather than the layout imposed by most IDEs or tools such as gofmt. Put a bag over Java’s ugly face, that kind of thing. I don’t want to look at IntelliJ’s “staircase of doom” formatting, I want to see one parameter per line when there is more than one parameter. I don’t want to trace down data “COME FROM”s (local variables that are only used once and have to be tracked down from whence they came). Give me an alternate layout which directly tracks between the text and the diagram, similar to a structure layout, but showing details within functions/methods as a tree, as well. Show me clearly how values/expressions funnel or pipeline into each other, but filter out some of the noise. And auto-display fuglyCaps names as kebab-case, while we’re at it :-). Cuz, science.

mjul 7y ago
What you are describing is closely related to some of the ideas of Intentional Programming. The core idea is that the code is just a model (AST, graph...) and your editor projects it to whatever format or syntax you like. Development then is coding in these representations of the model or building new smart abstractions and model transformations (DSLs) for going from your screen representation to lower-level building blocks and eventually compiling to e.g. something that runs on the JVM. It quickly becomes just as complicated as it sounds. The best implementation in my experience is that of JetBrains’ free tool MPS (Meta Programming System). They start with the model (your AST) rather than the code. The learning curve is steep and it is difficult to adopt it in teams where people are more focused on shipping features than building exotic tooling to help them do it.
- Roboprog 7y ago
  Thanks. That goes on my to-do list, then.

airstrike 7y ago

I just want a modern spreadsheet-like interface that isn't limited to the concept of sheets. A visually-represented functional language that lets a user with no programming knowledge run a pipeline of data transformation and ultimately output the data in a management/client-ready format. I want to build it myself, but I have neither the time nor the expertise.

thanksforfish 7y ago
What limitations are you looking to remove? 2d to multi-dimensional? Tables that float instead of living on sheets?
- airstrike 7y ago
  EDIT: I think users can only really easily operate with 2D data, but to borrow someone else's acronym, a ZUI with multiple layers / levels of data could be more ergonomic than a vanilla GUI, so in a way I am indeed arguing for 3D. And yes to floating tables. Probably allowing users to also reference other sheets/tables semantically rather than just by row-column. Also creating explicitly defined (visual) pipelines of how the data is being transformed so users can audit and understand complex "spreadsheets". Auditing deeply linked calculations by following text formulas is too cumbersome for most users.
  - thewebcount 7y ago
    I've sat through a presentation done with a ZUI (Zoomable User Interface) and it was nauseating. I literally almost threw up due to the unexpected motion every few seconds.
    - airstrike 7y ago
      I believe you, but I think there's a difference between watching someone work and working – if you control the motions, they're not unexpected after all
RedNifre 7y ago

crimsonalucard 7y ago

All programming is visual, think about it. Text is visual. I'm not just pointing out a semantic discrepancy here. Text is a specific method of encoding meaning and processes in a visual way. In short, textual programming is just a subset of visual. People think of the dichotomy as a duality of text vs. visual, it is not. Text just has features people tend to like: A Fixed line based structure. Isomorphism with the english vocabulary... Who says other visual programs can't have this either? The issue with a lot of these "visual programs" is that they also represent a subset of Visual Programming similar to how text is just a tiny subset of visual programming. What you are in fact seeing with most "visual programs" is mostly programs represented as graphs. The real dichotomy people are discussing is not text vs. visual but text vs. graph. Whenever I see something like labView or something labeled as a "visual program" it's always a program represented with a series of nodes and lines, aka a graph. Hopefully by illustrating the reality of what's going on, people can see beyond representing programs as just text or just graphs. There are other ways to represent modules and processes.

DonHopkins 7y ago
Well put! Text is graphics, but technology has historically limited how it looks and what you can do with it. The idea of text with control characters like "carriage return" and "newline", fixed width text, limited length lines, and text editors that wrap words when the cursor gets to the edge of the screen, and terminals with multiple lines that scroll, all incrementally evolved out of emulating punched card machines. And now we emulate VT100s emulating punched card machines. But it doesn't have to be that way. Here's a great piece of exploratory research that imagines how things could be different, by an amazing visionary visual thinker and artist, Scott Kim: "Viewpoint" is part of Scott Kim's PhD dissertation that he did at Stanford in 1988 (with Donald Knuth as principle advisor). It was programmed in Cedar, and ran on a color Dorado workstation at Xerox PARC. https://www.youtube.com/watch?v=9G0r7jL3xl8 https://www.youtube.com/watch?v=9G0r7jL3xl8 >Demo and explanation of Viewpoint, a computer system that imagines how computers might be different had they been designed by visual thinkers instead of mathematicians. Caution: this is basic research, not a proposal for a practical piece of software. Part of my PhD Dissertation at Stanford University in 1988. At 12:22, he starts with a blank screen, and performs a "visual boot", to use Viewpoint to build itself. http://www.scottkim.com/ http://www.scottkim.com/ https://en.wikipedia.org/wiki/Scott_Kim https://en.wikipedia.org/wiki/Scott_Kim https://visualraccoon.wordpress.com/2013/08/14/the-viewpoint-thesis/ https://visualraccoon.wordpress.com/2013/08/14/the-viewpoint... This is a visualraccoon Perspective on Scott Kim‘s very graphic and revolutionary exploration, Viewpoint: Toward a Computer for Visual Thinker. What would it be like to go back to visual first principles and take a fresh look at graphic user interfaces? The Viewpoint Thesis is that a small number of pixel manipulation primitives can be defined such that if they are bound to keyboard and mouse actions it is then possible to build a simple text-graphic editor by drawing it, and that that editor can be used to draw-build itself.* The Viewpoint Thesis & Editor is part of a larger project founded on the hypothesis that: “Only by treating the screen itself as a first class citizen will we be able to build computers that are truly for visual thinkers.” Scott, 1987. This project includes building visual programming languages for such thinkers. Viewpoint: Toward a Computer for Visual Thinkers http://www.scottkim.com.previewc40.carrierzone.com/viewpoint/viewpoint-dissertation.pdf http://www.scottkim.com.previewc40.carrierzone.com/viewpoint... http://www.scottkim.com.previewc40.carrierzone.com/viewpoint/index.html http://www.scottkim.com.previewc40.carrierzone.com/viewpoint... When I started my PhD project in 1981, the IBM PC was brand new, and it would be years before the Mac and Microsoft Windows would appear. Nonetheless I was familiar with graphical user interfaces because of my internship at nearby Xerox PARC, the visionary research center that spawned, among other things, the laser printer and the bitmapped display. I was well-acquainted with Alan Kay's vision of a dynabook (he envisioned the notebook computer in 1975), and understood the power of graphic user interfaces and graphic tools like paint programs to bring the power of computers to artists and visual thinkers. At the time I was part of the digital typography program within the Stanford computer science department, which built computer programs for digital typeface designers. I was also enthralled by the Visual Thinking course at Stanford, which taught engineers in the product design program how to think visually. It struck me as odd, and deeply wrong, that we were building tools for visual artists in a programming language that was utterly symbolic, and lacking in visual sophistication. I yearned for a programming language that had the same visual clarity that graphic user interfaces had. So I set about wondering what a visual programming language might look like. If computers had been invented by artists and visually oriented people, instead of by mathematicians and engineers, how might they write programs? It seemed to me an important question, but one that hardly bothered most computer scientists. I read about a few attempts to build visual programming languages, and decided there was something fundamental I needed to understand. My journey took me deep into the foundations of computer science, where I asked fundamental questions like “what is programming” and “what is a user interaction” — questions that often get passed over in computer science (any definition of “programming” that starts with “a sequence of symbols that…” is not deep enough to encompass visual programming languages). I never did build a visual programming language, but I did figure out a fundamental idea (things get interesting when the user and the computer share the same mental model of data), and built a rudimentary visual editor that demonstrated my ideas. I’m posting my dissertation and the accompanying video demo to re-open the conversation. What in my dissertation is interesting to you? What other work do you know of along these lines? Do you know of foundational research in interaction design and visual programming? And most importantly, what would be a good next step to push the work forward? (My research stalled for lack of a juicy application domain.) Please email me your thoughts. I'd love to hear them.

tartoran 7y ago

I think this is relevant to the discussion: Fructure: A Structured Editing Engine in Racket https://m.youtube.com/watch?v=CnbVCNIh1NA https://m.youtube.com/watch?v=CnbVCNIh1NA

indymike 7y ago

Every time I try a visual programming environment, I end up ditching it for code not because I can't to it in the visual language, but because it is very slow, tedious and boring.

wrnr 7y ago

Their might be an under-explored middle ground here, somewhere between representing a program as ASCII text and a full blown graph ZUI. Language is already a very efficient way to layout a graph structure, so lets keep that and borrow from maths the notation for summation, integration, ect, and from physics maybe Dirac's bracket notation for matrixes. I have thought of doing somethings like that in the browser with a WYSIWYG editor (document.designMode='on') and web components. The DOM is like the AST of the program and is fed into a runtime. Non-expression tags are basically just comments. Not for regular programming, but for a type of scientific notebook interface to do exploratory programming. Anyway one of this days.

alricb 7y ago
Maybe something like Fortress? [1] 1: https://web.archive.org/web/20060819201513/http://research.sun.com/projects/plrg/PLDITutorialSlides9Jun2006.pdf https://web.archive.org/web/20060819201513/http://research.s...

ibains 7y ago

Maybe the path is to merge visual and code programming. We're doing that for data engineering - you can draw in visual editor, and press a button and you have code. So as long as you constrain the domain, it seems doable. https://prophecy.io/ https://prophecy.io/, https://medium.com/prophecy-io/spark-deserves-a-better-ide-92d23175f3b4 https://medium.com/prophecy-io/spark-deserves-a-better-ide-9...

ilaksh 7y ago

https://en.m.wikipedia.org/wiki/Intentional_programming https://en.m.wikipedia.org/wiki/Intentional_programming

Too 7y ago

Here is a long list of reasons this is a bad idea: https://blueprintsfromhell.tumblr.com/ https://blueprintsfromhell.tumblr.com/ Having used my fair share of labview i know such examples are reality. Common mistake of the model is the sentiment that because it's visual everybody can do it! Biggest wtf i experienced was when system engineers wrote half completed "reference-implementations" in labiew, which coders then had to translate to Java for production, to sort out all the edge cases, because these visual tools (and the people that use them) are very bad at expressing edge cases. They might as well have given us graphs drawn on paper because the references were so far from reality you could never run them. Together with that, when the only tool you have is a hammer, everything looks like a nail. Take https://blueprintsfromhell.tumblr.com/image/162487395171 https://blueprintsfromhell.tumblr.com/image/162487395171 as an example, it's actually one of the simpler graphs from that page. But if you take a look at it you see it's just doing standard function calls in a sequence, like trunctate(), toText() setText(), what's then the point of making it visual?! For simple signal control loops with only mathematical operations the labview model works really well, but when you start doing things like subString, groupBy or indexOf and don't realize you should change domain, that's when you get the worst spaghetti. The benefit of a visual representation is that there are different ways to visualize things and you can choose which one fits the problem best. You have sequence, dependency, state, class, call-graphs, signals, lexical parsers, tables, charts, etc. Most visual environments only implement one of these and very soon you are using the wrong tool for the job.

dkersten 7y ago
And yet Blueprints enables a lot of non-programmers to write game logic. Actually, that’s why you have so many bad examples: they’re not programmers, they don’t necessarily know about software engineering principles like abstraction, or they don’t care because they consider it throwaway code. There are a LOT of similarly bad examples for textual languages (dailywtf anyone?), so its unfair to say that this is proof that visual bad textual good. I would very strongly argue that Media Molecules’ Dreams would not be as popular as it is if it used textual programming to implement game logic.

tomasGiden 7y ago

I’ve seen some projects succeed with (because of?) visual programming languages and others fail because of them. My general feeling is that a visual programming language excels as domain specific languages where the domain is quite visual. Ive seen two problems with visual programming languages which are quite severe in my opinion. 1. Merging sucks. Merging visuals are not straight forwards. I’ve seen companies do trunk based development before it is was hip just to get around it. 2. As with all high level languages, without an understanding about the overhead of different constructs, it is easy to create very inefficient solutions. On cloud systems it may be easy to throw more processing power at the problem. On embedded systems, less so.

wwarner 7y ago

I like the idea of visual programming for the same reason I like the idea of my compiler acting as a theorem prover. However, my sense is that both visual programming and formal systems run into the same problem, in that the space of possibilities, the space that needs to be expressed visually, explodes combinatorially, and so we find that the process of working in a visual environment is necessarily less expressive, always a bit behind what can be said by just writing it out in a program. That's why we do our best with compilers and type systems, but still have to write tests.

DonHopkins 7y ago

If you're going to criticize visual programming languages in general based on a 31-year-old paper written by Brad Myers in 1989, then you should at least look at some of the work he has done addressing those problems he set out in that paper, over the intervening three decades. http://www.cs.cmu.edu/~bam/ http://www.cs.cmu.edu/~bam/ >Brad A. Myers is a Professor in the Human-Computer Interaction Institute in the School of Computer Science at Carnegie Mellon University. He was chosen to receive the ACM SIGCHI Lifetime Achievement Award in Research in 2017, for outstanding fundamental and influential research contributions to the study of human-computer interaction. He is an IEEE Fellow, ACM Fellow, member of the CHI Academy, and winner of 15 Best Paper type awards and 5 Most Influential Paper Awards. He is the author or editor of over 500 publications, including the books "Creating User Interfaces by Demonstration" and "Languages for Developing User Interfaces," and he has been on the editorial board of six journals. He has been a consultant on user interface design and implementation to over 90 companies, and regularly teaches courses on user interface design and software. Myers received a PhD in computer science at the University of Toronto where he developed the Peridot user interface tool. He received the MS and BSc degrees from the Massachusetts Institute of Technology during which time he was a research intern at Xerox PARC. From 1980 until 1983, he worked at PERQ Systems Corporation. His research interests include user interfaces, programming environments, programming language design, end-user software engineering (EUSE), API usability, developer experience (DevX or DX), interaction techniques, programming by example, mobile computing, and visual programming. He belongs to ACM, SIGCHI, IEEE, and the IEEE Computer Society. For example, he wrote this one year later in 1990: Creating User Interfaces Using Programming by Example, Visual Programming, and Constraints. Brad A Myers. ACM Transactions on Programming Languages and Systems, Vol 12, No. 2, April 1990. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.460.2323&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.460... I really enjoyed this 1999 paper “A Taxonomy of Simulation Software: A work in progress” from Learning Technology Review by Kurt Schmucker at Apple. It covered many of my favorite visual programming languages. http://donhopkins.com/home/documents/taxonomy.pdf http://donhopkins.com/home/documents/taxonomy.pdf And it's also worth reading the more modern and comprehensive "Gadget Background Survey" that Chaim Gingold did at HARC, which includes Alan Kay's favorites, Rockey’s Boots and Robot Odyssey, and Chaim's amazing SimCity Reverse Diagrams and lots of great stuff I’d never seen before: http://chaim.io/download/Gingold%20(2017)%20Gadget%20(1)%20Survey.pdf http://chaim.io/download/Gingold%20(2017)%20Gadget%20(1)%20S... I've also been greatly inspired by the systems described in the classic books “Visual Programming” by Nan C Shu, and “Watch What I Do: Programming by Demonstration” edited by Alan Cypher. Visual Programming. Nan C Shu: https://archive.org/details/visualprogrammin00shu_2pf https://archive.org/details/visualprogrammin00shu_2pf Watch What I Do: Programming by Demonstration. Allen Cypher,. Daniel Conrad Halbert. https://archive.org/details/watchwhatido00alle https://archive.org/details/watchwhatido00alle

gampleman 7y ago

I think the lack of auto layout is actually a pretty significant part of the problem. It is an unsolved problem in data viz on how to layout complex graphs in a way that humans can parse. That also means that most humans are not very good at it. Personally my favourite VPL was Automator, which had the control flow primitives equivalent to the bash operators | and ;. This made layout and control flow trivial at the expense of expressiveness. However, as the goal of the thing was basically “terminal for the rest of us”, arguably those two operators are what most people end up using at the command line anyway. Of course Automator failed because of a number of other issues (lack of abstraction and bad package management I think are at the top of the list). But I think tools like that have a good amount of potential.

yummypaint 7y ago

Surprised to not see schematic design entry for FPGAs mentioned in the comments. It's still considered heresy by professionals, but i think there can be some benefits to doing the top level design schematically so you can see at a glance how the major components connect to each other and the external interfaces. Things do become quickly unmanageable if you try to define all the logic that way, but if you limit it to just wiring black boxes together i would argue it can improve readability.