13 ms·

Folk wisdom on visual programming

266 points by telotortium 5y ago

markus_zhang 5y ago
I guess this is turning into another "visual programming" post. On my side I'd like to ask your opinion about Blueprint. It really turned me off back in the day and I always wonder why they couldn't optimize UnrealScript further instead of getting something totally new. Yeah for sure eventually everyone get used to it and it becomes the new standard ("How can I NOT do Blueprint? It's so splendid" type of posts will show up) because UE is an extremely competetent engine. But why did we bring it forward in the first place? Is there enough reason to do so aside from that some teams get to have their own marks in the world and some managers can grab more power?
- dharmab 5y ago
  This presentation might enlighten why visual programming is in demand for gamedev. It's about a similar system developed in-house for The Division series: https://www.gdcvault.com/play/1023382/AI-Behavior-Editing-and-Debugging https://www.gdcvault.com/play/1023382/AI-Behavior-Editing-an... In short: There are many game designers and artists who prefer visual scripting tools over traditional code.
  - markus_zhang 5y ago
    But I also see some hate this style. This guy is an artist/programmer. https://forums.unrealengine.com/t/blueprints-are-a-toy-which-serious-unity-dev-would-use-ue4-without-c-f-or-unrealscript/71045/5 https://forums.unrealengine.com/t/blueprints-are-a-toy-which... BTW I do agree visual programming makes sense in certain area, but to use it for a general scripting language? Hell no...
    - jcelerier 5y ago
      The ratio of people I know in gamedev who love vs hate visual programming is on the order of 10:1
      - username90 5y ago
        I assume you work in unreal? Unity is more popular though and doesn't have visual scripting. Unreal Engine just having visual scripting and C++ with nothing in between is probably its biggest weakness, people prefer C# over those.
  - gridspy 5y ago
    If you cannot access the vault, here is a YT link to the same talk: https://www.youtube.com/watch?v=rYQQRIY_zcM https://www.youtube.com/watch?v=rYQQRIY_zcM
- ironmagma 5y ago
  It’s nice for asynchronous programming. Promises or callbacks take some getting used to, but a block diagram is as transparent of an abstraction for asynchronous flow as it gets.
- koolhaas 5y ago
  > But why did we bring it forward in the first place? Speculation: visual programming is a bit more beginner friendly, and more compatible with the brains of people designing stuff in UE. I wish blueprint was around when I was 13, messing around in UT’s version of UE and having no idea what unreal script could do. Visual language can break out all possible components into UI menus, just like everything else in UE. Visual oriented people really click with tools like visual programming. Personally I think it’s cynical / arbitrary to pin the decision to corporate power struggles. Sometimes people propose new things out of interest or as a side project and then it grows from there.
  - markus_zhang 5y ago
    Yeah let's ignore the cynical part. Whatever, it's already done and text scripting language is not coming back for UE.
    - spywaregorilla 5y ago
      Yes it is https://twitter.com/saji8k/status/1339709691564179464?s=20 https://twitter.com/saji8k/status/1339709691564179464?s=20
- spywaregorilla 5y ago
  I'm a programmer who does hobbyist gamedev and is a big fan of blueprints despite writing in text all day, for a few reasons. * A lot of game engine functionality is extremely helpful but not discoverable. Visual scripting languages make the full spectrum of the engine's API more easily discovered * Games are very stateful, and I find it is much easier to conceptualize this with blueprints than with code, which is better for functional stuff * Collapsing nodes in blueprints is much more powerful as a documentation tool than collapsable lines of code. When you collapse nodes, you can show the execution running through long text high level descriptors of what hidden blocks are doing, and without needing to create a function for that task specifically. e.g. code might do something like def update_target(): target = find_new_target() # set target to nearest enemy unit attack_pattern = choose_attack_pattern() # choose next attack pattern execute_attack() # perform attack montage and execute effects ____________________ whereas the bp would be Set next target to nearest enemy -> Choose next attack pattern -> Perform attack montage and execute effects It becomes pseudocode that is bright and on the forefront of your attention, vs. most code editors which will highlight code and lowlight comments.
ping_pong 5y ago
I worked on a visual programming tool from the late 90s to the early 2000s. The same problems apply then as they do now. There's an unwinnable war between keeping things simple and being complex-enough to do useful things. Visual stuff is great for simple things, but simple things aren't very useful. When you really need to do more complex things, you reach a limit very quickly. It becomes pretty unmaintainable very quickly and then people will "graduate" from it and go to something more convenient. My kid is learning Scratch right now, and it's taught him a lot of great stuff. But after about a year, he's ready to move onto Python. Scratch has taught him some very valuable concepts and he was able to jump into some Python concepts with ease (others still escape him). But you can only go so far with Scratch vs other programming languages. The same goes for other visual tools. In the end, people graduate from the visual style because it ironically becomes too complex because they want to keep things simple, and then you've lost a customer.
- jimsmart 5y ago
  > But you can only go so far with Scratch vs other programming languages. A friend has built some pretty amazing stuff in Scratch, including a BBC Micro emulator (runs 60+ games), a few arcade game ports, and lots more [1]. As a developer myself, I find it quite surprising how far one can go with Scratch. (FWIW I've never coded with it myself) [1] https://scratch.mit.edu/users/RokCoder/ https://scratch.mit.edu/users/RokCoder/
  - jjoonathan 5y ago
    Were there advantages? Or was it an exercise in determination?
    - jimsmart 5y ago
      He's been in the games industry for some years now, and as a side-project he teaches kids how to code — I'm pretty certain that he builds such crazy projects mostly as a fun hobby, perhaps also partly as a showcase for his teaching. IMO, for these kinds of projects, I suspect Scratch doesn't offer many advantages (if any at all). He likes a challenge :) — More details, including an interview, can be found here: https://www.coderkids.com/blog/who-is-rokcoder https://www.coderkids.com/blog/who-is-rokcoder He's pretty responsive to comments on his Scratch homepage — and can also be found under the same name on Twitter/FB/etc. He's always been happy to help others. Give him a shout if you have any specific questions. (FWIW: I worked with him — Cliff — for a short while, back in the 90s)
Vermeulen 5y ago
Our attempt at a Scratch-like language for our game creation app Modbox: https://docs.modboxgame.com/docs/mbscript https://docs.modboxgame.com/docs/mbscript It's definitely allowed non-programmers to make complex AI/game state for their creations, without having to deal with syntax errors or other issues. We choose Scratch/Blocky since this would be used for more managing states/logic, rather than flowing data (where node based makes more sense)
marcodiego 5y ago
Something I'd like: a modern maintained DRAKON version: https://en.wikipedia.org/wiki/DRAKON https://en.wikipedia.org/wiki/DRAKON . Tried it once, generates reasonably good C code. It would be great to have a modern FLOSS version.
- pritambaral 5y ago
  Not for C, but someone shares your motivation enough to build a version for JS: https://drakon.tech/ https://drakon.tech/ & https://github.com/stepan-mitkin/drakon.tech https://github.com/stepan-mitkin/drakon.tech
- deleted 5y ago
  [deleted]
divan 5y ago
Great job on summarizing and categorizing tons of opinions in one post! Shameless plug of my take on rethinking visual programming couple of years ago: https://divan.dev/posts/visual_programming_go/ https://divan.dev/posts/visual_programming_go/
- patterns 5y ago
  Oh, this is great. I especially like the fire torch analogy and the animation you came up with (https://divan.dev/images/torch.gif https://divan.dev/images/torch.gif). That's exactly how I feel when reading code. Recently, I explained the problem to a dear colleague when working on a code base (with about 600K lines of code). I compared the situation to working in a large office building with all lights turned off and you have to skim through thousands of documents with a flashlight. It's comforting to know that other people feel the same way.
deleted 5y ago
[deleted]
shadowgovt 5y ago
The biggest challenge with visual programming is you have to reinvent every text-based tool. And there are a lot of them. Diff, interactive debugging, keyboard-accelerated inputs and shortcuts, search, version control... The list goes on.
- koolhaas 5y ago
  Side note: interactive debugging is kinda inherent with the flow based programming paradigm. The program is continuously executing with every new action and immediately reflecting the current state. To program with it is to debug. If there’s an error with a component, it turns red and everything down stream breaks as well.
  - taeric 5y ago
    That falls on is face at scale. There is a reason not every bullet in a gun is a tracer bullet.
    - koolhaas 5y ago
      Flow paradigm doesn’t claim scale though. It’s about being a DSL for specific environments, often visually-oriented software like CAD or interactive art, which doesn’t have heavy 10K person team requirements.
      - taeric 5y ago
        Fair. And I think there is more here. If you are programming a singular thing, this probably has good legs. As soon as you are programming many things, though? This is why cad systems can be seen as visual programming of parts, but are never considered for this debate.
        koolhaas 5y ago
        I’d be curious to hear you elaborate on the singular/many things point. Trying to understand which deficiency you are pointing out.
spindle 5y ago
Wow! This is so awesome! Would you please write a page like this for every Hacker News post?
- physicsgraph 5y ago
  Probably worth triaging which topics are recurring to perform a meta-analysis. See my comment elsewhere in this thread.
crispyambulance 5y ago
Yeah, this is going to be yet another "visual programming" topic. I sort of disagree with the intro to LabVIEW: > There are a large number of visual programming tools that are roughly in the paradigm of ‘boxes with some arrows between them’, like the LabVIEW example above. I think the technical term for these is ‘node-based’, so that’s what I’ll call them. There are NOT a "large number" of visual programming tools that are roughly in the paradigm [...] like LabVIEW. I am sure there are some but, really, how many active ones? Also, "node-based" is not the main concept. The most essential concept of LabVIEW is known as "data-flow" (https://labviewwiki.org/wiki/Data_flow https://labviewwiki.org/wiki/Data_flow). Data generated by a node "flows" to the other nodes it is connected to and when any node has data on all it's inputs, it executes and produces data which flows to whatever nodes it's connected to. This happens asynchronously. You get concurrency "out of the box" with LabVIEW. That's extremely powerful and much easier to reason about, IMHO, than the concurrency constructs in virtually any mainstream computer language. I think there's plenty of room for all kinds of languages (both text and visual) as DSL's. If they're well-designed and intuitive, they're like a hot-knife through butter for a domain expert. The nice thing about a really good DSL is that you use it more like a library, whereas with a general purpose language you would be reaching for a framework. It gives you a lot of creative latitude and it feels right. The problems start when people start using them as golden hammers, or use them for things which they're not good at.
- jayd16 5y ago
  >There are NOT a "large number" of visual programming tools that are roughly in the paradigm [...] like LabVIEW. They list half a dozen or so. Most of the node based systems are also flow oriented, aka nodes are functional and stateless which allows them to be mapped across any amount of input. It's a neat pattern but it's really not as uncommon as you think. The mainstream shader languages work this way, for example.
- koolhaas 5y ago
  > Data generated by a node "flows" to the other nodes it is connected to and when any node has data on all it's inputs, it executes and produces data which flows to whatever nodes it's connected to. I think you just described flow programming. There are at least 10+ visual languages / environments that work like this.
kingcobraninja 5y ago
I've been using LabVIEW for 15 years, first in an academic and scientific setting, then for machine vision in surveillance, then as the HMI for industrial robots. It is, for all intents and purposes, a general purpose programming language, and I think it would be more popular if it wasn't so stupid expensive. I was never interested in or good at coding, but LabVIEW made sense to me. Now I'm using programming principles I learned in LabVIEW and re-implementing them in C++ and C#. It's been a journey.
Jarwain 5y ago
I think what the author did here hits hard on why hacker news is such a great resource. To quote: > Most fields have a problem with ‘ghost knowledge’, hard-won practical understanding that is mostly passed on verbally between practitioners and not written down anywhere public. At least in programming some chunk of it makes it into forum posts. I'd be incredibly interested in seeing a series of posts like this, where authors mine hacker news and other forums for insights that they consolidate (then repost here). What other topics do y'all think HN provides unique insight on?
- physicsgraph 5y ago
  I agree that this type of compendium/synthesis is useful. I was amused to open this thread and see comments focused on visual programming rather than the meta-aspect. As for what HN could be mined for, a big topic is software languages. There are ways to subdivide that: by language, by optimization aspect (e.g., comparative performance). Geographic topics like cost of living and quality of life. Remote work pros and cons. Another way of identifying topics worth mining would be identify posts where @dang provides a link to previous posts and those previous posts have a sufficient number of comments.
- Zababa 5y ago
  > What other topics do y'all think HN provides unique insight on? Career managment I think (not sure if it's the good term for it). I see lots of people in their 40's or more that quit some FANG job for another job that earns less but is more aligned with how they want to live their life. I see lots of people advocating to quit jobs as soon as you see red flags. I think both of these are good wisdom. On the other hand, a point that's still a mystery to me is "programming language choice". I've learned useful heuristics like you can do web backend in pretty much anything, if you have no libraries for something that needs them like an AWS SDK you're going to suffer, what helps in the long run slows you down at first, but that's about it. I don't know if it's because I'm too young in this field, because I don't understand/missed something, or because there is no big truth to find.
  - gridspy 5y ago
analog31 5y ago
The author alludes to problems related to complexity, modularity, and formatting, of visual code. I think a hard problem for visual languages is the sheer physical labor of creating code using drawing tools. I used LabVIEW extensively for a few months, and ended up with crippling wrist fatigue and eyestrain headaches from all of the fine mouse work and menu selections needed to write even small programs. This is an ergonomic problem with all graphical software, but is particularly acute with programming because of the amount of code that gets written. Another problem is innovation. You can quickly spin up an experimental language and try it out because the infrastructure -- editor and command shell -- already exist. This is one reason why we enjoy such a proliferation of new language ideas. New ideas in graphical software require intensive effort to try out. On the other hand, there does seem to be something about data flow programming (LabVIEW, Excel) that is attractive to beginners, that can't be overlooked. Excel is probably the number-one programming tool in use right now.
- bmitc 5y ago
  > I used LabVIEW extensively for a few months I’m not sure that a few months is enough to understand the true pros and cons of a language. I’m using Elixir in my day job now and have used F# for side projects, and it’s not clear to me any one is faster to program in, especially for GUIs. LabVIEW actually shines, contrary to popular but inexperienced opinion, for large projects. I have used LabVIEW rather extensively, and it seems any project would have seen massive delays if I used something else, which is why I didn’t. But I can indeed see an argument for strain when using the mouse so much. I have experienced that before and improved my ergonomic setup. As an aside, it is curious to me that many programmers are heavy PC gamers as the one true way to game and don’t complain about the mouse use there but will throw out all sorts of complaints when using it to program or use tools. > New ideas in graphical software require software intensive effort to try out. This “problem” isn’t inherent to graphical programming though. It’s because legions of programmers are convinced that text=programming and have spent decades writing text-only tools. You see this friction when using LabVIEW with GitHub, whose tooling is incompatible with the diff and merge tools that come with LabVIEW, which makes GitHub look outdated and not LabVIEW. I’m actually working on ideas to build a toolkit for graphical programming languages. There are certainly challenges though, such as the desktop GUI ecosystem being a mess and reliance upon graph data structures and algorithms (like automatic layout). > On the other hand, there does seem to be something about data flow programming (LabVIEW, Excel) that is attractive to beginners Dataflow is attractive to advanced users as well, which is part of the draw to languages like F#, Elixir, Racket, Clojure, etc. Although, the dataflow in LabVIEW is actually more advanced, which is basically a multithreaded and branched version of the single threaded linear pipelines found in the languages I mentioned. Working in F# and Elixir with pipes can be pretty irritating sometimes given my knowledge of LabVIEW.
forinti 5y ago
Visual Programming is great for selling: look how simple it is to do this simple task!
- taeric 5y ago
  The cynical side of me thinks that is the basic pitch for every framework and methodology.
  - jpindar 5y ago
    Look at the kinds of examples people are using to demo Copilot.
erichocean 5y ago
Post was useful, more of this kind of thing please!
jcims 5y ago
There are certainly many many instances where visual programming doesn't do the job. One instance where I found it incredibly useful, though, was with Node-RED. I used it to build a hobby-grade process automation solution, including a number of software PID control loops, that ran current limiters, valves and pumps, several load cells and half a dozen temperature sensors. All of this was orchestrated from Node-RED running on a Raspberry Pi communicating with the Tasmota esp8266 sensor/controller nodes over MQTT and I built it from scratch, learning along the way, over the course of 2-3 days. Something about the visual pipelines and processing nodes really helped with handling concurrency, priority and maintaining tidy boundaries between systems. It was the first time I really had fun programming something in quite some time.
vvanders 5y ago
One thing missing here and in most of the visual programming discussions since it sits in a somewhat adjacent SCADA industry is Ladder Logic[1]. It was pretty illuminating how many parallels ladder has with the visual and scripting solutions I'd worked with in GameDev. Hot-reload under running processes, visual debugging very reminiscent of node based visual editors. I ended up automating our greenhouse with a P100[2] from Automation Direct and was pretty impressed with how straightforward it was to use. Structured text is making in-roads in some places from what I can tell but ladder still seems to dominate because it's easily understood and maps well to a "relay based" mental model. It seems to be very sticky, at least more than most visual scripting languages I'm familiar with. [1] https://en.wikipedia.org/wiki/Ladder_logic https://en.wikipedia.org/wiki/Ladder_logic [2] https://www.automationdirect.com/adc/shopping/catalog/programmable_controllers/productivity_series_controllers/productivity1000_(stackable_micro_plc)/cpus/p1-540 https://www.automationdirect.com/adc/shopping/catalog/progra...
- mikewarot 5y ago
  Ladder logic is powerful stuff. I learned it at Westinghouse in the 1980s. It's a non-Von Neuman based programming methodology well suited to real time environments. Every ladder gets executed once per cycle, and there are usually a few hundred cycles per second. Special functions are implemented in boxes, like delays, counters, etc. My task back then was to feed the entire I/O state into a ring buffer, and when a "crash" of the steel mill happened, a host PC could then download the entire buffer, giving access to information about what happened in the seconds BEFORE the crash. It would be fairly easy to port to the Arduino, if someone hasn't done it already. Yep - it's been done https://cq.cx/ladder.pl https://cq.cx/ladder.pl Because Ladders are text based, you could use Teco/EDT/Vi/Emacs/WordStar macros on them.
  - vvanders 5y ago
    Yeah, it's a pretty neat ecosystem. PLCs are kinda like the Legos of automation, incredibly easy to mix and match with a fair number of (insecure) standard protocols. I briefly toyed with the idea of building out some open source hardware/software but I didn't really seem like there was enough of a userbase to make it viable. Anyone using it professionally wants support and 10+ years of parts, most maker/hobby projects want direct access to code. Still some pretty neat concepts for putting together highly customized processes with extensible and interchangeable hardware.
w_t_payne 5y ago
I believe that diagrams are at their most useful when they are deployed as a communications tool. When discussing our software, we commonly draw a diagram to help illustrate how functionality is divided up among components, and to show which components are impacted by particular functional chains. They are a powerful tool for communicating and coordinating the work of multiple people or multiple teams. If our software does not provide us with diagrams itself, then we will commonly create them ourselves in our technical documentation or on whiteboards as an aide to collaboration. However, as we descend in our system towards finer and more detailed levels of granularity, two things happen. First of all, fewer people are impacted by very ‘local’ design decisions, so the importance of communication diminishes. Secondly, the frequency of change increases, so the importance of mutability and ease-of-editing increases. The requirements that we place on our engineering tools depend largely on what we are trying to do with them. Communicating and coordinating work across a large engineering organization places fundamentally different requirements on our engineering tools than does, say, working alone to solve a challenging problem that requires the creation of new concepts and abstractions. There are many different ways to work with software, and although each share much of the same flavor (dealing with informational complexity), the nature of these challenges is often distinct enough that the different requirements may lead to vastly different solutions. For my part, I have a lot of faith in the general utility of tools which allow for the easy generation of diagrammatic representations of high-level structure, but which naturally decay to a more conventional textual representation at finer levels of granularity.
- slx26 5y ago
  Yeah, to make sense of complex systems they need to be well structured and modularized, and visuals have a lot to contribute here. If you can't fit your structure comfortably in a diagram, your structure is too complex for most people to manage. Further down the line you stop needing to know where you are (global view, understanding), and you only need to deal with implementation: actually doing what you need to do. Implementation can be often very complex on its own too, that's why algorithms are so scary at the beginning... and text is a better tool here. There's not much more structure to distill, and some parts are tricky, and you have edge cases and you often need to get your hands dirty. You need to keep all this complexity in mind at the same time anyway, and after you are done you can treat the implementation as a black box. So text is superior here. That's how we manage complexity. Now we only need tools that can represent it better. But maybe it's not even visual programming languages, but rather visual system "structurers"? There are other parts of programming that could use better visual cues, and visual editors for visual stuff (colors, shapes, borders, etc.) will always be relevant, but those are all very different concerns.
Timothycquinn 5y ago
I wrote a spreadsheet based rules engine at Honeywell in around 2003 which enabled the business to maintain the business rules for all sales contracts that ran through the automation and control systems division. This gave them the ability to define forms, subsets of forms (legal or otherwise), notifications and approvals. I had its own natural language based expression language which the super users and business analysts could easily understand. The spreadsheets, were change controlled by the business an imported into the system when they were approved. I considered this design a form of a visual language and the real beauty was that it did not require any re-coding of business rules, which did change quite frequently. The developers did not have to be re-writing any business logic as the rules changed. The back end was Java and TCL based built on a graph database. Not super fast but super easy to understand and train up new developers. It was quite successful and ran all sales contracts for for several years with billions of dollars of contracts passing through per year. I lost contact with the teams so I'm not sure how long it survived but I know that SAP charged millions of dollars trying to build a solution in their systems but could not get one working at a reasonable price.
- cruise9 5y ago
  I'd love to hear how you integrated a Java and TCL backend with spreadsheets. I'm assuming this isn't something you can do with excel? (Logistics at the small business I work at, is run on a mess of excel spreadsheets with complicated formulas and occasionally VBA for some email functionality. I have a feeling we're doing things horribly wrong, but I don't know of any better way to do things with our zero IT budget.)
  - Timothycquinn 5y ago
    I did not leverage any VBA or VB. I learned to code in VBA and written way too much code in that space and was well aware of the limitations. I'm pretty sure the way it went was, on check-in of a new version, I had a Java program that read the spreadsheet validated that all the right sheets were there and that the columns on each sheet conformed with the spec and also validated rows of data. The sheets were translated on each front end node for easy access. Regarding the expression engine, the the expressions themselves were then translated into TCL boolean expressions. The attributes for the related objects where already loaded into memory into dictionaries and then the expressions were just evaluated to get a boolean result.
dannywarner 5y ago
I like the line "There’s a comforting familiarity in reading the same internet argument over and over again." The problem I think with visual programming tools is people get stuck. I watched my dad's career die stuck in Visual Basic. It's because people get stuck in thinking the way those tools force you to think. Code in text has a magnificent universality.
- mikewarot 5y ago
  I'm curious what you mean by how Visual Basic makes you think.
manbart 5y ago
I have to write scripts for Cisco UCCX (call center software) at work sometimes. Never heard of it before this job. I was horrified that uses a visual ‘drag-n-drop’ IDE to create the call handling scripts, no other supported method. It’s the only ‘visual’ coding I’ve ever used. And in the 2 years I’ve used it, I actually don’t hate it. As other posters have mentioned, a visual approach can be decent DSL. Btw, you can run still run arbitrary Java (1.7) in UCCX scripts too, which is kinda cool. Used that to make email alerts in case of certain situations in the call queue
epage 5y ago
My feedback, based on my both long but limited LabVIEW experience > IDEs for text-based languages normally have features like code folding and call hierarchies for moving between levels, but these conventions are less developed in node-based tools. This may be just because these tools are more niche and have had less development time, or it may genuinely be a more difficult problem for a 2D layout — I don’t know enough about the details to tell. NI demoed a system to function zoom in/out feature at one point but never fully developed it. LabVIEW does have a call hierarchy. Like with Doxygen, my problem with these is they show everything rather than finding ways to highlight important relationships (e.g. you don't want to show every array append). > Input I suffered from RSI when using LabVIEW. Maybe I hadn't mastered enough but there is a lot more than just dropping that needs to be improved, but the wiring. > Formatting I think LabVIEW suffers from giving people so much pixel-perfect control, it encourages people to waste time on it because its "so close". At one point, they demoed dynamic formatting (reflows the diagram as you add/move nodes without any manual management). This really needs to be made the default. > Version control and code review LabVIEW NXG has a text-based GVI format. It is not easily reviewable, mergeable, etc because it has all of the graphical minutia in it. My proposal would be - Remove block diagram formatting (see above) - Separate top-level hand-designed UI from VI auto-placed front panel - Reduce reliance on icons - Encourage naming wires At that point, a text-based VI format would be simple enough to be manageable within existing tools.
- ziml77 5y ago
  I had to suffer through a semester of LabVIEW. It seems like it might be okay if you have high enough level blocks of functionality available that you're never hooking many together, but for what I was trying to do it was a massive pain (doable, but tedious and hard to organize and follow)
- bmitc 5y ago
  > At one point, they demoed dynamic formatting (reflows the diagram as you add/move nodes without any manual management). This really needs to be made the default. Do you by any chance have a link to a video or article where this is shown or discussed? I’m a heavy user of LabVIEW, and I am also convinced in my own explorations of creating visual languages that dynamic automatic layout is the way to go, but it is a very hard problem.
tehsauce 5y ago
really cool survey of the topic, nice work!
mikewarot 5y ago
For completeness, I feel compelled to mention GNU Radio. It is a node based visual programming tool. It has its quirks, but you can do fairly complex digital signal processing with it, and it will do I/O from your sound card, as well as RF stuff.
- jpindar 5y ago
  And its one of the more common ways to make those little Software Defined Radio boards do something.
MetaWhirledPeas 5y ago
Years ago I bounced around from one visual tool to another, never mastering any of them but getting deep enough to see the flaws. Hypercard, Authorware, Director, Flash. They were all very approachable, but they were all incredibly flawed in the end. You'd spend most of your time jumping from one limitation to the next, trying to hack your way out of the spaghetti jungle. I think the better approach is to use conventional languages with UI tools layered on top that at worst generate readable code. (Like I assume is often done in game development.) Source control and diffing are indispensable. The lure of these tools is that they are approachable, which sometimes is exactly what you need to get started. It's just very rarely what you need to finish. So let these tools be used to get people interested in programming, or to give people an idea of how to accomplish a project. But more importantly let's make sure to continue taking steps to demystify coding. Not because we want everyone to be programmers, but because when people do choose to program we want them to be able to collaborate with the rest of the world, and when they leave we don't want to inherit some visual pasta dish.
- musiccog 5y ago
  I loved MacroMind (later Macromedia) (later Adobe) Director. Delivered dozens of educational CD's, corporate CD's and many of my own 'experiments'. One of those was a map for EverQuest to help find dead bodies (it was a big thing at the time), and help swimmers who fell off a ship to get back to land alive. Another was a project to scan every user-authored book in Ultima Online (using basic OCR) Director was extensible in that you could not only import 3rd party libraries, but write your own modules in Pascal. Much more powerful and extensible than Flash IMHO. Flash won that battle after the buy-out, and Adobe extinguished Director. Still miss it to create proof-of-concept, simple or amazing cross-platform apps.
Sophistifunk 5y ago
We need both. And more. Visual projection and editing is valuable when looking at a zoomed-out "components and wires" sort of architectural view. It's also good for showing high-level overviews of a process, like various UML diagrams should have been. It can sometimes be used well for the nitty-gritty implementation details of a single unit, although this seems to be where a lot of it breaks down. But it's terrible for doing all of these things at once. And also, if you can control your level of detail/zoom well enough to do it right, you'd get similar benefits projecting to simpler text formats as well. What's most important is that all this stuff needs to be declarative. Not in the sense that you hide the order of operations within a block of code, in a way that final performance characteristics can become a surprise, but that the more complex structures of your project that organises the blocks into modules, classes, functions, records, etc need to be declarative, rather than an imperative set of instructions that build a structure at run-time (usually difficult or impossible to explore and instrument) which is later executed to achieve your actual goals.
raphlinus 5y ago
> Is ‘folk wisdom from internet forums’ worth exploring as a genre of blog post? Yes. I found this very well done and useful. One other (not entirely unrelated) potential area that immediately comes to mind for me is the architecture of UI toolkits. This is an area where there is very little serious (ie academic) writing, and even less good writing. The vast majority of what's set to paper is marketing material for a particular toolkit. There are some potentially interesting blog posts, but these tend to be highly opinionated. Adding to this, while most of the discussion today is around shiny new stuff like SwiftUI and React, there's actually a pretty significant "literature" of older systems. It's hard to know what's worth digging into. And, as with visual programming, there's not much in the way of synthesized understanding about the best way to do things. Game programmers like imgui, but it hasn't caught on outside that domain. Why? React is massively popular, but there is criticism of its performance (and of web-based technology stacks). There are other debates about the value of being "native," which perhaps used to be a clear distinction but is fuzzier today when you look at it closely. So I think something along these lines for that topic could be pretty valuable.
galaxyLogic 5y ago
What I find interesting is that the human society and economy at large is able to function without there being any genius programmer who designed it all. If you look at roads, there is a road everywhere and still people don't get lost. But in a big chunk of source-code I do feel I get lost. I believe that may be because there are too few constraints when programming textually. Whereas when designing roads and bridges we have the constraints of physical space. It is possible to draw a 2-dimensional map of all the roads and thus easily understand it all. I think visual programming was inspired by this notion, how physical and geometrical constraints make things simpler because we know those constraints intuitively, which thus helps us reason about the structure of the whole thing. It can not violate physics, nor geometry. Textual programs can, do anything their syntax allows. That makes their full structure hard to understand
- hermitcrab 5y ago
  Habe you seen an aerial view of Birmingham's "spaghetti junction"? ;0)
- username90 5y ago
  A city with streets is just a graph with 1-4 pointers per node. It is super simple in code, just a few lines, not sure how you could get lost in that. If you make code anything more complex than that then it isn't really equivalent to navigating the streets on a map. > I believe that may be because there are too few constraints when programming textually No that is not the issue, you can write extremely complex and useful programs in just 100 lines of code and you can translate those to visual programming easily with no improvement in comprehension. Code just takes long to read since it is so powerful.
  - galaxyLogic 5y ago
    > If you make code anything more complex than that then it isn't really equivalent to navigating the streets on a map. My point is, you should strive to make coding not more complex than that, if possible. And if you have constraints such as must represent the program as a 2-D dataflow structure then it must stay simple as that. Now is there a need to make more complicated programs? Probably yes. But human society is still able to function with the simple dataflows that it has, which makes it understandable and thus very resilient.
arduinomancer 5y ago
Really high quality post, would love to see more like this
ffhhj 5y ago
HN is a great resource to gather different points of views from experts. Here is some folk wisdom I compiled on load and performance testing for a presentation: Selenium/Web Driver handles the UI tests. Postman and a handful of others can perform the API tests. JMeter is the de facto for load testing at every place I've been to. And you can write any of these into your CI. There are three separate reasons to do load testing: 1) Performance testing. Confirm the system does not degrade under a specified load and find out what performance can be expected under these circumstances. This is basically ensuring your system can handle X amount of traffic without issues and knowing your baseline performance. You should get the same kind of response times as you are getting from live server telemetry. 2) Stress testing. finding out what happens when the system is stressed beyond its specs. How does it degrade & where. 3) Reliability testing. Find out how your system breaks and when. The goal here is to try to break the system and test things like failover and making sure you don't lose or corrupt data. Better to die gracefully then abruptly. I've used Jmeter quite a bit over the last year, and what seems to be the largest issue is that it doesn't "break" apps because it works at the protocol level and isn't a full "browser". As you increase load and response times start to increase, since it runs sequentially through the test plan the time between the requests also increases. But, when actual humans with browsers use apps there are loads of AJAX requests being fired which don't necessarily go in order or wait for others to complete first. Most managers only want to see how traffic will perform in what-if scenarios (can we handle Black Friday, what will happen if our traffic goes up 10x during a special event, etc). For these, JMeter and whatever service you're using for analytics/metrics work perfectly fine (New Relic, is the current favorite). Then we can compare average latency, Error Rates, etc. with what we get during JMeter. The issue is that most managers or other PHB want more than just a one off for load testing. And this article greatly covers that. A lot of time people want a Swiss Army Knife of tools for "automation", where load testing falls under that category for them. This magical tool should be able to: * Test your API calls from a functional, integration, and a performance (not load testing; just making sure we're under specific latency) * Test your Web-sites from the same testing perspectives, as well as being modular (ie: Selenium's POM) * Integrate with CI so specific tests, test suites/flows can be tested for every new build, and we can run specific workflows by clicking a button * Be used during load testing so we can measure latency and run tests while recreating customer experience from the Web side of things. Last time I had to do load testing in a professional context, we required 10+ million long-lived TCP and websocket connections transacting multiple times a second. There weren't any off-the-shelf solutions at that time that could come within an order of magnitude of that at a reasonable cost - the most viable solution we tested required a thousand EC2 instances to sustain that traffic. Do red line testing. Gradually increase load on your service with concurrent requests until it is saturated. Measure how the latency and errors progress with more concurrent requests and understand at what point and how your service starts to break down under heavy load. Based on this you may have to do many things – 1. Can you optimize your service or your service's downstream dependencies or the application calling your service. 2. Can you build in graceful degradation into your service – functionality reduction to get more useful throughput out of your service – with same resources. 3. Build circuit breakers and throttlers before your downstream dependencies so that you don't overload them and cause them to fail or you don't fail totally when they do indeed fail. 4. If you do get overloaded for some reason (say your server pool suddenly became half its size), are you able to recover quickly. 5. Do you have monitoring and alerts for these load scenarios? I tend to avoid having to do load testing as it sucks up time without telling me much of interest. I instead opt for having decent telemetry on the live system. It will tell me how it performs and where the bottlenecks are. I can set alerts and take action when things degrade (e.g. because of a bad change). Besides, there is no substitute for having real users doing real things with real data. And in any case, having telemetry is crucial to do any meaningful stress or reliability testing. Otherwise you just know it degrades without understanding why.
bmitc 5y ago
Funnily enough, I’ve been quoted in this. ;) To expand on modularity, it is indeed curious to me why many will often throw it out that visual programming somehow doesn’t scale for complex and large programs and fails at modularity. Let’s look at LabVIEW as compared to F# and Elixir, two text-based languages that are wonderful to work in and generally well loved, where LabVIEW is well loathed. In LabVIEW, you have projects to organize code, libraries to serve as modules or namespaces, the ability to create functions (including polymorphic and malleable VIs, which are a way to auto propagate types to make functions auto polymorphic), clusters (which are basically structs or records), immutable data, OOP with interfaces (value-based and not reference-based, which is a major plus and not found in many languages), concurrency features, and actors. F# has projects, namespaces, modules, immutable data, records, discriminated union, OOP (reference-based), pattern matching, concurrency features, and some actor frameworks. Elixir is similar with all its features with some small differences. Now, these are all essentially the same fundamental tools available to create programs and to modularize them and scale them. So where’s the rub? How is it that you can somehow create modular and large complex programs with Elixir and F# but not LabVIEW? Where’s the actual discrepancy? Throw in other languages to this discussion and you get hoards of inheritance hierarchies, memory access problems, and mutability, all of which are well known to cause issues with scale and complexity. So I’ve never understood the argument. (I actually do have arguments against LabVIEW for large complex problems but it’s not the arguments usually made and more relate to the IDE than the language). Anecdotally, I once interviewed with a place that was vehemently giving up on LabVIEW. They gave no real reason other than they hated it and said it didn’t work for large programs. So I asked to see one of their Python programs, which was their new language of choice. I was shown a single file that was something like 10,000-15,000 lines long with functions often having signatures taking 10-20 lines, meaning the functions had something like 10-30 function arguments. So at that point I knew that these people didn’t actually know how to properly architect software and that LabVIEW wasn’t actually their problem. They were their problem. As mentioned below my part that was quoted, it can indeed be argued that LabVIEW’s visual nature is a feature when it comes to showcasing poorly organized code. It makes it bare, right there in front of you and in more dimensions than what you get in text. You don’t necessarily get that with text, so people who write poorly organized code in any language but then use LabVIEW are greeted with a visual experience they’re unhappy with. They, incorrectly, see the problem as lying with the tool and not themselves and their programs. All that being said, there are absolutely problems with LabVIEW and other visual languages, just like there are with text-based languages. I’d just like to see more objective discussions of these things because I believe a text-only view holds things back. I’d love to see programming computers evolve beyond the languages, editors, and IDEs we have now.
jjcc 5y ago
off topic but just triggered by the article. I strongly suggest somebody write a bot on HN and maybe leverage GPT-3 to harvest collective wisdom on many topics. It's a golden mine.
henrikeh 5y ago
There is a type of visual programming which is applied to virtually everything around you of some mechanical complexity: interactive geometric constraint solvers. In CAD tools like SolveSpace, SolidWorks, Inventor and several others, the dimensions of objects are not described directly, but instead through a set of constraints such as “these are orthogonal”, “these have this distance”, “this is a tangent” etc. If you have never used it, try SolveSpace, which is free. Albeit highly declarative, it is absolutely a programming language for a specific domain. It doesn’t try to be good at describing imperative programs or flows; instead it presents a paradigm which is very different, but fits perfectly in the problem domain of mechanical designs.
imvetri 5y ago
One more into that list. Flow based with input as impulses.
knolan 5y ago
I really like the node based workflow used by Blender for creating materials, compositing and now for generating geometry. This is where I think visual programming works best, realtime interactive tweaking of something actually visual. It's also generally kept simple and is powerful enough to get the job done. I've always hated LabView however, it grows too unwieldy quickly and the limited nature of its visual display doesn't help. I remember once going to a NI training session around 2008 that was actually a sales session and their guy tried to convince us that LabView was easily parallelisable by drawing two empty while loops and showing them maxing out both CPUs on his laptop. The mind boggles.
- auxym 5y ago
  Arguably he's not wrong? I don't love LabVIEW but I make it work because their hardware is not bad compared with alternatives. Dragging two loops is pretty easy compared to figuring out threads and thread-safe communications in C++.
  - knolan 5y ago
    The point is that two empty loops should not be let run the CPU to 100%. But yes, the concept is nice at least. I agree on NÍ hardware, it’s very solid. I’ve had good success using session based control with Matlab and also with Python.
simonh 5y ago
"A picture paints a thousand words, sure, but here's a thousand word essay. Try painting me a picture of it." I don't remember where I hard that, or close to it, but there's a lot of truth to it. There do seem to be some very useful visual logic and work flow tools, I hope that continues, but there's something magical about the ability of language to explain, describe and precisely define things. Maybe eventually we'll arrive at a state where really good programming environments incorporate the advantages of both textual and visual tools in a powerful and dynamic way. I love films and picture books, I also love novels, but even the best novels are enhanced by really good illustrations and sometimes even maps. Some authors find visual tools like relationship maps and even geographic maps help them in the actual authoring process too.
- hutzlibu 5y ago
  "Maybe eventually we'll arrive at a state where really good programming environments incorporate the advantages of both textual and visual tools in a powerful and dynamic way. " Yes, I agree. But not surprisingly, this is not easy to solve. You need a visual language as a consistent concept and then you need a actual visual framework, rendering it all, and enabling user input. You need a compiler compiling forth and back, between text and visu. And then it needs to scale. And be performant enough to actually work with it and not just have a visual demo. Sounds hard? Trust me, it is.
  - simonh 5y ago
    I listened to a podcast a while ago where someone was speculating why programs processing data can't just be connected together like devices in a water system. Pumps, sinks, boilers, showers, etc. Or like electrical devices, just channel the data between them. It occurred to me that data isn't like water, or electricity. Those are generic resources that can largely be easily standardised and treated consistently by any device. Data isn't like that. Data formats and encoding vary hugely and can have crazy different properties. It's more like piping chemicals between devices in a chemical factory. You can't just take the output from a device producing acid and plug the pipe into the input of a device that's designed to process water, or ammonia, or petrol. They're all 'just' fluids and you can pipe them about, but their properties vary wildly. You can only feed one into a device as input if that device is specifically designed to process that specific material. Yes it is possible to define standardised data formats, up to a point. XML, JSON, CSV, etc but even then you can't just feed arbitrary JSON into every program designed to ingest JSON and expect it to work, just because the data it expects is JSON formatted. Yes Unix has pipes, but each tool in the pipe chain has to be told exactly how to process the specific input it gets from the previous tool. You can't just look in history for two arbitrary examples of using pipes, and cut and paste the first half of one pipe chain, and paste the second half of another arbitrary pipe chain on the end, and expect to get something useful out of the combination. Maybe you'll get lucky, but usually you'll get garbage.
k__ 5y ago
It think, VP shines in specialized fields and not general programming. Visual domains are an obvious example. If your domain is constrained by something that maps good to spatial dimensions than you're good to go. A few extra dimensions can also put into styling. But after that it becomes too restricting. That's why interface builders work fabulous until you have to implement actual business logic and modularize your application. This always leads to adding things that aren't directly represented in the current visual model and get lost, because of indirections.
akavel 5y ago
Mandatory fanboi mention, and IMO a gross omission of the article: https://enso.org/ https://enso.org/ (née Luna) - an open-source language & IDE which aims to insta-kill all the "vs. textual programming" arguments by keeping a strict 1:1 visual<->textual representation translation as a core feature of the language. A 2.0 alpha was recently released, go check it out!
- auggierose 5y ago
  Very nice. I am also currently building a new environment for programming (I want my 5 year old niece to get a head start on modern creativity :-)). I think the main idea is simple: logic + visualisation + data. Make something that takes all of these three things seriously. Programming languages usually care about the logic part only (and even that not very well).
RobertKerans 5y ago
That was a good summary, was useful and really enjoyed reading it. The key takeaway for me, which really reified ideas I have about UI design tools, was this: > At the opposite end of the spectrum is, say, an oil painting, which is also a visual medium but much more of an unconstrained, freeform one, where brushstrokes can swirl in any arbitrary pattern. This freedom is useful in artistic fields, where rich ambiguous associative meaning is the whole point, but becomes a nuisance in technical contexts...[elided]...Drag-n-drop editors arguably lose a lot of the features of ‘true’ languages by giving up structure, and more programmatic elements are likely to still use a constrained set of primitives. That's the key problem afaics, and why UI design tools + codegen never works well outside of highly constrained situations. There are two conflicting practices: programming and visual design. And they overlap, obviously, but they don't properly mesh. The tool either goes in the programmatic direction, which makes codegen much easier but drastically curtails the ability to design properly. Or it goes in the design direction and makes codegen either very difficult or forces generation of absolute garbage code (for worst examples, see tools that generate enormous amounts of absolutely positioned HTML elements with inline styles on everything). And there's no real middle ground (not that this stops endless attempts to square the circle).
snidane 5y ago
The benefits of visual programming are usually that it captures the domain model better than using general programming languages without any guidance. Lot of the value is in well maintained integrations to other systems. Often, if the components of the visual language were released as a library or a framework, it would provide more benefit than having to drag and drop visual boxes and arrows which don't ever make it to version control or in some very clunky way. Releasing the composable parts as text based library will not happen, because of economics. There is little money in creating libraries for the creators. All generated value would be captured by infrastructure, cloud platform providers. With visual languages, the value can be captured by selling a no-code idea to non-programming folks and especially to enterprise with deep pockets wishing to get rid of expensive engineers. Visual programming languages are an economical problem, not a technical one.
ianhorn 5y ago
What a fantastic post. > Is ‘folk wisdom from internet forums’ worth exploring as a genre of blog post? I’d add another yes here, if they’re all as thorough as this post. I’ll add something that I didn’t see directly addressed in here: input methods strongly affect the medium. Last year I struggled intensely with RSI. I got to the point where I couldn’t type for months. I tried some voice coding tools (caster and talon). Some aspects of this were actually better than keyboard coding. Unfortunately speech recognition is still at a point where it drove me insane with inaccuracies. It also made me realize that with a keyboard, text code is incredibly natural. With keyboard+mouse, some new modes open up. With voice alone, I wanted to code a very different way. I won’t go into detail, because this is already a long comment, but I believe that when we get speech recognition as reliable as typing, we’ll see an explosion of new programming paradigms.
Lichtso 5y ago
Overall a really good essay. I would have split the "Types of visual programming" in two main categories: Visual and graphical. E.g. scratch is only visual because it is an over-the-top syntax highlighting but still has an AST like textual programming (they are based on hierarchies and sequences). Spreadsheets are also sequential but in multiple dimensions. Dataflow- / node- based systems on the other hand are actually graphical (graph-based) and can not be mapped to hierarchies in text directly.
grayrest 5y ago
> So presumably we could learn to distinguish closely packed shapes if we were familiar enough with the conventions. The data visualization people have already demonstrated this with sparklines and small multiples. There are fewer preattentive attributes than most people would assume so I expect increasingly dense visual representations to approach text (e.g. logograms). I'll also point out that data viz people have also come up with reasonable solutions to problems mentioned like wire bundling. I haven't seen an auto-layout algorithm I like but I've found a reasonable guess followed by manual manipulation with automatic wire routing to be okay for visualizations and the circuit drawing and VLSI I did 20 years ago in college.
getflourish 5y ago
The strength of visual programming is not in visualising code, but rather, visualising programs. At a small scale, textual code can be very concise, expressive, compact, efficient and fast to write. But at a large scale, writing thousands of lines of code in text files in folders makes it very difficult to understand how the pieces of a program connect. To overcome this, we rely on file naming conventions, frameworks or writing documentation to understand how the various parts of text relate to each other. Programming with nodes and wires is a way of visualising those relationships by creating a map of the program. Similar to maps of the real world, those maps may be complex and not easy to understand. But those maps show that there is complexity. what we can learn from maps of the real world? Well: zoom level matters. When we want to understand the world, we look at countries. When we want to get to the next city, we look at the road level. In my opinion, visual programming environments are transparent and honest about the underlying complexity of a program. Contrast this with countless folders full of files filled with thousands of lines of code. Such repositories of code can be very difficult to navigate and get around. Yes, frameworks and conventions can help to understand already learned organisational systems (example from the real world: “I learned to navigate American cities designed on a grid of streets, but I get lost in Europe“). If we compare textual coding to visual programming, we should not only focus on the expression of low level logic and primitives, but also on the organisational aspect of programming. I think there is still a lot of potential for both textual coding and visual programming tools to create better wayfinding systems.
n4kana 5y ago
“Better for data flow than control flow” Here’s a fun use-case that barely made the post: commercial/install Audio DSP. I say barely because Max/MSP did make the list, but that’s not typically for install work (think stadium, convention center, airport, etc.) The AV world is full of VPL examples. The audio DSP ones that come to mind are QSC’s Q-Sys Designer, Biamp’s Canvas, Symetrix’s Composer. There are many others, but they’re all built on the premise of an expensive hardware processor with free configuration software. Many of these processor/software pairings are configurators rather than VPLs. However, VPL is universal for more complex Audio DSP because audio schematics are very common, and essential processing algorithms are well defined with universally accepted names like “Compressor,” “Parametric EQ,” and “Mixer,” among many more. Where it’s been getting interesting in the last 5-10 years is the growth of Audio DSP into more flexible control products. Q-Sys Designer is leading the push with nodes that allow Lua scripting, which now supports Blockly. There’s an awkward transition between the primary audio VPL and the Lua/Blockly VPL though. Lua/Blockly supports the event handling features within a predominantly data flow driven application. They also allow the Audio DSP environment to interface with APIs that aren’t supported by plug-ins or other canned modules in the software. Recently QSC has been selling Dell servers configured to run as central processors, highlighting the Linux backend as opposed to other proprietary real-time DSP OSes. On some level, these VPLs have catered to keeping it simple for guys in vans with USB cables that need to service equipment on site. However, with more remote service possible, this has decoupled the van travel from the programming, allowing it to scale in complexity. It seems like the software could be licensed for use on just about any server if there was a financial incentive. It all points to an interesting dynamic between visual and text programming as programming talent crystallizes as an off-site role. E.g., GUI buttons are instantiated through the node-based VPL then copied to a GUI canvas where they can be visually customized. This could easily be replaced by a web programming paradigm but will likely remain because the AV industry has a long legacy of guys in vans with USB cables.

266 points by telotortium 5y ago

I guess this is turning into another "visual programming" post. On my side I'd like to ask your opinion about Blueprint. It really turned me off back in the day and I always wonder why they couldn't optimize UnrealScript further instead of getting something totally new. Yeah for sure eventually everyone get used to it and it becomes the new standard ("How can I NOT do Blueprint? It's so splendid" type of posts will show up) because UE is an extremely competetent engine. But why did we bring it forward in the first place? Is there enough reason to do so aside from that some teams get to have their own marks in the world and some managers can grab more power?

dharmab 5y ago
This presentation might enlighten why visual programming is in demand for gamedev. It's about a similar system developed in-house for The Division series: https://www.gdcvault.com/play/1023382/AI-Behavior-Editing-and-Debugging https://www.gdcvault.com/play/1023382/AI-Behavior-Editing-an... In short: There are many game designers and artists who prefer visual scripting tools over traditional code.
- markus_zhang 5y ago
  But I also see some hate this style. This guy is an artist/programmer. https://forums.unrealengine.com/t/blueprints-are-a-toy-which-serious-unity-dev-would-use-ue4-without-c-f-or-unrealscript/71045/5 https://forums.unrealengine.com/t/blueprints-are-a-toy-which... BTW I do agree visual programming makes sense in certain area, but to use it for a general scripting language? Hell no...
  - jcelerier 5y ago
    The ratio of people I know in gamedev who love vs hate visual programming is on the order of 10:1
    - username90 5y ago
      I assume you work in unreal? Unity is more popular though and doesn't have visual scripting. Unreal Engine just having visual scripting and C++ with nothing in between is probably its biggest weakness, people prefer C# over those.
- gridspy 5y ago
  If you cannot access the vault, here is a YT link to the same talk: https://www.youtube.com/watch?v=rYQQRIY_zcM https://www.youtube.com/watch?v=rYQQRIY_zcM
ironmagma 5y ago
It’s nice for asynchronous programming. Promises or callbacks take some getting used to, but a block diagram is as transparent of an abstraction for asynchronous flow as it gets.
koolhaas 5y ago
> But why did we bring it forward in the first place? Speculation: visual programming is a bit more beginner friendly, and more compatible with the brains of people designing stuff in UE. I wish blueprint was around when I was 13, messing around in UT’s version of UE and having no idea what unreal script could do. Visual language can break out all possible components into UI menus, just like everything else in UE. Visual oriented people really click with tools like visual programming. Personally I think it’s cynical / arbitrary to pin the decision to corporate power struggles. Sometimes people propose new things out of interest or as a side project and then it grows from there.
- markus_zhang 5y ago
  Yeah let's ignore the cynical part. Whatever, it's already done and text scripting language is not coming back for UE.
  - spywaregorilla 5y ago
    Yes it is https://twitter.com/saji8k/status/1339709691564179464?s=20 https://twitter.com/saji8k/status/1339709691564179464?s=20
spywaregorilla 5y ago
I'm a programmer who does hobbyist gamedev and is a big fan of blueprints despite writing in text all day, for a few reasons. * A lot of game engine functionality is extremely helpful but not discoverable. Visual scripting languages make the full spectrum of the engine's API more easily discovered * Games are very stateful, and I find it is much easier to conceptualize this with blueprints than with code, which is better for functional stuff * Collapsing nodes in blueprints is much more powerful as a documentation tool than collapsable lines of code. When you collapse nodes, you can show the execution running through long text high level descriptors of what hidden blocks are doing, and without needing to create a function for that task specifically. e.g. code might do something like def update_target(): target = find_new_target() # set target to nearest enemy unit attack_pattern = choose_attack_pattern() # choose next attack pattern execute_attack() # perform attack montage and execute effects ____________________ whereas the bp would be Set next target to nearest enemy -> Choose next attack pattern -> Perform attack montage and execute effects It becomes pseudocode that is bright and on the forefront of your attention, vs. most code editors which will highlight code and lowlight comments.

ping_pong 5y ago

I worked on a visual programming tool from the late 90s to the early 2000s. The same problems apply then as they do now. There's an unwinnable war between keeping things simple and being complex-enough to do useful things. Visual stuff is great for simple things, but simple things aren't very useful. When you really need to do more complex things, you reach a limit very quickly. It becomes pretty unmaintainable very quickly and then people will "graduate" from it and go to something more convenient. My kid is learning Scratch right now, and it's taught him a lot of great stuff. But after about a year, he's ready to move onto Python. Scratch has taught him some very valuable concepts and he was able to jump into some Python concepts with ease (others still escape him). But you can only go so far with Scratch vs other programming languages. The same goes for other visual tools. In the end, people graduate from the visual style because it ironically becomes too complex because they want to keep things simple, and then you've lost a customer.

jimsmart 5y ago
> But you can only go so far with Scratch vs other programming languages. A friend has built some pretty amazing stuff in Scratch, including a BBC Micro emulator (runs 60+ games), a few arcade game ports, and lots more [1]. As a developer myself, I find it quite surprising how far one can go with Scratch. (FWIW I've never coded with it myself) [1] https://scratch.mit.edu/users/RokCoder/ https://scratch.mit.edu/users/RokCoder/
- jjoonathan 5y ago
  Were there advantages? Or was it an exercise in determination?
  - jimsmart 5y ago
    He's been in the games industry for some years now, and as a side-project he teaches kids how to code — I'm pretty certain that he builds such crazy projects mostly as a fun hobby, perhaps also partly as a showcase for his teaching. IMO, for these kinds of projects, I suspect Scratch doesn't offer many advantages (if any at all). He likes a challenge :) — More details, including an interview, can be found here: https://www.coderkids.com/blog/who-is-rokcoder https://www.coderkids.com/blog/who-is-rokcoder He's pretty responsive to comments on his Scratch homepage — and can also be found under the same name on Twitter/FB/etc. He's always been happy to help others. Give him a shout if you have any specific questions. (FWIW: I worked with him — Cliff — for a short while, back in the 90s)

Vermeulen 5y ago

Our attempt at a Scratch-like language for our game creation app Modbox: https://docs.modboxgame.com/docs/mbscript https://docs.modboxgame.com/docs/mbscript It's definitely allowed non-programmers to make complex AI/game state for their creations, without having to deal with syntax errors or other issues. We choose Scratch/Blocky since this would be used for more managing states/logic, rather than flowing data (where node based makes more sense)

marcodiego 5y ago

Something I'd like: a modern maintained DRAKON version: https://en.wikipedia.org/wiki/DRAKON https://en.wikipedia.org/wiki/DRAKON . Tried it once, generates reasonably good C code. It would be great to have a modern FLOSS version.

pritambaral 5y ago
Not for C, but someone shares your motivation enough to build a version for JS: https://drakon.tech/ https://drakon.tech/ & https://github.com/stepan-mitkin/drakon.tech https://github.com/stepan-mitkin/drakon.tech
deleted 5y ago
[deleted]

divan 5y ago

Great job on summarizing and categorizing tons of opinions in one post! Shameless plug of my take on rethinking visual programming couple of years ago: https://divan.dev/posts/visual_programming_go/ https://divan.dev/posts/visual_programming_go/

patterns 5y ago
Oh, this is great. I especially like the fire torch analogy and the animation you came up with (https://divan.dev/images/torch.gif https://divan.dev/images/torch.gif). That's exactly how I feel when reading code. Recently, I explained the problem to a dear colleague when working on a code base (with about 600K lines of code). I compared the situation to working in a large office building with all lights turned off and you have to skim through thousands of documents with a flashlight. It's comforting to know that other people feel the same way.

deleted 5y ago

[deleted]

shadowgovt 5y ago

The biggest challenge with visual programming is you have to reinvent every text-based tool. And there are a lot of them. Diff, interactive debugging, keyboard-accelerated inputs and shortcuts, search, version control... The list goes on.

koolhaas 5y ago
Side note: interactive debugging is kinda inherent with the flow based programming paradigm. The program is continuously executing with every new action and immediately reflecting the current state. To program with it is to debug. If there’s an error with a component, it turns red and everything down stream breaks as well.
- taeric 5y ago
  That falls on is face at scale. There is a reason not every bullet in a gun is a tracer bullet.
  - koolhaas 5y ago
    Flow paradigm doesn’t claim scale though. It’s about being a DSL for specific environments, often visually-oriented software like CAD or interactive art, which doesn’t have heavy 10K person team requirements.
    - taeric 5y ago
      Fair. And I think there is more here. If you are programming a singular thing, this probably has good legs. As soon as you are programming many things, though? This is why cad systems can be seen as visual programming of parts, but are never considered for this debate.
      - koolhaas 5y ago
        I’d be curious to hear you elaborate on the singular/many things point. Trying to understand which deficiency you are pointing out.

spindle 5y ago

Wow! This is so awesome! Would you please write a page like this for every Hacker News post?

physicsgraph 5y ago
Probably worth triaging which topics are recurring to perform a meta-analysis. See my comment elsewhere in this thread.

crispyambulance 5y ago

Yeah, this is going to be yet another "visual programming" topic. I sort of disagree with the intro to LabVIEW: > There are a large number of visual programming tools that are roughly in the paradigm of ‘boxes with some arrows between them’, like the LabVIEW example above. I think the technical term for these is ‘node-based’, so that’s what I’ll call them. There are NOT a "large number" of visual programming tools that are roughly in the paradigm [...] like LabVIEW. I am sure there are some but, really, how many active ones? Also, "node-based" is not the main concept. The most essential concept of LabVIEW is known as "data-flow" (https://labviewwiki.org/wiki/Data_flow https://labviewwiki.org/wiki/Data_flow). Data generated by a node "flows" to the other nodes it is connected to and when any node has data on all it's inputs, it executes and produces data which flows to whatever nodes it's connected to. This happens asynchronously. You get concurrency "out of the box" with LabVIEW. That's extremely powerful and much easier to reason about, IMHO, than the concurrency constructs in virtually any mainstream computer language. I think there's plenty of room for all kinds of languages (both text and visual) as DSL's. If they're well-designed and intuitive, they're like a hot-knife through butter for a domain expert. The nice thing about a really good DSL is that you use it more like a library, whereas with a general purpose language you would be reaching for a framework. It gives you a lot of creative latitude and it feels right. The problems start when people start using them as golden hammers, or use them for things which they're not good at.

jayd16 5y ago
>There are NOT a "large number" of visual programming tools that are roughly in the paradigm [...] like LabVIEW. They list half a dozen or so. Most of the node based systems are also flow oriented, aka nodes are functional and stateless which allows them to be mapped across any amount of input. It's a neat pattern but it's really not as uncommon as you think. The mainstream shader languages work this way, for example.
koolhaas 5y ago
> Data generated by a node "flows" to the other nodes it is connected to and when any node has data on all it's inputs, it executes and produces data which flows to whatever nodes it's connected to. I think you just described flow programming. There are at least 10+ visual languages / environments that work like this.

kingcobraninja 5y ago

I've been using LabVIEW for 15 years, first in an academic and scientific setting, then for machine vision in surveillance, then as the HMI for industrial robots. It is, for all intents and purposes, a general purpose programming language, and I think it would be more popular if it wasn't so stupid expensive. I was never interested in or good at coding, but LabVIEW made sense to me. Now I'm using programming principles I learned in LabVIEW and re-implementing them in C++ and C#. It's been a journey.

Jarwain 5y ago

I think what the author did here hits hard on why hacker news is such a great resource. To quote: > Most fields have a problem with ‘ghost knowledge’, hard-won practical understanding that is mostly passed on verbally between practitioners and not written down anywhere public. At least in programming some chunk of it makes it into forum posts. I'd be incredibly interested in seeing a series of posts like this, where authors mine hacker news and other forums for insights that they consolidate (then repost here). What other topics do y'all think HN provides unique insight on?

physicsgraph 5y ago
I agree that this type of compendium/synthesis is useful. I was amused to open this thread and see comments focused on visual programming rather than the meta-aspect. As for what HN could be mined for, a big topic is software languages. There are ways to subdivide that: by language, by optimization aspect (e.g., comparative performance). Geographic topics like cost of living and quality of life. Remote work pros and cons. Another way of identifying topics worth mining would be identify posts where @dang provides a link to previous posts and those previous posts have a sufficient number of comments.
Zababa 5y ago
> What other topics do y'all think HN provides unique insight on? Career managment I think (not sure if it's the good term for it). I see lots of people in their 40's or more that quit some FANG job for another job that earns less but is more aligned with how they want to live their life. I see lots of people advocating to quit jobs as soon as you see red flags. I think both of these are good wisdom. On the other hand, a point that's still a mystery to me is "programming language choice". I've learned useful heuristics like you can do web backend in pretty much anything, if you have no libraries for something that needs them like an AWS SDK you're going to suffer, what helps in the long run slows you down at first, but that's about it. I don't know if it's because I'm too young in this field, because I don't understand/missed something, or because there is no big truth to find.
- gridspy 5y ago

analog31 5y ago

The author alludes to problems related to complexity, modularity, and formatting, of visual code. I think a hard problem for visual languages is the sheer physical labor of creating code using drawing tools. I used LabVIEW extensively for a few months, and ended up with crippling wrist fatigue and eyestrain headaches from all of the fine mouse work and menu selections needed to write even small programs. This is an ergonomic problem with all graphical software, but is particularly acute with programming because of the amount of code that gets written. Another problem is innovation. You can quickly spin up an experimental language and try it out because the infrastructure -- editor and command shell -- already exist. This is one reason why we enjoy such a proliferation of new language ideas. New ideas in graphical software require intensive effort to try out. On the other hand, there does seem to be something about data flow programming (LabVIEW, Excel) that is attractive to beginners, that can't be overlooked. Excel is probably the number-one programming tool in use right now.

bmitc 5y ago
> I used LabVIEW extensively for a few months I’m not sure that a few months is enough to understand the true pros and cons of a language. I’m using Elixir in my day job now and have used F# for side projects, and it’s not clear to me any one is faster to program in, especially for GUIs. LabVIEW actually shines, contrary to popular but inexperienced opinion, for large projects. I have used LabVIEW rather extensively, and it seems any project would have seen massive delays if I used something else, which is why I didn’t. But I can indeed see an argument for strain when using the mouse so much. I have experienced that before and improved my ergonomic setup. As an aside, it is curious to me that many programmers are heavy PC gamers as the one true way to game and don’t complain about the mouse use there but will throw out all sorts of complaints when using it to program or use tools. > New ideas in graphical software require software intensive effort to try out. This “problem” isn’t inherent to graphical programming though. It’s because legions of programmers are convinced that text=programming and have spent decades writing text-only tools. You see this friction when using LabVIEW with GitHub, whose tooling is incompatible with the diff and merge tools that come with LabVIEW, which makes GitHub look outdated and not LabVIEW. I’m actually working on ideas to build a toolkit for graphical programming languages. There are certainly challenges though, such as the desktop GUI ecosystem being a mess and reliance upon graph data structures and algorithms (like automatic layout). > On the other hand, there does seem to be something about data flow programming (LabVIEW, Excel) that is attractive to beginners Dataflow is attractive to advanced users as well, which is part of the draw to languages like F#, Elixir, Racket, Clojure, etc. Although, the dataflow in LabVIEW is actually more advanced, which is basically a multithreaded and branched version of the single threaded linear pipelines found in the languages I mentioned. Working in F# and Elixir with pipes can be pretty irritating sometimes given my knowledge of LabVIEW.

forinti 5y ago

Visual Programming is great for selling: look how simple it is to do this simple task!

taeric 5y ago
The cynical side of me thinks that is the basic pitch for every framework and methodology.
- jpindar 5y ago
  Look at the kinds of examples people are using to demo Copilot.

erichocean 5y ago

Post was useful, more of this kind of thing please!

jcims 5y ago

There are certainly many many instances where visual programming doesn't do the job. One instance where I found it incredibly useful, though, was with Node-RED. I used it to build a hobby-grade process automation solution, including a number of software PID control loops, that ran current limiters, valves and pumps, several load cells and half a dozen temperature sensors. All of this was orchestrated from Node-RED running on a Raspberry Pi communicating with the Tasmota esp8266 sensor/controller nodes over MQTT and I built it from scratch, learning along the way, over the course of 2-3 days. Something about the visual pipelines and processing nodes really helped with handling concurrency, priority and maintaining tidy boundaries between systems. It was the first time I really had fun programming something in quite some time.

vvanders 5y ago

One thing missing here and in most of the visual programming discussions since it sits in a somewhat adjacent SCADA industry is Ladder Logic[1]. It was pretty illuminating how many parallels ladder has with the visual and scripting solutions I'd worked with in GameDev. Hot-reload under running processes, visual debugging very reminiscent of node based visual editors. I ended up automating our greenhouse with a P100[2] from Automation Direct and was pretty impressed with how straightforward it was to use. Structured text is making in-roads in some places from what I can tell but ladder still seems to dominate because it's easily understood and maps well to a "relay based" mental model. It seems to be very sticky, at least more than most visual scripting languages I'm familiar with. [1] https://en.wikipedia.org/wiki/Ladder_logic https://en.wikipedia.org/wiki/Ladder_logic [2] https://www.automationdirect.com/adc/shopping/catalog/programmable_controllers/productivity_series_controllers/productivity1000_(stackable_micro_plc)/cpus/p1-540 https://www.automationdirect.com/adc/shopping/catalog/progra...

mikewarot 5y ago
Ladder logic is powerful stuff. I learned it at Westinghouse in the 1980s. It's a non-Von Neuman based programming methodology well suited to real time environments. Every ladder gets executed once per cycle, and there are usually a few hundred cycles per second. Special functions are implemented in boxes, like delays, counters, etc. My task back then was to feed the entire I/O state into a ring buffer, and when a "crash" of the steel mill happened, a host PC could then download the entire buffer, giving access to information about what happened in the seconds BEFORE the crash. It would be fairly easy to port to the Arduino, if someone hasn't done it already. Yep - it's been done https://cq.cx/ladder.pl https://cq.cx/ladder.pl Because Ladders are text based, you could use Teco/EDT/Vi/Emacs/WordStar macros on them.
- vvanders 5y ago
  Yeah, it's a pretty neat ecosystem. PLCs are kinda like the Legos of automation, incredibly easy to mix and match with a fair number of (insecure) standard protocols. I briefly toyed with the idea of building out some open source hardware/software but I didn't really seem like there was enough of a userbase to make it viable. Anyone using it professionally wants support and 10+ years of parts, most maker/hobby projects want direct access to code. Still some pretty neat concepts for putting together highly customized processes with extensible and interchangeable hardware.

w_t_payne 5y ago

I believe that diagrams are at their most useful when they are deployed as a communications tool. When discussing our software, we commonly draw a diagram to help illustrate how functionality is divided up among components, and to show which components are impacted by particular functional chains. They are a powerful tool for communicating and coordinating the work of multiple people or multiple teams. If our software does not provide us with diagrams itself, then we will commonly create them ourselves in our technical documentation or on whiteboards as an aide to collaboration. However, as we descend in our system towards finer and more detailed levels of granularity, two things happen. First of all, fewer people are impacted by very ‘local’ design decisions, so the importance of communication diminishes. Secondly, the frequency of change increases, so the importance of mutability and ease-of-editing increases. The requirements that we place on our engineering tools depend largely on what we are trying to do with them. Communicating and coordinating work across a large engineering organization places fundamentally different requirements on our engineering tools than does, say, working alone to solve a challenging problem that requires the creation of new concepts and abstractions. There are many different ways to work with software, and although each share much of the same flavor (dealing with informational complexity), the nature of these challenges is often distinct enough that the different requirements may lead to vastly different solutions. For my part, I have a lot of faith in the general utility of tools which allow for the easy generation of diagrammatic representations of high-level structure, but which naturally decay to a more conventional textual representation at finer levels of granularity.

slx26 5y ago
Yeah, to make sense of complex systems they need to be well structured and modularized, and visuals have a lot to contribute here. If you can't fit your structure comfortably in a diagram, your structure is too complex for most people to manage. Further down the line you stop needing to know where you are (global view, understanding), and you only need to deal with implementation: actually doing what you need to do. Implementation can be often very complex on its own too, that's why algorithms are so scary at the beginning... and text is a better tool here. There's not much more structure to distill, and some parts are tricky, and you have edge cases and you often need to get your hands dirty. You need to keep all this complexity in mind at the same time anyway, and after you are done you can treat the implementation as a black box. So text is superior here. That's how we manage complexity. Now we only need tools that can represent it better. But maybe it's not even visual programming languages, but rather visual system "structurers"? There are other parts of programming that could use better visual cues, and visual editors for visual stuff (colors, shapes, borders, etc.) will always be relevant, but those are all very different concerns.

Timothycquinn 5y ago

I wrote a spreadsheet based rules engine at Honeywell in around 2003 which enabled the business to maintain the business rules for all sales contracts that ran through the automation and control systems division. This gave them the ability to define forms, subsets of forms (legal or otherwise), notifications and approvals. I had its own natural language based expression language which the super users and business analysts could easily understand. The spreadsheets, were change controlled by the business an imported into the system when they were approved. I considered this design a form of a visual language and the real beauty was that it did not require any re-coding of business rules, which did change quite frequently. The developers did not have to be re-writing any business logic as the rules changed. The back end was Java and TCL based built on a graph database. Not super fast but super easy to understand and train up new developers. It was quite successful and ran all sales contracts for for several years with billions of dollars of contracts passing through per year. I lost contact with the teams so I'm not sure how long it survived but I know that SAP charged millions of dollars trying to build a solution in their systems but could not get one working at a reasonable price.

cruise9 5y ago
I'd love to hear how you integrated a Java and TCL backend with spreadsheets. I'm assuming this isn't something you can do with excel? (Logistics at the small business I work at, is run on a mess of excel spreadsheets with complicated formulas and occasionally VBA for some email functionality. I have a feeling we're doing things horribly wrong, but I don't know of any better way to do things with our zero IT budget.)
- Timothycquinn 5y ago
  I did not leverage any VBA or VB. I learned to code in VBA and written way too much code in that space and was well aware of the limitations. I'm pretty sure the way it went was, on check-in of a new version, I had a Java program that read the spreadsheet validated that all the right sheets were there and that the columns on each sheet conformed with the spec and also validated rows of data. The sheets were translated on each front end node for easy access. Regarding the expression engine, the the expressions themselves were then translated into TCL boolean expressions. The attributes for the related objects where already loaded into memory into dictionaries and then the expressions were just evaluated to get a boolean result.

dannywarner 5y ago

I like the line "There’s a comforting familiarity in reading the same internet argument over and over again." The problem I think with visual programming tools is people get stuck. I watched my dad's career die stuck in Visual Basic. It's because people get stuck in thinking the way those tools force you to think. Code in text has a magnificent universality.

mikewarot 5y ago
I'm curious what you mean by how Visual Basic makes you think.

manbart 5y ago

I have to write scripts for Cisco UCCX (call center software) at work sometimes. Never heard of it before this job. I was horrified that uses a visual ‘drag-n-drop’ IDE to create the call handling scripts, no other supported method. It’s the only ‘visual’ coding I’ve ever used. And in the 2 years I’ve used it, I actually don’t hate it. As other posters have mentioned, a visual approach can be decent DSL. Btw, you can run still run arbitrary Java (1.7) in UCCX scripts too, which is kinda cool. Used that to make email alerts in case of certain situations in the call queue

epage 5y ago

My feedback, based on my both long but limited LabVIEW experience > IDEs for text-based languages normally have features like code folding and call hierarchies for moving between levels, but these conventions are less developed in node-based tools. This may be just because these tools are more niche and have had less development time, or it may genuinely be a more difficult problem for a 2D layout — I don’t know enough about the details to tell. NI demoed a system to function zoom in/out feature at one point but never fully developed it. LabVIEW does have a call hierarchy. Like with Doxygen, my problem with these is they show everything rather than finding ways to highlight important relationships (e.g. you don't want to show every array append). > Input I suffered from RSI when using LabVIEW. Maybe I hadn't mastered enough but there is a lot more than just dropping that needs to be improved, but the wiring. > Formatting I think LabVIEW suffers from giving people so much pixel-perfect control, it encourages people to waste time on it because its "so close". At one point, they demoed dynamic formatting (reflows the diagram as you add/move nodes without any manual management). This really needs to be made the default. > Version control and code review LabVIEW NXG has a text-based GVI format. It is not easily reviewable, mergeable, etc because it has all of the graphical minutia in it. My proposal would be - Remove block diagram formatting (see above) - Separate top-level hand-designed UI from VI auto-placed front panel - Reduce reliance on icons - Encourage naming wires At that point, a text-based VI format would be simple enough to be manageable within existing tools.

ziml77 5y ago
I had to suffer through a semester of LabVIEW. It seems like it might be okay if you have high enough level blocks of functionality available that you're never hooking many together, but for what I was trying to do it was a massive pain (doable, but tedious and hard to organize and follow)
bmitc 5y ago
> At one point, they demoed dynamic formatting (reflows the diagram as you add/move nodes without any manual management). This really needs to be made the default. Do you by any chance have a link to a video or article where this is shown or discussed? I’m a heavy user of LabVIEW, and I am also convinced in my own explorations of creating visual languages that dynamic automatic layout is the way to go, but it is a very hard problem.

tehsauce 5y ago

really cool survey of the topic, nice work!

mikewarot 5y ago

For completeness, I feel compelled to mention GNU Radio. It is a node based visual programming tool. It has its quirks, but you can do fairly complex digital signal processing with it, and it will do I/O from your sound card, as well as RF stuff.

jpindar 5y ago
And its one of the more common ways to make those little Software Defined Radio boards do something.

MetaWhirledPeas 5y ago

Years ago I bounced around from one visual tool to another, never mastering any of them but getting deep enough to see the flaws. Hypercard, Authorware, Director, Flash. They were all very approachable, but they were all incredibly flawed in the end. You'd spend most of your time jumping from one limitation to the next, trying to hack your way out of the spaghetti jungle. I think the better approach is to use conventional languages with UI tools layered on top that at worst generate readable code. (Like I assume is often done in game development.) Source control and diffing are indispensable. The lure of these tools is that they are approachable, which sometimes is exactly what you need to get started. It's just very rarely what you need to finish. So let these tools be used to get people interested in programming, or to give people an idea of how to accomplish a project. But more importantly let's make sure to continue taking steps to demystify coding. Not because we want everyone to be programmers, but because when people do choose to program we want them to be able to collaborate with the rest of the world, and when they leave we don't want to inherit some visual pasta dish.

musiccog 5y ago
I loved MacroMind (later Macromedia) (later Adobe) Director. Delivered dozens of educational CD's, corporate CD's and many of my own 'experiments'. One of those was a map for EverQuest to help find dead bodies (it was a big thing at the time), and help swimmers who fell off a ship to get back to land alive. Another was a project to scan every user-authored book in Ultima Online (using basic OCR) Director was extensible in that you could not only import 3rd party libraries, but write your own modules in Pascal. Much more powerful and extensible than Flash IMHO. Flash won that battle after the buy-out, and Adobe extinguished Director. Still miss it to create proof-of-concept, simple or amazing cross-platform apps.

Sophistifunk 5y ago

We need both. And more. Visual projection and editing is valuable when looking at a zoomed-out "components and wires" sort of architectural view. It's also good for showing high-level overviews of a process, like various UML diagrams should have been. It can sometimes be used well for the nitty-gritty implementation details of a single unit, although this seems to be where a lot of it breaks down. But it's terrible for doing all of these things at once. And also, if you can control your level of detail/zoom well enough to do it right, you'd get similar benefits projecting to simpler text formats as well. What's most important is that all this stuff needs to be declarative. Not in the sense that you hide the order of operations within a block of code, in a way that final performance characteristics can become a surprise, but that the more complex structures of your project that organises the blocks into modules, classes, functions, records, etc need to be declarative, rather than an imperative set of instructions that build a structure at run-time (usually difficult or impossible to explore and instrument) which is later executed to achieve your actual goals.

raphlinus 5y ago

> Is ‘folk wisdom from internet forums’ worth exploring as a genre of blog post? Yes. I found this very well done and useful. One other (not entirely unrelated) potential area that immediately comes to mind for me is the architecture of UI toolkits. This is an area where there is very little serious (ie academic) writing, and even less good writing. The vast majority of what's set to paper is marketing material for a particular toolkit. There are some potentially interesting blog posts, but these tend to be highly opinionated. Adding to this, while most of the discussion today is around shiny new stuff like SwiftUI and React, there's actually a pretty significant "literature" of older systems. It's hard to know what's worth digging into. And, as with visual programming, there's not much in the way of synthesized understanding about the best way to do things. Game programmers like imgui, but it hasn't caught on outside that domain. Why? React is massively popular, but there is criticism of its performance (and of web-based technology stacks). There are other debates about the value of being "native," which perhaps used to be a clear distinction but is fuzzier today when you look at it closely. So I think something along these lines for that topic could be pretty valuable.

galaxyLogic 5y ago

What I find interesting is that the human society and economy at large is able to function without there being any genius programmer who designed it all. If you look at roads, there is a road everywhere and still people don't get lost. But in a big chunk of source-code I do feel I get lost. I believe that may be because there are too few constraints when programming textually. Whereas when designing roads and bridges we have the constraints of physical space. It is possible to draw a 2-dimensional map of all the roads and thus easily understand it all. I think visual programming was inspired by this notion, how physical and geometrical constraints make things simpler because we know those constraints intuitively, which thus helps us reason about the structure of the whole thing. It can not violate physics, nor geometry. Textual programs can, do anything their syntax allows. That makes their full structure hard to understand

hermitcrab 5y ago
Habe you seen an aerial view of Birmingham's "spaghetti junction"? ;0)
username90 5y ago
A city with streets is just a graph with 1-4 pointers per node. It is super simple in code, just a few lines, not sure how you could get lost in that. If you make code anything more complex than that then it isn't really equivalent to navigating the streets on a map. > I believe that may be because there are too few constraints when programming textually No that is not the issue, you can write extremely complex and useful programs in just 100 lines of code and you can translate those to visual programming easily with no improvement in comprehension. Code just takes long to read since it is so powerful.
- galaxyLogic 5y ago
  > If you make code anything more complex than that then it isn't really equivalent to navigating the streets on a map. My point is, you should strive to make coding not more complex than that, if possible. And if you have constraints such as must represent the program as a 2-D dataflow structure then it must stay simple as that. Now is there a need to make more complicated programs? Probably yes. But human society is still able to function with the simple dataflows that it has, which makes it understandable and thus very resilient.

arduinomancer 5y ago

Really high quality post, would love to see more like this

ffhhj 5y ago

HN is a great resource to gather different points of views from experts. Here is some folk wisdom I compiled on load and performance testing for a presentation: Selenium/Web Driver handles the UI tests. Postman and a handful of others can perform the API tests. JMeter is the de facto for load testing at every place I've been to. And you can write any of these into your CI. There are three separate reasons to do load testing: 1) Performance testing. Confirm the system does not degrade under a specified load and find out what performance can be expected under these circumstances. This is basically ensuring your system can handle X amount of traffic without issues and knowing your baseline performance. You should get the same kind of response times as you are getting from live server telemetry. 2) Stress testing. finding out what happens when the system is stressed beyond its specs. How does it degrade & where. 3) Reliability testing. Find out how your system breaks and when. The goal here is to try to break the system and test things like failover and making sure you don't lose or corrupt data. Better to die gracefully then abruptly. I've used Jmeter quite a bit over the last year, and what seems to be the largest issue is that it doesn't "break" apps because it works at the protocol level and isn't a full "browser". As you increase load and response times start to increase, since it runs sequentially through the test plan the time between the requests also increases. But, when actual humans with browsers use apps there are loads of AJAX requests being fired which don't necessarily go in order or wait for others to complete first. Most managers only want to see how traffic will perform in what-if scenarios (can we handle Black Friday, what will happen if our traffic goes up 10x during a special event, etc). For these, JMeter and whatever service you're using for analytics/metrics work perfectly fine (New Relic, is the current favorite). Then we can compare average latency, Error Rates, etc. with what we get during JMeter. The issue is that most managers or other PHB want more than just a one off for load testing. And this article greatly covers that. A lot of time people want a Swiss Army Knife of tools for "automation", where load testing falls under that category for them. This magical tool should be able to: * Test your API calls from a functional, integration, and a performance (not load testing; just making sure we're under specific latency) * Test your Web-sites from the same testing perspectives, as well as being modular (ie: Selenium's POM) * Integrate with CI so specific tests, test suites/flows can be tested for every new build, and we can run specific workflows by clicking a button * Be used during load testing so we can measure latency and run tests while recreating customer experience from the Web side of things. Last time I had to do load testing in a professional context, we required 10+ million long-lived TCP and websocket connections transacting multiple times a second. There weren't any off-the-shelf solutions at that time that could come within an order of magnitude of that at a reasonable cost - the most viable solution we tested required a thousand EC2 instances to sustain that traffic. Do red line testing. Gradually increase load on your service with concurrent requests until it is saturated. Measure how the latency and errors progress with more concurrent requests and understand at what point and how your service starts to break down under heavy load. Based on this you may have to do many things – 1. Can you optimize your service or your service's downstream dependencies or the application calling your service. 2. Can you build in graceful degradation into your service – functionality reduction to get more useful throughput out of your service – with same resources. 3. Build circuit breakers and throttlers before your downstream dependencies so that you don't overload them and cause them to fail or you don't fail totally when they do indeed fail. 4. If you do get overloaded for some reason (say your server pool suddenly became half its size), are you able to recover quickly. 5. Do you have monitoring and alerts for these load scenarios? I tend to avoid having to do load testing as it sucks up time without telling me much of interest. I instead opt for having decent telemetry on the live system. It will tell me how it performs and where the bottlenecks are. I can set alerts and take action when things degrade (e.g. because of a bad change). Besides, there is no substitute for having real users doing real things with real data. And in any case, having telemetry is crucial to do any meaningful stress or reliability testing. Otherwise you just know it degrades without understanding why.

bmitc 5y ago

Funnily enough, I’ve been quoted in this. ;) To expand on modularity, it is indeed curious to me why many will often throw it out that visual programming somehow doesn’t scale for complex and large programs and fails at modularity. Let’s look at LabVIEW as compared to F# and Elixir, two text-based languages that are wonderful to work in and generally well loved, where LabVIEW is well loathed. In LabVIEW, you have projects to organize code, libraries to serve as modules or namespaces, the ability to create functions (including polymorphic and malleable VIs, which are a way to auto propagate types to make functions auto polymorphic), clusters (which are basically structs or records), immutable data, OOP with interfaces (value-based and not reference-based, which is a major plus and not found in many languages), concurrency features, and actors. F# has projects, namespaces, modules, immutable data, records, discriminated union, OOP (reference-based), pattern matching, concurrency features, and some actor frameworks. Elixir is similar with all its features with some small differences. Now, these are all essentially the same fundamental tools available to create programs and to modularize them and scale them. So where’s the rub? How is it that you can somehow create modular and large complex programs with Elixir and F# but not LabVIEW? Where’s the actual discrepancy? Throw in other languages to this discussion and you get hoards of inheritance hierarchies, memory access problems, and mutability, all of which are well known to cause issues with scale and complexity. So I’ve never understood the argument. (I actually do have arguments against LabVIEW for large complex problems but it’s not the arguments usually made and more relate to the IDE than the language). Anecdotally, I once interviewed with a place that was vehemently giving up on LabVIEW. They gave no real reason other than they hated it and said it didn’t work for large programs. So I asked to see one of their Python programs, which was their new language of choice. I was shown a single file that was something like 10,000-15,000 lines long with functions often having signatures taking 10-20 lines, meaning the functions had something like 10-30 function arguments. So at that point I knew that these people didn’t actually know how to properly architect software and that LabVIEW wasn’t actually their problem. They were their problem. As mentioned below my part that was quoted, it can indeed be argued that LabVIEW’s visual nature is a feature when it comes to showcasing poorly organized code. It makes it bare, right there in front of you and in more dimensions than what you get in text. You don’t necessarily get that with text, so people who write poorly organized code in any language but then use LabVIEW are greeted with a visual experience they’re unhappy with. They, incorrectly, see the problem as lying with the tool and not themselves and their programs. All that being said, there are absolutely problems with LabVIEW and other visual languages, just like there are with text-based languages. I’d just like to see more objective discussions of these things because I believe a text-only view holds things back. I’d love to see programming computers evolve beyond the languages, editors, and IDEs we have now.

jjcc 5y ago

off topic but just triggered by the article. I strongly suggest somebody write a bot on HN and maybe leverage GPT-3 to harvest collective wisdom on many topics. It's a golden mine.

henrikeh 5y ago

There is a type of visual programming which is applied to virtually everything around you of some mechanical complexity: interactive geometric constraint solvers. In CAD tools like SolveSpace, SolidWorks, Inventor and several others, the dimensions of objects are not described directly, but instead through a set of constraints such as “these are orthogonal”, “these have this distance”, “this is a tangent” etc. If you have never used it, try SolveSpace, which is free. Albeit highly declarative, it is absolutely a programming language for a specific domain. It doesn’t try to be good at describing imperative programs or flows; instead it presents a paradigm which is very different, but fits perfectly in the problem domain of mechanical designs.

imvetri 5y ago

One more into that list. Flow based with input as impulses.

knolan 5y ago

I really like the node based workflow used by Blender for creating materials, compositing and now for generating geometry. This is where I think visual programming works best, realtime interactive tweaking of something actually visual. It's also generally kept simple and is powerful enough to get the job done. I've always hated LabView however, it grows too unwieldy quickly and the limited nature of its visual display doesn't help. I remember once going to a NI training session around 2008 that was actually a sales session and their guy tried to convince us that LabView was easily parallelisable by drawing two empty while loops and showing them maxing out both CPUs on his laptop. The mind boggles.

auxym 5y ago
Arguably he's not wrong? I don't love LabVIEW but I make it work because their hardware is not bad compared with alternatives. Dragging two loops is pretty easy compared to figuring out threads and thread-safe communications in C++.
- knolan 5y ago
  The point is that two empty loops should not be let run the CPU to 100%. But yes, the concept is nice at least. I agree on NÍ hardware, it’s very solid. I’ve had good success using session based control with Matlab and also with Python.

simonh 5y ago

"A picture paints a thousand words, sure, but here's a thousand word essay. Try painting me a picture of it." I don't remember where I hard that, or close to it, but there's a lot of truth to it. There do seem to be some very useful visual logic and work flow tools, I hope that continues, but there's something magical about the ability of language to explain, describe and precisely define things. Maybe eventually we'll arrive at a state where really good programming environments incorporate the advantages of both textual and visual tools in a powerful and dynamic way. I love films and picture books, I also love novels, but even the best novels are enhanced by really good illustrations and sometimes even maps. Some authors find visual tools like relationship maps and even geographic maps help them in the actual authoring process too.

hutzlibu 5y ago
"Maybe eventually we'll arrive at a state where really good programming environments incorporate the advantages of both textual and visual tools in a powerful and dynamic way. " Yes, I agree. But not surprisingly, this is not easy to solve. You need a visual language as a consistent concept and then you need a actual visual framework, rendering it all, and enabling user input. You need a compiler compiling forth and back, between text and visu. And then it needs to scale. And be performant enough to actually work with it and not just have a visual demo. Sounds hard? Trust me, it is.
- simonh 5y ago
  I listened to a podcast a while ago where someone was speculating why programs processing data can't just be connected together like devices in a water system. Pumps, sinks, boilers, showers, etc. Or like electrical devices, just channel the data between them. It occurred to me that data isn't like water, or electricity. Those are generic resources that can largely be easily standardised and treated consistently by any device. Data isn't like that. Data formats and encoding vary hugely and can have crazy different properties. It's more like piping chemicals between devices in a chemical factory. You can't just take the output from a device producing acid and plug the pipe into the input of a device that's designed to process water, or ammonia, or petrol. They're all 'just' fluids and you can pipe them about, but their properties vary wildly. You can only feed one into a device as input if that device is specifically designed to process that specific material. Yes it is possible to define standardised data formats, up to a point. XML, JSON, CSV, etc but even then you can't just feed arbitrary JSON into every program designed to ingest JSON and expect it to work, just because the data it expects is JSON formatted. Yes Unix has pipes, but each tool in the pipe chain has to be told exactly how to process the specific input it gets from the previous tool. You can't just look in history for two arbitrary examples of using pipes, and cut and paste the first half of one pipe chain, and paste the second half of another arbitrary pipe chain on the end, and expect to get something useful out of the combination. Maybe you'll get lucky, but usually you'll get garbage.

k__ 5y ago

It think, VP shines in specialized fields and not general programming. Visual domains are an obvious example. If your domain is constrained by something that maps good to spatial dimensions than you're good to go. A few extra dimensions can also put into styling. But after that it becomes too restricting. That's why interface builders work fabulous until you have to implement actual business logic and modularize your application. This always leads to adding things that aren't directly represented in the current visual model and get lost, because of indirections.

akavel 5y ago

Mandatory fanboi mention, and IMO a gross omission of the article: https://enso.org/ https://enso.org/ (née Luna) - an open-source language & IDE which aims to insta-kill all the "vs. textual programming" arguments by keeping a strict 1:1 visual<->textual representation translation as a core feature of the language. A 2.0 alpha was recently released, go check it out!

auggierose 5y ago
Very nice. I am also currently building a new environment for programming (I want my 5 year old niece to get a head start on modern creativity :-)). I think the main idea is simple: logic + visualisation + data. Make something that takes all of these three things seriously. Programming languages usually care about the logic part only (and even that not very well).

RobertKerans 5y ago

That was a good summary, was useful and really enjoyed reading it. The key takeaway for me, which really reified ideas I have about UI design tools, was this: > At the opposite end of the spectrum is, say, an oil painting, which is also a visual medium but much more of an unconstrained, freeform one, where brushstrokes can swirl in any arbitrary pattern. This freedom is useful in artistic fields, where rich ambiguous associative meaning is the whole point, but becomes a nuisance in technical contexts...[elided]...Drag-n-drop editors arguably lose a lot of the features of ‘true’ languages by giving up structure, and more programmatic elements are likely to still use a constrained set of primitives. That's the key problem afaics, and why UI design tools + codegen never works well outside of highly constrained situations. There are two conflicting practices: programming and visual design. And they overlap, obviously, but they don't properly mesh. The tool either goes in the programmatic direction, which makes codegen much easier but drastically curtails the ability to design properly. Or it goes in the design direction and makes codegen either very difficult or forces generation of absolute garbage code (for worst examples, see tools that generate enormous amounts of absolutely positioned HTML elements with inline styles on everything). And there's no real middle ground (not that this stops endless attempts to square the circle).

snidane 5y ago

The benefits of visual programming are usually that it captures the domain model better than using general programming languages without any guidance. Lot of the value is in well maintained integrations to other systems. Often, if the components of the visual language were released as a library or a framework, it would provide more benefit than having to drag and drop visual boxes and arrows which don't ever make it to version control or in some very clunky way. Releasing the composable parts as text based library will not happen, because of economics. There is little money in creating libraries for the creators. All generated value would be captured by infrastructure, cloud platform providers. With visual languages, the value can be captured by selling a no-code idea to non-programming folks and especially to enterprise with deep pockets wishing to get rid of expensive engineers. Visual programming languages are an economical problem, not a technical one.

ianhorn 5y ago

What a fantastic post. > Is ‘folk wisdom from internet forums’ worth exploring as a genre of blog post? I’d add another yes here, if they’re all as thorough as this post. I’ll add something that I didn’t see directly addressed in here: input methods strongly affect the medium. Last year I struggled intensely with RSI. I got to the point where I couldn’t type for months. I tried some voice coding tools (caster and talon). Some aspects of this were actually better than keyboard coding. Unfortunately speech recognition is still at a point where it drove me insane with inaccuracies. It also made me realize that with a keyboard, text code is incredibly natural. With keyboard+mouse, some new modes open up. With voice alone, I wanted to code a very different way. I won’t go into detail, because this is already a long comment, but I believe that when we get speech recognition as reliable as typing, we’ll see an explosion of new programming paradigms.

Lichtso 5y ago

Overall a really good essay. I would have split the "Types of visual programming" in two main categories: Visual and graphical. E.g. scratch is only visual because it is an over-the-top syntax highlighting but still has an AST like textual programming (they are based on hierarchies and sequences). Spreadsheets are also sequential but in multiple dimensions. Dataflow- / node- based systems on the other hand are actually graphical (graph-based) and can not be mapped to hierarchies in text directly.

grayrest 5y ago

> So presumably we could learn to distinguish closely packed shapes if we were familiar enough with the conventions. The data visualization people have already demonstrated this with sparklines and small multiples. There are fewer preattentive attributes than most people would assume so I expect increasingly dense visual representations to approach text (e.g. logograms). I'll also point out that data viz people have also come up with reasonable solutions to problems mentioned like wire bundling. I haven't seen an auto-layout algorithm I like but I've found a reasonable guess followed by manual manipulation with automatic wire routing to be okay for visualizations and the circuit drawing and VLSI I did 20 years ago in college.

getflourish 5y ago

The strength of visual programming is not in visualising code, but rather, visualising programs. At a small scale, textual code can be very concise, expressive, compact, efficient and fast to write. But at a large scale, writing thousands of lines of code in text files in folders makes it very difficult to understand how the pieces of a program connect. To overcome this, we rely on file naming conventions, frameworks or writing documentation to understand how the various parts of text relate to each other. Programming with nodes and wires is a way of visualising those relationships by creating a map of the program. Similar to maps of the real world, those maps may be complex and not easy to understand. But those maps show that there is complexity. what we can learn from maps of the real world? Well: zoom level matters. When we want to understand the world, we look at countries. When we want to get to the next city, we look at the road level. In my opinion, visual programming environments are transparent and honest about the underlying complexity of a program. Contrast this with countless folders full of files filled with thousands of lines of code. Such repositories of code can be very difficult to navigate and get around. Yes, frameworks and conventions can help to understand already learned organisational systems (example from the real world: “I learned to navigate American cities designed on a grid of streets, but I get lost in Europe“). If we compare textual coding to visual programming, we should not only focus on the expression of low level logic and primitives, but also on the organisational aspect of programming. I think there is still a lot of potential for both textual coding and visual programming tools to create better wayfinding systems.

n4kana 5y ago

“Better for data flow than control flow” Here’s a fun use-case that barely made the post: commercial/install Audio DSP. I say barely because Max/MSP did make the list, but that’s not typically for install work (think stadium, convention center, airport, etc.) The AV world is full of VPL examples. The audio DSP ones that come to mind are QSC’s Q-Sys Designer, Biamp’s Canvas, Symetrix’s Composer. There are many others, but they’re all built on the premise of an expensive hardware processor with free configuration software. Many of these processor/software pairings are configurators rather than VPLs. However, VPL is universal for more complex Audio DSP because audio schematics are very common, and essential processing algorithms are well defined with universally accepted names like “Compressor,” “Parametric EQ,” and “Mixer,” among many more. Where it’s been getting interesting in the last 5-10 years is the growth of Audio DSP into more flexible control products. Q-Sys Designer is leading the push with nodes that allow Lua scripting, which now supports Blockly. There’s an awkward transition between the primary audio VPL and the Lua/Blockly VPL though. Lua/Blockly supports the event handling features within a predominantly data flow driven application. They also allow the Audio DSP environment to interface with APIs that aren’t supported by plug-ins or other canned modules in the software. Recently QSC has been selling Dell servers configured to run as central processors, highlighting the Linux backend as opposed to other proprietary real-time DSP OSes. On some level, these VPLs have catered to keeping it simple for guys in vans with USB cables that need to service equipment on site. However, with more remote service possible, this has decoupled the van travel from the programming, allowing it to scale in complexity. It seems like the software could be licensed for use on just about any server if there was a financial incentive. It all points to an interesting dynamic between visual and text programming as programming talent crystallizes as an off-site role. E.g., GUI buttons are instantiated through the node-based VPL then copied to a GUI canvas where they can be visually customized. This could easily be replaced by a web programming paradigm but will likely remain because the AV industry has a long legacy of guys in vans with USB cables.