5 ms·

> So what do you compose? A functional block with ins and outs? What's the equivalent of a subroutine? That's where graphical systems usually fall down. Most v

> So what do you compose? A functional block with ins and outs? What's the equivalent of a subroutine? That's where graphical systems usually fall down. Most visual programming languages have nodes with ins and outs that contain other nodes, just like you’re suggesting. And yes, I’d consider them the equivalent of subroutines. Programmers seem to talk about visual programming languages as if they’ve failed, but outside of software engineering they appear to me to be more popular than text-based languages. Some examples are VFX, compositing, audio/MIDI, prototyping, and automation. My own working hypothesis is that plain-text based version control is why most software engineering uses text files. The big differentiator between software engineering and other fields is there’s very little collaboration with visual programming language projects, they’re almost always made by one person.

munk-a 6y ago
I think version control is a decent reason, but it's more the full ecosystem. As an example, how can I grep a set of visual functions for some specific pattern - what even is a meaningful pattern to look for. That lack of an ecosystem makes most devs nervous and we tend to be (rightfully) distrustful of all-in-one ecosystems since your program (probably your livelihood) is then tied to the continued support of that ecosystem. It's one reason this movement is interesting to me, if we can establish a common communication medium, specifically a non-proprietary one, then tool builders can come along and start building out that ecosystem - likely as inefficient hacks initially but ones that can be refined as the platform gets more focus.
mypalmike 6y ago
I don't think version control has anything to do with it. If there were a highly compelling and productive general purpose visual language, storing the data as text, and even providing tools to aid in things like merge conflicts, would be minor details. But when you actually dig into things like (the excellent) Prograph and its descendents, you find out that shapes and lines are actually less intuitive than text for nontrivial code bases. Contrary to what one might expect, a picture is not worth a thousand words. Perhaps a new visual paradigm will be introduced that changes things, but there's really nothing today that competes with even your least favorite popular text based language.
- robenkleene 6y ago
  Why wouldn't you say the examples I gave above don't compete with "your least favorite popular text based language"? My whole point is that not only do they compete, but they are actually winning in most use cases outside of software engineering. Once you take that assumption (which I'd love to hear someone debate), the question just becomes why do plain text programming languages win for some uses cases and not others? The discussion of "general purpose" programming languages doesn't really make sense with respect to visual programming languages because visual programming languages necessarily exist within a GUI, and GUIs themselves don't scale to general purpose. So what we actually have is specialized visual programming languages that double as (or exist within) specialized GUI applications. A system that appears to be working quite well in creating much more efficient systems for non-programmers to accomplish programmatic tasks. This kind of poor mapping of textual programming languages to certain problems is why the Stripe globe is state of the art text-based rendering (https://stripe.com/blog/globe https://stripe.com/blog/globe), when people are doing algorithmic architecture in Houdini (https://mvsm.com/project/imac-pro https://mvsm.com/project/imac-pro). It's not even comparable, text-based languages are getting absolutely decimated outside of software engineering by better tools that leverage visual programming languages.
  - creata 6y ago
    Your link to "algorithmic architecture" was such a letdown. These guys weren't doing architecture, they were just making a flashy Apple ad! Can you expand on what you mean when you call the Stripe globe and the Apple ad "state of the art"? I assume you don't mean that these are some of the most technically proficient graphics you've seen, because that would just be silly. And comparing something developed by a design studio with god-knows-how-much rendering time to something designed in-house that runs live in your browser seems a little strange.
    - robenkleene 6y ago
      Yes, I would consider the examples "some of the most technically proficient graphics", but you can find practically infinite examples of impressive procedural work done with visual-programming-language-based tools. I'd love to find more impressive examples using text-based programming languages though, those are harder to find. I agree that comparing a video to something that runs live obviously isn't fair, but that's not really the axis I'm concerned about here. Plain-text code seems to struggle at the artistic level vs. visual tools, not the performance level. E.g., the counterargument I'm looking for would be impressive visuals created purely in code that's either pre-render (e.g., as a video), or running live, I don't really care, I just want to see visually impressive work.
Animats 6y ago
There are version control systems that aren't text oriented. There's Autodesk Vault, for drawings and 3D designs. There's Alienbrain, which is for games, movies, and even automotive designs, where assets can include a game level, a movie clip, a 3D model, a motion capture session, and a fender. "Previews are available for several hundred file formats". There's even the new NVidia Omniverse, which is even fancier.
jmiskovic 6y ago
I have another theory why it doesn't scale. Visual programming is very spatial. This may seem helpful at first but actually it gets in the way of development process. Developing isn't just writing (drawing) new code. It is iterative process of juggling many pieces of functionality so they work well together. When you have to refactor some logic, suddenly blocks move to different places, and their relative positions change. Whenever related blocks change relative orientation, your internal model for navigation becomes obsolete. Human brain is not used to this and it creates discomfort and frustration. Traditional programming done with text files in which functions have 1D organization. Because 1D has less spatial relations (just above and below), code reorganization is cognitively easier. It's easier to find good place for piece of code and it's easier to remember where it lives. I think a truly usable visual programming tool should not allow users to place blocks at arbitrary positions, but should automatically (and predictably) arrange pieces on screen. Structural text editors (for example [1]) are step in right direction, while connected blocks seem to me like a dead end. Another problem is structuring the code - the 2D space is simply too crowded to represent layers of abstractions in flattened way. The hyperbolic plane would help here [2]. Not every bit of whole design can be shown at once (this would result in information overload), but editor could show what is in focus as well as one or two levels of context. As to success of existing visual languages, I'd argue that they are all targeted at non-technical people and they all run within sandbox environment developed with traditional text code. Why aren't their creators dog-fooding? My own answer to this is that existing visual languages cannot support the necessary amount of complexity. [1] https://git.sr.ht/~technomancy/anarres https://git.sr.ht/~technomancy/anarres [2] https://www.youtube.com/watch?v=8bhq08BQLDs https://www.youtube.com/watch?v=8bhq08BQLDs
- CyberDildonics 6y ago
  > Visual programming is very spatial. I think this is worth stating, but I think it really means that expressions take up much more space. This can be confronted from two directions. First, don't use nodes and noodles for expressions, because it doesn't help clarity and hurts density. Use nodes at a higher level instead of searching for a silver bullet. Second, of course one flat work space does not scale. Node based tools already have ways of grouping nodes as well as making groups that can be instanced instead of duplicated. > connected blocks seem to me like a dead end. Connected blocks of any sort mirror how programs work in the first place. Automatic organization is typically pretty helpful though. > Another problem is structuring the code - the 2D space is simply too crowded to represent layers of abstractions in flattened way. The hyperbolic plane would help here [2]. I'm not sure what this means, but I'm guessing it is also solved by grouping nodes and making components by groups that can be instanced. > As to success of existing visual languages, I'd argue that they are all targeted at non-technical people and they all run within sandbox environment developed with traditional text code. Why aren't their creators dog-fooding? My own answer to this is that existing visual languages cannot support the necessary amount of complexity. This is definitely not true. There are many extremely technical people using visual languages because they are huge speed ups in productivity - more interactive and less error prone by a huge degree. Shader writers use nodes instead of text all the time. Lots of tools transitioned from being text based to being graphs. The difference is that they are domain specific and not general purpose programming languages. It is the general purpose nut that hasn't been cracked.
CyberDildonics 6y ago
Visually programming shaders and composites works extremely well, though it is important to realize what they don't do. Anything that needs to be sequential is not done well with nodes as well as branching and looping. Also programs like nuke, houdini etc. are basically limited to one data type (images) or a few data types (images, geometry, 1D channels). Shaders are limited to their primitive data types as well. The advantages are the ability to see output at every stage, work in real time (including seeing errors in real time), etc. You also get to see descriptions, limitations and special interfaces for each parameter. Not only that, but parameters can be switched between constants, expressions and arrays (channels) easily for debugging. Overall there is a lot more information going on. I don't think source control has anything to do with it. Anyone can save more versions of a file and many are text. A lot of times version control is used with text fragments that make up reusable groups of nodes.
mcphage 6y ago
> Programmers seem to talk about visual programming languages as if they’ve failed, but outside of software engineering they appear to me to be more popular than text-based languages. There's a large category of conventional wisdom on HN that's categorically false yet constantly repeated—the idea that "visual programming languages failed and are now dead" is one of those. It's... irritating :-)

munk-a 6y ago

I think version control is a decent reason, but it's more the full ecosystem. As an example, how can I grep a set of visual functions for some specific pattern - what even is a meaningful pattern to look for. That lack of an ecosystem makes most devs nervous and we tend to be (rightfully) distrustful of all-in-one ecosystems since your program (probably your livelihood) is then tied to the continued support of that ecosystem. It's one reason this movement is interesting to me, if we can establish a common communication medium, specifically a non-proprietary one, then tool builders can come along and start building out that ecosystem - likely as inefficient hacks initially but ones that can be refined as the platform gets more focus.

mypalmike 6y ago

I don't think version control has anything to do with it. If there were a highly compelling and productive general purpose visual language, storing the data as text, and even providing tools to aid in things like merge conflicts, would be minor details. But when you actually dig into things like (the excellent) Prograph and its descendents, you find out that shapes and lines are actually less intuitive than text for nontrivial code bases. Contrary to what one might expect, a picture is not worth a thousand words. Perhaps a new visual paradigm will be introduced that changes things, but there's really nothing today that competes with even your least favorite popular text based language.

robenkleene 6y ago
Why wouldn't you say the examples I gave above don't compete with "your least favorite popular text based language"? My whole point is that not only do they compete, but they are actually winning in most use cases outside of software engineering. Once you take that assumption (which I'd love to hear someone debate), the question just becomes why do plain text programming languages win for some uses cases and not others? The discussion of "general purpose" programming languages doesn't really make sense with respect to visual programming languages because visual programming languages necessarily exist within a GUI, and GUIs themselves don't scale to general purpose. So what we actually have is specialized visual programming languages that double as (or exist within) specialized GUI applications. A system that appears to be working quite well in creating much more efficient systems for non-programmers to accomplish programmatic tasks. This kind of poor mapping of textual programming languages to certain problems is why the Stripe globe is state of the art text-based rendering (https://stripe.com/blog/globe https://stripe.com/blog/globe), when people are doing algorithmic architecture in Houdini (https://mvsm.com/project/imac-pro https://mvsm.com/project/imac-pro). It's not even comparable, text-based languages are getting absolutely decimated outside of software engineering by better tools that leverage visual programming languages.
- creata 6y ago
  Your link to "algorithmic architecture" was such a letdown. These guys weren't doing architecture, they were just making a flashy Apple ad! Can you expand on what you mean when you call the Stripe globe and the Apple ad "state of the art"? I assume you don't mean that these are some of the most technically proficient graphics you've seen, because that would just be silly. And comparing something developed by a design studio with god-knows-how-much rendering time to something designed in-house that runs live in your browser seems a little strange.
  - robenkleene 6y ago
    Yes, I would consider the examples "some of the most technically proficient graphics", but you can find practically infinite examples of impressive procedural work done with visual-programming-language-based tools. I'd love to find more impressive examples using text-based programming languages though, those are harder to find. I agree that comparing a video to something that runs live obviously isn't fair, but that's not really the axis I'm concerned about here. Plain-text code seems to struggle at the artistic level vs. visual tools, not the performance level. E.g., the counterargument I'm looking for would be impressive visuals created purely in code that's either pre-render (e.g., as a video), or running live, I don't really care, I just want to see visually impressive work.

Animats 6y ago

There are version control systems that aren't text oriented. There's Autodesk Vault, for drawings and 3D designs. There's Alienbrain, which is for games, movies, and even automotive designs, where assets can include a game level, a movie clip, a 3D model, a motion capture session, and a fender. "Previews are available for several hundred file formats". There's even the new NVidia Omniverse, which is even fancier.

jmiskovic 6y ago

I have another theory why it doesn't scale. Visual programming is very spatial. This may seem helpful at first but actually it gets in the way of development process. Developing isn't just writing (drawing) new code. It is iterative process of juggling many pieces of functionality so they work well together. When you have to refactor some logic, suddenly blocks move to different places, and their relative positions change. Whenever related blocks change relative orientation, your internal model for navigation becomes obsolete. Human brain is not used to this and it creates discomfort and frustration. Traditional programming done with text files in which functions have 1D organization. Because 1D has less spatial relations (just above and below), code reorganization is cognitively easier. It's easier to find good place for piece of code and it's easier to remember where it lives. I think a truly usable visual programming tool should not allow users to place blocks at arbitrary positions, but should automatically (and predictably) arrange pieces on screen. Structural text editors (for example [1]) are step in right direction, while connected blocks seem to me like a dead end. Another problem is structuring the code - the 2D space is simply too crowded to represent layers of abstractions in flattened way. The hyperbolic plane would help here [2]. Not every bit of whole design can be shown at once (this would result in information overload), but editor could show what is in focus as well as one or two levels of context. As to success of existing visual languages, I'd argue that they are all targeted at non-technical people and they all run within sandbox environment developed with traditional text code. Why aren't their creators dog-fooding? My own answer to this is that existing visual languages cannot support the necessary amount of complexity. [1] https://git.sr.ht/~technomancy/anarres https://git.sr.ht/~technomancy/anarres [2] https://www.youtube.com/watch?v=8bhq08BQLDs https://www.youtube.com/watch?v=8bhq08BQLDs

CyberDildonics 6y ago
> Visual programming is very spatial. I think this is worth stating, but I think it really means that expressions take up much more space. This can be confronted from two directions. First, don't use nodes and noodles for expressions, because it doesn't help clarity and hurts density. Use nodes at a higher level instead of searching for a silver bullet. Second, of course one flat work space does not scale. Node based tools already have ways of grouping nodes as well as making groups that can be instanced instead of duplicated. > connected blocks seem to me like a dead end. Connected blocks of any sort mirror how programs work in the first place. Automatic organization is typically pretty helpful though. > Another problem is structuring the code - the 2D space is simply too crowded to represent layers of abstractions in flattened way. The hyperbolic plane would help here [2]. I'm not sure what this means, but I'm guessing it is also solved by grouping nodes and making components by groups that can be instanced. > As to success of existing visual languages, I'd argue that they are all targeted at non-technical people and they all run within sandbox environment developed with traditional text code. Why aren't their creators dog-fooding? My own answer to this is that existing visual languages cannot support the necessary amount of complexity. This is definitely not true. There are many extremely technical people using visual languages because they are huge speed ups in productivity - more interactive and less error prone by a huge degree. Shader writers use nodes instead of text all the time. Lots of tools transitioned from being text based to being graphs. The difference is that they are domain specific and not general purpose programming languages. It is the general purpose nut that hasn't been cracked.

CyberDildonics 6y ago

Visually programming shaders and composites works extremely well, though it is important to realize what they don't do. Anything that needs to be sequential is not done well with nodes as well as branching and looping. Also programs like nuke, houdini etc. are basically limited to one data type (images) or a few data types (images, geometry, 1D channels). Shaders are limited to their primitive data types as well. The advantages are the ability to see output at every stage, work in real time (including seeing errors in real time), etc. You also get to see descriptions, limitations and special interfaces for each parameter. Not only that, but parameters can be switched between constants, expressions and arrays (channels) easily for debugging. Overall there is a lot more information going on. I don't think source control has anything to do with it. Anyone can save more versions of a file and many are text. A lot of times version control is used with text fragments that make up reusable groups of nodes.

mcphage 6y ago

> Programmers seem to talk about visual programming languages as if they’ve failed, but outside of software engineering they appear to me to be more popular than text-based languages. There's a large category of conventional wisdom on HN that's categorically false yet constantly repeated—the idea that "visual programming languages failed and are now dead" is one of those. It's... irritating :-)