15 ms·
Grab bag of miscellaneous thoughts on this topic: - Visual computing is pretty heavily used, unreal engine, blender, houdini, etc. All have a very similar node
by gpm 8y ago
Grab bag of miscellaneous thoughts on this topic:
- Visual computing is pretty heavily used, unreal engine, blender, houdini, etc. All have a very similar node based visual programming system. It seems to work pretty well (better than text) for most of what they use it for. I think because of the ease of jumping in the middle and making small understandable changes.
- Many programming languages today have a format that is like <tree of files, each containing> <set of items, each containing> <list of expressions>. It would be nice if that <set of items> step was treated as an unordered set instead of an ordered one, with editors having a better understanding of how to bring up relevant elements of the set onto your screen at the same time. Split pane editors, "peeking" in editors at code definitions, etc. hint at how this should work, but I don't feel like they do it as well as possible.
- A small amount of "visual augmentation" might benefit most programming languages, I'm not sure I can explain this better than linking to a few images of what emacs auctex package does to latex http://lh6.ggpht.com/_egN-3IJO0Xg/SpIj6AtHOTI/AAAAAAAABj8/O1aKgTb-Njw/preview-latex.png http://lh6.ggpht.com/_egN-3IJO0Xg/SpIj6AtHOTI/AAAAAAAABj8/O1... https://upload.wikimedia.org/wikipedia/commons/4/42/Emacs%2BAucTeX.png https://upload.wikimedia.org/wikipedia/commons/4/42/Emacs%2B...
- agumonkey 8y agodataflow / lazydag has so many advantages when it fits the domain. for non linear, you'd end up with full on circuit theory which requires an abstraction jump (but in a way, that's still visual ~programming)
- _yawn 8y agoI feel like a lot of users on HN are producers rather than users of software, and haven't used node-based systems like in blender or nuke. They are extremely productive, and end up being very similar to functional programming while being super easy to pick up. It's a great "inbetween" representation for non-programmers that really need to do domain-specific programming.
- mattnewport 8y agoExperience with tools like Blender or Nuke and particularly with visual programming in games engines is actually where a lot of the better informed dislike of visual programming comes from. The biggest problem with these tools is scalability and maintainability. You will hear many stories in the games and VFX industries of nightmare literal spaghetti code created with visual programming tools that is impossible to debug, refactor or optimize. Visual programming seems easy for very small examples but it doesn't scale. It has no effective means of versioning, diffing or merging and usually lacks good mechanisms for abstraction, reuse and hierarchical structure. It doesn't have tooling for refactoring and typically lacks tooling for performance profiling. Some of these problems seem to be more fundamental and others like they could potentially be addressed with better tooling but that tooling never seems to emerge. I've got a lot of experience with shader programming and have never found node based shader editors to be better than text over the long term, although there are some nice visual debugging features which are rarely implemented in text based IDEs (though I have seen it done). I've also found visual scripting to all too frequently get out of hand and have to be replaced with code due to being unmaintainable, undebuggable or unoptimizable. I think there is possibly fertile terrain to explore in trying to get some of the benefits of visual programming approaches while avoiding all these downsides but many of us have been burned enough to be very skeptical of the majority of visual programming systems that don't even try to fix the worst problems.
- candiodari 8y agoYou shouldn't generalize like this. Even in massive projects visual programming can be enormously helpful. Nobody seriously tries to script game levels in an IDE. Nobody tries to match sprite parameters, which eventually become code, to image files in an IDE. Nobody tries to design GUIs in IDEs. The right tool for the right job. For 50% of a game's code visual programming is absolutely the right tool. For some other parts it probably isn't.
- mattnewport 8y agoYou seem to be using a different definition of visual programming than the normal one. The article and my comment are talking about visual programming as a graphical representation of code / logic rather than a textual one. Examples would be things like Unreal's Blueprint visual scripting, node based shader editors (Unreal has one of those too) or the Petri nets described in the article. Visual / GUI tools like sprite editors, level editors or GUI builders are not what is usually meant by visual programming. Unity is a very popular game engine that doesn't have an official visual programming solution (they're previewing one in the very latest version). Unity has a powerful level editor that is used to lay out the levels in a GUI tool but no visual scripting / programming tools. The majority of Unity games that currently exist therefore do all level scripting in C# code. Many other games engines have no visual scripting solution and all level scripting is done in either a scripting language like Lua or in some cases in C++ code. Unity has sprite editors, visual GUI builder tools etc. but those are not what is generally meant by "visual programming". The closest Unity has had until recently was its graphical animation state machine editor.
- wool_gather 8y agoI did some programming in Max/MSP way back when, which was fun, and you have a point about the "functional" aspect. It was sort of the opposite of say, C, in that it was actually hard to create side effects even when you needed them. But overall I agree with other people that I wouldn't want to maintain anything particularly large in that format. The Max "patches" were on the order of big scripts at most.
- wires 8y ago:) I also did a lot of Max/MSP and many similar systems, [nord modular](http://nmedit.sourceforge.net/ http://nmedit.sourceforge.net/) , and also wrote such tools back then, for audio and 3d and video, which is what led to statebox ultimately. I would say the spaghetti aspect of max is the main complaint with visual programming. To modularise it (small diagrams), you need to contain the behaviour of the boxes (ie. typed purely functional code). And to generalise it (audio, video, microservices, ...) you need to separate the syntax from the semantics. (took me about 15yrs to figure out a way to do this properly :-)
- pfranz 8y agoI don't know if any of them are extremely easy to pick up. Any re-use between applications are the domain-specific parts (knowing about color space, UVs, shaders, and cameras) not the UI. Node based interfaces are pretty limiting. A lot of the power of these systems are things like having a Python runtime or text representations of scene files that you can find/replace. The other pro/power features are shortcut keys and templates, just like other apps like Lightroom or Avid. Node based systems is a great pattern for certain things, but large scenes get unwieldy quick. Even if the project files are text, diffing with version control is useless. Profiling is often difficult--a pet peeve of mine when apps load all the whole scene file and often the geometry when you just want to change a parameter. You can see node based systems "grow up" by adding variables or attribute references, which means your data isn't just flowing down the graph, but you have to track references in this new dimension. You then often see encapsulation (hda/otl, Gizmos), which can really help with re-use but create more limitations.
- p0nce 8y agoThe problems with visual programming have never changed: at one point you need functions, macros, identifiers, user-defined types, error paths, diffs... (EDIT: it's an idea that doesn't work at all for _general purpose_ programming, but for some reason it sounds like a good idea from a distance).
- pacala 8y agoOrdered <set of items> have the advantage of working well with the brain's ability to map its surroundings. "function foo is defined 5 arrow downs above function bar". When the <set of items> jump around our eyes in arbitrary order, we tend to experience disorientation.
- pintxo 8y agoSame here. OTOH I always wonder if it would not be beneficial if our brain and the code would not rely on spacial distances in code files, but on _call_ distances. Should our spacial map of the code not be the call graph instead of the structure in files and disk?
- klhugo 8y agoThe last bullet you mentioned is the reason I use Jupyter (even though I'm a HUGE fan of vim).
- finder83 8y agoI recently learned Smalltalk (Pharo), and since it's image based it treats classes/functions as more of a collection than trees of files. Of course, there's still structure, but the way you can navigate from function to function is great. Smalltalk of course has its own problems. It suffers from one of the major problems of many less popular languages, namely the lack of libraries. But I agree that I wish more (or all) languages were as good at debugging, peeking at definitions, jumping to different things, searching code, etc. Of course editors can make up some of the problems, but it's not quite the same.
- jayd16 8y agoI think the take away from existing visual languages is that they can be quite nice but the less they mimic imperative blocks with drag and drop the better. Functional programming is a better fit. They work well for stream processing. They can often be more elegant in how they define and consume inputs. Many more parameters can be supported in a visually pleasing way, which cleans up parameter over loading and makes it easier to compose function block together without making Tuple types. This could very well exist but I think a visual Lisp would be interesting.
- wires 8y agoyep, the diagrams are like typed purely functional programs also stream processing is possible, we can (at least in theory) use the same diagram to compose state machines or stream processing functions or DB queries or ...
- lispm 8y agoThere are visual Lisps. They are for example used in music composition: http://www2.siba.fi/PWGL/pwgl.html http://www2.siba.fi/PWGL/pwgl.html http://repmus.ircam.fr/openmusic/overview http://repmus.ircam.fr/openmusic/overview
- ierfgoehrgwar 8y agoVisual programming of data-flow does not add much to programming, I think that's the lesson of history. That said, I think that visual programming which is not aimed to data-flow can be very interesting. For example Lisp is pretty visual (or topological) in a sense.
- stcredzero 8y agoMany programming languages today have a format that is like <tree of files, each containing> <set of items, each containing> <list of expressions>. It would be nice if that <set of items> step was treated as an unordered set instead of an ordered one, with editors having a better understanding of how to bring up relevant elements of the set onto your screen at the same time. Split pane editors, "peeking" in editors at code definitions, etc. hint at how this should work, but I don't feel like they do it as well as possible. Back in the day, in VisualWorks, with the RefactoringBrowser, you could bring up a browser for a search, say, everyone who implemented methodNamedFoo: Then you could rapidly narrow that down (effectively and-ing it) to those methods which also sent thisOtherMethod: These were lightning quick operations, just right-click and done! But what's more, you could even compose little queries (think like you would in a SQL query client, but using snippets of Smalltalk code) that would be applied to/and-ed with the contents of each browser window. Done right, this resulted in a few accurate, complete, and highly focused sets of code applied to the exact problem you were working on. What's more, you could write scripts to pop up such query browsers automatically. They would also be saved in the "image" and just pop up to the same state when you restarted the environment. On top of that, you could write syntactically accurate code transformations against all of the above, even writing ad-hoc code against the meta level or even runtime state from the middle of a runtime debug session. A small amount of "visual augmentation" might benefit most programming languages Agreed. Where various visual programming have fallen down over the past 3 decades: 1) Scaling complexity -- If diagrams get too busy, and there's no good way of managing complexity. This especially applies to multiple programmers changing the same diagram. 2) Scaling size/optimization -- Many visual programming systems in past decades could bog down and become marginally responsive or unusual when managing large systems. If you can handle those two, you will have a huge leg up towards a viable visual programming augmentation.
- _trampeltier 8y agoVisual programming is also used a lot in plc programming (Siemens S7/TIA, Mitsubishi MELSEC, etc..). Programming visually is usually called FBD or LAD. We use often FBD for simple logic. It's easy to read, even for inexperienced maintenance guys. LAD is a no go for me, but it seems a lot of guys still like it. http://szirty.uw.hu/lang/Siemens_TIA_FBD.png http://szirty.uw.hu/lang/Siemens_TIA_FBD.png As long there is a simple and clear code structure, it is a good thing. Today, special since Siemens made SCL (kind/like of Pascal for PLCs) usable and in there new IDE (old was Step7, new is TIA), we use it also a lot. Today it's even possible to mix FBD/LAD and SCL. So you can make al simple logic in FBD or LAD, and then calculate things in between in a SCL network. https://planken.org/images/screendumps/201408-tiav13-network-offline.png https://planken.org/images/screendumps/201408-tiav13-network... (the only (not so great) ex. I found) As user of both worlds in my job, I can say both worlds have a right to exist. It's just like is C better than C++ or is Python even the most best ... is a car better than a bike .. is a house better than a tent ..
- wires 8y agothere is some pretty solid theory on how to translate between the kind of diagrams statebox uses and digital circuits. In fact we are doing some experiments with direct diagram to wafer (chip) translation using LibreSilicon http://libresilicon.com/ http://libresilicon.com/ this is certainly not a done thing, but something we'd love to work more on in the future
- _trampeltier 8y agoIt was possible since Simens Step 5 (since late 70s) to change your view between FBD/LAD and AWL. But AWL is just like Assembler, so .. yes. In old machines (we still have about 20 machines with Siemens S5) sometimes it's a must to use AWL. https://en.wikipedia.org/wiki/Simatic_S5_PLC https://en.wikipedia.org/wiki/Simatic_S5_PLC Translate an IF statement you need to add jumps, so it's not really the same code. Also for ex. the very simple Siemens LOGO controller has a simple software to program it. But if you use FBD, it's just possible to use a tag once. So if you need a tag multiple times, so you have to draw lines from the single tag. Even for super simple stuff it get's messy super quick. https://w3.siemens.com/mcms/programmable-logic-controller/de/logikmodul-logo/logo-software/seiten/default.aspx https://w3.siemens.com/mcms/programmable-logic-controller/de...
- brobdingnagians 8y agoDepending on the use, I think it can be fantastic. Jetbrains makes an editor called MPS that is used to make DSLs and you can include things like tables and diagrams into the code. In places where you have very specific requirements and structure, it can help experts in the domain produce the logic for it. That's the same with level editing, level editors need good creativity and views of spaces not experts in C++, so a DSL and graphical editing is great, because it lets them focus on something else. That said, when it comes to the code behind things, text gives you an immense amount of expressibility that can't be replicated very well with graphical things. It's the same reason why a lot of developers prefer the command line to graphical configuration; you get far more expression for your expertise in a text environment. You get every combination of letters/symbols on the keyboard entered through a large physical interface; using the mouse to click on things feels slow by comparison.
- Nigredo 8y agoWhat you say is true. Text is way more expressive, but sometimes it is difficult to spot the overall code structure just by using text. What Statebox does is this: It implements Petri nets as categories and maps them functorially to a "normal" functional programming semantics. What this means is that: - You draw the structure of your code in a behavior-oriented way using petri nets. This step is completely visual. - You use a ton of formal tools to verify that this net has nice properties. - You map places and transitions of the net to datatypes and functions, respectively. You see that in this approach nets serve the purpose of giving an high-level understanding of how the code behaves. You still have the freedom that you get by using text in filling the net with meaning, but you gain also this high-level overview that saves a ton of work!
- Firadeoclus 8y agoIt is certainly true that keyboard input can be much quicker when what you're trying to convey to the machine is logical rather than spatial/kinetic information. But it is entirely possible to use keyboard input to manipulate a graph instead of an array of characters.
- k__ 8y agoVisual programming works best with functional programming because everything is neatly ordered. Most programmers don't like FP, so they probably don't tend to such tooling.
- lenocinor 8y agoI think saying that most programmers don't like FP is a bit more extreme than the actual case here, which is that I think most programmers aren't familiar with FP (although this seems to be growing rapidly due to introduction in Java and other places). That said, I think many of them seem to find it harder, especially in the beginning. But I think we have to conquer familiarity first.
- taco_emoji 8y agoRegarding visual augmentation, the Visual Studio XAML designer is another good example. You can edit the XAML markup and see the generated UI in another pane. https://www.michaelcrump.net/files/SNAGHTMLb3b1ee0_634588657951361750.png https://www.michaelcrump.net/files/SNAGHTMLb3b1ee0_634588657...
- gary__ 8y agoReminds me of this editor project from Kent Beck for working with the AST instead of with text alone when developing, somewhat related https://www.facebook.com/notes/kent-beck/prune-a-code-editor-that-is-not-a-text-editor/1012061842160013/ https://www.facebook.com/notes/kent-beck/prune-a-code-editor...
- ekzy 8y agoI do a bit of Clojure and it feels like manipulating an AST rather than text, especially with structural editing (because code is data). It applies to any Lisp I suppose
- aasasd 8y ago> It would be nice if that <set of items> step was treated as an unordered set instead of an ordered one, with editors having a better understanding of how to bring up relevant elements of the set onto your screen at the same time. IDEs sorta move in that direction, though not too fast. You can list methods in a class, jump to the definition of a method, etc. Not many seem to realize that Java's strict OOP structure is (or was) a mover of IDE functionality: you can statically describe the structure of the entire program at a high level, and code is only contained in methods, so you have methods as organizational units, to which you navigate and otherwise reason. So now we have IDE functionality that can move methods and variables around like they're toy blocks.