6 ms·

I discovered TouchDesigner [1] a few months back- it’s an incredibly powerful visual programming tool used in the entertainment industry. It’s been around well

I discovered TouchDesigner [1] a few months back- it’s an incredibly powerful visual programming tool used in the entertainment industry. It’s been around well for over a decade and is stable enough to be used for live shows. Deadmau5 uses it to control parts of his Cube rig [2]. I’ve seen a few art installations based around it as well [3]. There are some really amazing tutorials and examples here: https://youtu.be/wubew8E4rZg https://youtu.be/wubew8E4rZg [1] https://derivative.ca https://derivative.ca [2] https://derivative.ca/community-post/made-love-touchdesigner-v99-cusersdeadmau5 https://derivative.ca/community-post/made-love-touchdesigner... [3] https://derivative.ca/community-post/making-go-robots-interactive-motion-tracked-dance-installation https://derivative.ca/community-post/making-go-robots-intera...

bmitc 6y ago
TouchDesigner is indeed super cool. And I correctly guessed what that tutorial was going to be before I clicked on it. :) Mathew Ragan is an excellent tutorial maker. He's also relaxing to listen to. TouchDesigner really showcases the enabling nature of visual programming languages. You can see your program working and inspect and modify it while it is working. These are very powerful ideas, and visual programming languages are much better platforms for ideas like this. People, i.e. traditional programmers, are really hard and down on visual programming languages. Meanwhile, people who use LabVIEW, TouchDesigner, vvvv, Pure Data, Max, and Grasshopper for Rhino are all extremely effective and move quickly. People who are experts in these environments cannot be kept up with people using text-based environments when doing the same application. Text-based programming is limited in dimensionality. This can become very constraining.
- mjfisher 6y ago
  > Text-based programming is limited in dimensionality. This can become very constraining In contrast, I've often thought that visual programming tools are much more limited dimensionally, and that's why they can become difficult to manage beyond a low level of complexity: you only have two dimensions to work with. With the visual programming tools, the connections between components need very careful management to prevent them becoming a tangled and overlapping web. In a 2D tool (e.g. LabVIEW), you could make a lot of layouts simpler by introducing a third dimension and lifting some components higher or lower to detangle connections - but then you'd face similar hard restrictions in 3D. Text based programs suffer from no such restrictions; the conceptual space your abstractions can make use of is effectively unlimited, and you can manage connections and information flow between pieces of code to maximize readability and simplicity, rather than artificially minimizing the number of dimensions.
  - bmitc 6y ago
    > the conceptual space your abstractions can make use of is effectively unlimited, and you can manage connections and information flow between pieces of code to maximize readability and simplicity, rather than artificially minimizing the number of dimensions. How does this not apply to a visual language like LabVIEW? Just because you draw the code on a 2D surface doesn't prevent abstraction and arbitrary programs. The way I program LabVIEW and the way it executes is actually very similar to Elixir/Erlang and OTP. Asynchronous execution and dataflow are core to visual languages. You are not "bound" by wires. When you write text-based code, you are also restricted to 2 dimensions, but it's really more like 1.5 because there is a heavy directionality bias that's like a waterfall, down and across. I cannot copy pictures or diagrams into a text document. I cannot draw arrows between comments to the relevant code; I have to embed the comment within the code because of this dimensionality/directionality constraint. I cannot "touch" a variable (wire) while the program is running to inspect its value. In LabVIEW, not only do I have a 2D surface for drawing my program, I also get another 2D surface to create user interfaces for any function if I need. In text-languages, you only have colors and syntax to distinguish datatypes. In LabVIEW, you also have shape. These are all additional dimensions of information.
    - sinker 6y ago
      > Text-based programming is limited in dimensionality. This can become very constraining. > When you write text-based code, you are also restricted to 2 dimensions, but it's really more like 1.5 because there is a heavy directionality bias that's like a waterfall, down and across. These are two really good points. Text-based code is just a constrained version of a visual programming environment. So far most attempts at VP have attempted to represent code in terms of nodes and lines (trees), but that does not necessarily need to be the case. The interesting about VP is that is presents a way to better map the concept and structure of programming as an abstract concept to how we physically interface with our coding environments. It's far from likely that character and line-based editing is the mode of the future. Line-editing maps to the reality of programming in, to my eyes, such a limited way that is seems the potential for new interfaces, and modes of representation is wide open. It's not that, as some people stubbornly say, there's no better alternative to text-based programming, but I think we just haven't conceived of a better way yet. We're biased to think in a certain way because most of us have programmed in a certain way almost solely by text and most of our tools are built to work with text. But that doesn't necessarily mean that the way we've done things is the best way indefinitely. There's so much unexplored territory. VR opens up new frontiers. What if the concepts of files, lines, workspaces were to map to something else more elemental to programming as an abstract concept. What if we didn't think so much in terms of spacial and physical delineations, and instead something else? Blind programmers have a different idea of what an editor is. Spreadsheet programs "think" in terms of cells in relation to one another. What about different forms of input? Dark Souls can be beaten on a pair of bongos. Smash Bros players mod their controllers because their default mode of input isn't good enough at a high level. Aural and haptic interfaces are unexplored. Guitars and pianos are different "interfaces" to music. Sheet music is not a pure representation of music. I think there's the mistaken belief that text == code, that text is the most essential form of code. Lines, characters are not the essential form of code. As soon as we've assigned something a variable name, we've already altered our code into a form to assist our cognition. Same with newlines, comments, filenames, the names of statements and keywords. When we program in terms of text, we're already transforming our interpretation of code and programming; we've already chosen and contrained ourselves to a particular palette. What is the most essential form of code are (depends on the language, but generally) data structures and data flow. So far, our best interpretation of this is in the form of text, lines, characters, inputted by keyboard onto a flat screen - but this is still just one category of interpretation. All this is to say is that text is not necessarily the one and only way, and it's too soon to say that it's the best way.

bmitc 6y ago

TouchDesigner is indeed super cool. And I correctly guessed what that tutorial was going to be before I clicked on it. :) Mathew Ragan is an excellent tutorial maker. He's also relaxing to listen to. TouchDesigner really showcases the enabling nature of visual programming languages. You can see your program working and inspect and modify it while it is working. These are very powerful ideas, and visual programming languages are much better platforms for ideas like this. People, i.e. traditional programmers, are really hard and down on visual programming languages. Meanwhile, people who use LabVIEW, TouchDesigner, vvvv, Pure Data, Max, and Grasshopper for Rhino are all extremely effective and move quickly. People who are experts in these environments cannot be kept up with people using text-based environments when doing the same application. Text-based programming is limited in dimensionality. This can become very constraining.

mjfisher 6y ago
> Text-based programming is limited in dimensionality. This can become very constraining In contrast, I've often thought that visual programming tools are much more limited dimensionally, and that's why they can become difficult to manage beyond a low level of complexity: you only have two dimensions to work with. With the visual programming tools, the connections between components need very careful management to prevent them becoming a tangled and overlapping web. In a 2D tool (e.g. LabVIEW), you could make a lot of layouts simpler by introducing a third dimension and lifting some components higher or lower to detangle connections - but then you'd face similar hard restrictions in 3D. Text based programs suffer from no such restrictions; the conceptual space your abstractions can make use of is effectively unlimited, and you can manage connections and information flow between pieces of code to maximize readability and simplicity, rather than artificially minimizing the number of dimensions.
- bmitc 6y ago
  > the conceptual space your abstractions can make use of is effectively unlimited, and you can manage connections and information flow between pieces of code to maximize readability and simplicity, rather than artificially minimizing the number of dimensions. How does this not apply to a visual language like LabVIEW? Just because you draw the code on a 2D surface doesn't prevent abstraction and arbitrary programs. The way I program LabVIEW and the way it executes is actually very similar to Elixir/Erlang and OTP. Asynchronous execution and dataflow are core to visual languages. You are not "bound" by wires. When you write text-based code, you are also restricted to 2 dimensions, but it's really more like 1.5 because there is a heavy directionality bias that's like a waterfall, down and across. I cannot copy pictures or diagrams into a text document. I cannot draw arrows between comments to the relevant code; I have to embed the comment within the code because of this dimensionality/directionality constraint. I cannot "touch" a variable (wire) while the program is running to inspect its value. In LabVIEW, not only do I have a 2D surface for drawing my program, I also get another 2D surface to create user interfaces for any function if I need. In text-languages, you only have colors and syntax to distinguish datatypes. In LabVIEW, you also have shape. These are all additional dimensions of information.
  - sinker 6y ago
    > Text-based programming is limited in dimensionality. This can become very constraining. > When you write text-based code, you are also restricted to 2 dimensions, but it's really more like 1.5 because there is a heavy directionality bias that's like a waterfall, down and across. These are two really good points. Text-based code is just a constrained version of a visual programming environment. So far most attempts at VP have attempted to represent code in terms of nodes and lines (trees), but that does not necessarily need to be the case. The interesting about VP is that is presents a way to better map the concept and structure of programming as an abstract concept to how we physically interface with our coding environments. It's far from likely that character and line-based editing is the mode of the future. Line-editing maps to the reality of programming in, to my eyes, such a limited way that is seems the potential for new interfaces, and modes of representation is wide open. It's not that, as some people stubbornly say, there's no better alternative to text-based programming, but I think we just haven't conceived of a better way yet. We're biased to think in a certain way because most of us have programmed in a certain way almost solely by text and most of our tools are built to work with text. But that doesn't necessarily mean that the way we've done things is the best way indefinitely. There's so much unexplored territory. VR opens up new frontiers. What if the concepts of files, lines, workspaces were to map to something else more elemental to programming as an abstract concept. What if we didn't think so much in terms of spacial and physical delineations, and instead something else? Blind programmers have a different idea of what an editor is. Spreadsheet programs "think" in terms of cells in relation to one another. What about different forms of input? Dark Souls can be beaten on a pair of bongos. Smash Bros players mod their controllers because their default mode of input isn't good enough at a high level. Aural and haptic interfaces are unexplored. Guitars and pianos are different "interfaces" to music. Sheet music is not a pure representation of music. I think there's the mistaken belief that text == code, that text is the most essential form of code. Lines, characters are not the essential form of code. As soon as we've assigned something a variable name, we've already altered our code into a form to assist our cognition. Same with newlines, comments, filenames, the names of statements and keywords. When we program in terms of text, we're already transforming our interpretation of code and programming; we've already chosen and contrained ourselves to a particular palette. What is the most essential form of code are (depends on the language, but generally) data structures and data flow. So far, our best interpretation of this is in the form of text, lines, characters, inputted by keyboard onto a flat screen - but this is still just one category of interpretation. All this is to say is that text is not necessarily the one and only way, and it's too soon to say that it's the best way.