5 ms·

It's interesting that a lot of sound programming environments are primarily visual -- I'm thinking pd, Max/MSP, and Reaktor. Probably because many audio synthes

It's interesting that a lot of sound programming environments are primarily visual -- I'm thinking pd, Max/MSP, and Reaktor. Probably because many audio synthesis concepts trace back to when people literally had to wire hardware modules together with patch cords. Anyway, I don't think the problem is that the idea of visual programming hasn't been around. It's all over the place, if you look. And has been for ages. The problem is likely that it doesn't solve a problem that developers who use text-based programming languages have. My main concern would simply be: What do I lose by going visual? With text-based code I can read every single character and make sure it's all precisely how I want it. Would that be easier or harder in a visual environment? Or: What if I need to do something not explicitly planned for in the visual environment? Am I stuck cracking open the visual modules, anyway, and writing in hacky Javascript or something that then becomes difficult to see because it's stuffed awkwardly inside of some visual element? (This happens fairly often in the above-mentioned sound programming environments.) If the gains can't definitively offset those sorts of costs, then I'm not going to be very interested.

jcelerier 8y ago
> It's interesting that a lot of sound programming environments are primarily visual -- I'm thinking pd, Max/MSP, and Reaktor. Probably because many audio synthesis concepts trace back to when people literally had to wire hardware modules together with patch cords. Visual sound environments are also tools that are taught to music students in conservatories, who don't have any programming ability. These are tools to make sound / art, and people who make sound / art aren't necessarily programmers. > With text-based code I can read every single character and make sure it's all precisely how I want it. and when you make art you aren't necessarily looking for the precise. Some people just throw random objects on their patches and wire them almost randomly until it sounds good ; writing the patch is entirely part of the artistic process. That's a completely different mindset that "client wants feature X in Y time, what's the most easy way for me to achieve it".
- chasing 8y ago
  > and when you make art you aren't necessarily looking for the precise. Some people just throw random objects on their patches and wire them almost randomly until it sounds good ; writing the patch is entirely part of the artistic process. That's a completely different mindset that "client wants feature X in Y time, what's the most easy way for me to achieve it". You're right in that art is guided by a different set of goals than client work -- it's inherently more exploratory. But it's inaccurate (and unfair) to call it imprecise. (Also unfair to assume artists work "randomly.") If you're an artist who cares about your work, you put a tremendous amount of effort into achieving your vision just as you see it. If a tool fails to work as you need it to, you'll abandon the tool, whether it's a paintbrush, a chisel, or Max/MSP. The fact that two major commercial sound programming environments are visual doesn't necessarily mean people who use them don't understand computers and are just randomly throwing crap at the wall: It means they work best for the professionals who use them to get their creative work done. They are, after all, relatively expensive pieces of software. (It's also inaccurate to assume artists and musicians don't ever have programming ability. I'll point to myself as an example.)
  - pessimizer 8y ago
    I don't think that art and precision in the way that the comment you're responding to are compatible. I haven't experienced any artistic situations that haven't involved throwing random impressionistic shit at the wall, then precisely shaping, reducing, and exaggerating aspects of that random shit to make something new. Precision comes in after the randomness, in the craftsmanship; that's what differentiates a skilled person from an unskilled person; both can do the first part. When you're programming, or participating in any craft that doesn't prioritize uniqueness or expression, the precision starts from the beginning, though. The only randomness sometimes is where you start, not what you start.
    - wires 8y ago
      also, ideally the precision in the statebox kernel is hidden from the user.. nobody needs to really know about profunctors or monoidal categories, unless you want to work on the language tooling itself. anyway, interesting discussion
rhinoceraptor 8y ago
I think the problem is visual systems make it harder to create your own abstractions. You can't have all the complexity exposed in a single level, you need to be able to hide details at a lower level. For example, each module in an audio program you're wiring together is basically a function with a ton of internal state. But you also need to be able to create your own modules composed of other modules, and maybe some lower level functions, e.g. EQ, signal sources, delay/reverb, etc. The only visual system I've used is GNU Radio, which requires XML/Python/C++ to create your own blocks AFAICT.
- wires 8y ago
  gnu radio is cool but these are different problems: - macros for diagrams, n-bit-adder for any n - "boxing up" diagrams, or some sort of nesting of diagrams - management of state there are separated concepts in our tool, macros are a 'meta language', in principle one can write functions in the host language that generate diagrams and then proof that those diagrams are well behaved in some way. there is theory on how to do this for diagrams, but we have not implemented any of this yet boxing up is a natural operation of the system; we get that for free sort of state is handled by folding functions over the history, so you get a nice and clean way of dealing with state but you are right, this ability is very important; it is a lot harder to do tho, but it's possible now I think
aaaaaaaaaaab 8y ago
And as soon as you want polyphony (dynamic instantiations of a subgraph) the whole circuit metaphor goes out the window...
- wires 8y ago
  You are right about this, it is exactly what is hard to do! it is possible to do some form of this, where you have well behaved "macros" that create diagrams of particular shape "at runtime" (forall n. the diagram with n copies of n)

jcelerier 8y ago

> It's interesting that a lot of sound programming environments are primarily visual -- I'm thinking pd, Max/MSP, and Reaktor. Probably because many audio synthesis concepts trace back to when people literally had to wire hardware modules together with patch cords. Visual sound environments are also tools that are taught to music students in conservatories, who don't have any programming ability. These are tools to make sound / art, and people who make sound / art aren't necessarily programmers. > With text-based code I can read every single character and make sure it's all precisely how I want it. and when you make art you aren't necessarily looking for the precise. Some people just throw random objects on their patches and wire them almost randomly until it sounds good ; writing the patch is entirely part of the artistic process. That's a completely different mindset that "client wants feature X in Y time, what's the most easy way for me to achieve it".

chasing 8y ago
> and when you make art you aren't necessarily looking for the precise. Some people just throw random objects on their patches and wire them almost randomly until it sounds good ; writing the patch is entirely part of the artistic process. That's a completely different mindset that "client wants feature X in Y time, what's the most easy way for me to achieve it". You're right in that art is guided by a different set of goals than client work -- it's inherently more exploratory. But it's inaccurate (and unfair) to call it imprecise. (Also unfair to assume artists work "randomly.") If you're an artist who cares about your work, you put a tremendous amount of effort into achieving your vision just as you see it. If a tool fails to work as you need it to, you'll abandon the tool, whether it's a paintbrush, a chisel, or Max/MSP. The fact that two major commercial sound programming environments are visual doesn't necessarily mean people who use them don't understand computers and are just randomly throwing crap at the wall: It means they work best for the professionals who use them to get their creative work done. They are, after all, relatively expensive pieces of software. (It's also inaccurate to assume artists and musicians don't ever have programming ability. I'll point to myself as an example.)
- pessimizer 8y ago
  I don't think that art and precision in the way that the comment you're responding to are compatible. I haven't experienced any artistic situations that haven't involved throwing random impressionistic shit at the wall, then precisely shaping, reducing, and exaggerating aspects of that random shit to make something new. Precision comes in after the randomness, in the craftsmanship; that's what differentiates a skilled person from an unskilled person; both can do the first part. When you're programming, or participating in any craft that doesn't prioritize uniqueness or expression, the precision starts from the beginning, though. The only randomness sometimes is where you start, not what you start.
  - wires 8y ago
    also, ideally the precision in the statebox kernel is hidden from the user.. nobody needs to really know about profunctors or monoidal categories, unless you want to work on the language tooling itself. anyway, interesting discussion

rhinoceraptor 8y ago

I think the problem is visual systems make it harder to create your own abstractions. You can't have all the complexity exposed in a single level, you need to be able to hide details at a lower level. For example, each module in an audio program you're wiring together is basically a function with a ton of internal state. But you also need to be able to create your own modules composed of other modules, and maybe some lower level functions, e.g. EQ, signal sources, delay/reverb, etc. The only visual system I've used is GNU Radio, which requires XML/Python/C++ to create your own blocks AFAICT.

wires 8y ago
gnu radio is cool but these are different problems: - macros for diagrams, n-bit-adder for any n - "boxing up" diagrams, or some sort of nesting of diagrams - management of state there are separated concepts in our tool, macros are a 'meta language', in principle one can write functions in the host language that generate diagrams and then proof that those diagrams are well behaved in some way. there is theory on how to do this for diagrams, but we have not implemented any of this yet boxing up is a natural operation of the system; we get that for free sort of state is handled by folding functions over the history, so you get a nice and clean way of dealing with state but you are right, this ability is very important; it is a lot harder to do tho, but it's possible now I think

aaaaaaaaaaab 8y ago

And as soon as you want polyphony (dynamic instantiations of a subgraph) the whole circuit metaphor goes out the window...

wires 8y ago
You are right about this, it is exactly what is hard to do! it is possible to do some form of this, where you have well behaved "macros" that create diagrams of particular shape "at runtime" (forall n. the diagram with n copies of n)