5 ms·

SOUL: A New Efficient, Portable, Low-Latency Audio Programming Language

70 points by ArtWomb 8y ago

ArtWomb 8y ago
SOUL Announcement – keynote at ADC 2018 https://www.youtube.com/watch?v=-GhleKNaPdk https://www.youtube.com/watch?v=-GhleKNaPdk Sorry. No release yet. But in development by Roli. Makers of cool beat making machines. Seeks to be "OpenGL API for audio" Found via the wonderful "This Is LINES" HN-style forum for sound technology ;) https://llllllll.co/t/soul-sound-language-juce-roli/17752 https://llllllll.co/t/soul-sound-language-juce-roli/17752
- IshKebab 8y ago
  So OpenAL?
  - shittyadmin 8y ago
    I assume they're more referring to the shader language and such. An easy way to achieve effects in a performance friendly manner.
- orblivion 8y ago
  I thought code-defined synths have existed for some time now. At some point he sort of makes it sound like he's inventing that idea. Am I understanding right that the language _per se_ is not that special, but the integrations with hardware and other languages are what's special?
  - emsy 8y ago
    That's my takeaway from the keynote. I've done a bit of audio coding and couldn't believe that all the processing is done on the CPU. Afaik, existing audio languages don't care much about latency (as in trying to improve it) or don't aim to become a portable standard. The ones I tried (Collider and Faust) are also not supposed to be used to implement your audio code (for media apps/games), whereas SOUL aims to fulfil these roles as well.
    - jungler 8y ago
      Audio apps have been the red-headed stepchildren of consumer operating systems since forever: the APIs aren't tuned towards latency, and they often stop providing functionality once "decode an MP3" works. And audio programming languages themselves have lingered in a research/prototyping/hobbyist zone for decades. CSound, one of the oldest examples, dates back to the 70's, but production hardware and software tends to ends up with funky embedded-code style toolchains because the higher level provisions aren't good enough for professional apps. That said I have seen a few games(indie scale projects) do something like run an instance of Pure Data rather than code up their own custom audio system. It's all a bit related to the difficulty of doing low-level, time-sensitive I/O in a computing world that has towers of abstraction. Graphics have been ushered up that abstraction path while audio is still living in the 90's, so you can see cool modern shader effects in your phone browser, but the same browser can't play Amiga tracker music without stuttering and popping. The attractive part of the SOUL pitch is in making the base tooling experience better, and then down the line using that to boost performance.
      - jancsika 8y ago
        > That said I have seen a few games(indie scale projects) do something like run an instance of Pure Data rather than code up their own custom audio system. Spore used Pure Data to do algorithmic music. IIRC somebody there went through Fux's Gradus Ad Parnassum and reduced it to a set of branches to automatically generate common practice harmonic treatment of pseudo-randomly generated melodies. :) > but the same browser can't play Amiga tracker music without stuttering and popping. You're on a much tighter schedule for the Amiga audio emulator, and the failures are more glaring. A video stream that moshes for awhile is kind of beautiful as long as the audio remains pristine IMO. On the other hand, pristine video with stuttering audio starts a countdown for canceling the streaming service. But even given that, yes-- the webaudio interface has been quite lacking.
CyberDildonics 8y ago
I really like what the Juce people have done but I still question most approaches to problems take the path of creating a new language. It seems to me something that could just as easily be a library is now constrained into sandbox where it can't be used nearly as easily or flexibly.
- teilo 8y ago
  This is not something that can be replaced with a library, anymore than a library can replace OpenGL itself. Keep in mind that this is more analagous to a GL shader. It is not a general purpose language.
  - CyberDildonics 8y ago
    There are already shading languages and DSLs that work on buffers like Halide. Halide especially seems very good at transformations on buffers.
- stabbles 8y ago
  Maybe a better approach is to take an existing language and build a DSL for it
  - kjeetgill 8y ago
    Respectfully disagree. DSLs embedded into another language are doubly terrible. Independent DSLs like GLSL or SQL are tricky to get right, but work; but they're just their own languages. With embedded DSLs you don't get the portability of a library and reusability of a library, but instead get a horribly crooked, compromised, limited grammar based on the meta-programming limitations of the hosting language (looking at you gradle!).
doitLP 8y ago
I'd never heard of audio programming as such so what I expected this to be was a kind of talk-to-text keyboard-less programming.
aaaaaaaaaab 8y ago
TLDR: audio “shaders”.
grawprog 8y ago
I'm not too sure I really understand what this is and how it's different than say the Jack2 api? http://jackaudio.org/api/ http://jackaudio.org/api/ >SOUL unlocks native-level speed, even when used within slower, safer languages. The SOUL language makes audio coding more accessible and less error-prone, improving productivity for beginners and expert professionals. This part confuses me. Is it a language or a library or both? I'm sorry if they explain this in the keynote i'm at work and can't watch it and their site seems to be lacking in information. From the comments here I understand it's trying to be the glsl of audio...but that doesn't really make sense to me. Audio doesn't really work like video shaders.
- charlesism 8y ago
  Audio doesn't really work like video shaders. I get the impression it will include a grab bag of common DSP functions, transcoding, etc. So if you have hardware that supports a function, it runs accelerated. The project looks fantastic to me. I don't know why there is so much negativity in the comments here.
  - awalton 8y ago
    > I don't know why there is so much negativity in the comments here. Probably because as of right now it's vaporware with a marketing budget. Drumming up hype for something you haven't released yet is a pretty good way to get mixed reactions.
    - julesrms 8y ago
      That's true, but actually we've seen very little negativity, which given the expected grumpiness and cynicism level of developers is a good sign! (I wish we had a marketing budget!)
  - grawprog 8y ago
    I wasn't trying to be negative. I was genuinely curious. As I said the website has very little information. It sounds like something i'd be interested in. I just don't entirely understand the purpose of it and what features it will have over existing solutions. Or even whether it's a language or a library. It says it's a language but that it can speed up higher level languages. I don't understand that statement. Will they provide bindings for other languages to link to or use? Will you have to write your own for your language of choice? Do you compile your SOUL binary first and link to it like a .so or .dll? Do you write standalone programs with it? It's fairly unclear.
pixelpoet 8y ago
How can a language itself be low latency? "Less CPU" than what? The whole page is full of nothing but nonsensical hype, I was half expecting it to say "blockchain" shortly after "paradigm shift". Also, if it's GPU based (shaders?), that's going to increase latency not decrease it, as anyone who understands bus transfers and that GPUs are optimised for throughput not latency will know.
- deleted 8y ago
  [deleted]
- charlesism 8y ago
  I think he mentioned GPU just as an example of acceleration via dedicated hardware. In the keynote, one of his example use-cases was sending SOUL code to execute on a set of digital studio monitors (ie: speakers with their own sound card).
  - pixelpoet 8y ago
    I like how you deleted your "did you even watch the video" comment before replacing it with this one, effectively turning the question back on yourself :) With that in mind, did you read my comment? I explicitly mentioned how GPU "acceleration" is going to increase latency, which contradicts one of the very few definite statements they make.
    - charlesism 8y ago
      I was hoping you wouldn't read that. I hit delete as soon as I realized I was being rude. Sorry about that.
      - pixelpoet 8y ago
        It was much more ironic than it was rude, because yes I was referring to the video, as you would surely know because you also watched it... right? Anyway, I can understand the supposition that GPUs are good for audio processing because they have oodles of processing power and memory bandwidth, however this will hurt latency because there's at least one extra bus hop (assuming the GPU can DMA to the sound card, which I doubt, and if not it's two extra hops!) and it's not like modern multicore CPUs lack throughput for audio purposes. tl;dr They claim some latency advantage and then use GPU acceleration as an example; as someone doing lots of GPU coding I find that contradictory because of the extra bus transfers.
cyberferret 8y ago
Unable to play the video on my current sketchy internet connection, but I am assuming that this language is something similar to Pure Data? [0] [0] - https://puredata.info/ https://puredata.info/
- julesrms 8y ago
  No, not really. It looks a bit like C++/Java syntax but is very different in structure
yellowapple 8y ago
Wasn't this just posted here a couple weeks ago? Same issue then as now: I ain't slogging through some keynote or giving these people my email address just to maybe learn about the actual language. No code examples, no VCS repositories, no documentation, nothing.
- chineseGoogle 8y ago
  It's trash. Dump spammy emails into the form. They have nothing to show and clearly are building a mailing list for sale to the highest bidders.
nn3 8y ago
Everything that calls itself "paradigm shift" in the first paragraph isn't.
- IshKebab 8y ago
  Well this seems to be an exception then - having watched the video it really is a new way of doing things.
jancsika 8y ago
> Its architecture improves latency and performance in ways that are impossible using current techniques, and it opens up the use of new computing platforms for audio acceleration. "Latency" may actually be the only term more subject to confusion than the term "blockchain." Is the page claiming that this language can achieve a lower latency under a realtime kernel than can be achieved by iterating over an array of function/arg pointers and delivering at regular intervals the output to ALSA at the lowest delay supported by the machine/chipset/audio hardware combination? Or is it claiming that the architecture of the language allows the programmer to more easily do complex DSP computation reliably and safely at that same round-trip latency achievable using the design I just described above? I'd imagine the answer is obviously the latter. But I'd bet most developers who read the advert would would think this mystical language allows the users to access some kind of new "el dorado latency" with commodity hardware that is not currently achievable using Supercollider or Pd. I bet that because I know users who have implied that running their single audio-generating app in conjunction with Jack-using-ALSA-backend delivers audio with lower round-trip latency than ALSA alone. They think this because the Jack page says that Jack adds zero latency to the system, and they then confuse the concept of Jack's "system latency" with the "round-trip latency" of their use-case.
- julesrms 8y ago
  It doesn't sound like you watched the talk because I do justify the latency claim in quite a lot of detail, and no, it's not just about the language being safe or fast. For example, even on commodity existing hardware, doing your audio processing inside the audio driver itself with kernel-level privileged control over its threading, affinity, and ability to write directly to the hardware buffers is going to be faster than than passing buffers and task-switching between multiple user-space threads, and dealing with all the scheduling jitter issues.
sriku 8y ago
Is SOUL intentionally made to sound like SAOL? I haven't watched the video and will do. But the dominant problem with audio programming languages has been expressivity. Things like code up single sample feedback loops (they normally are at frame level due to compute costs and other factors) -- I.e. the a-rate / k-rate divide. This is changing with wasm as a target though and we have stuff like Gibber popping up.
- julesrms 8y ago
  No - none of us had heard of SAOL, but it looks like it's so long defunct that I couldn't even find a very clear description of what it did. The website for it seems to have been bit-rotting since 2002. Obviously there's a ton of technical detail that we've not released yet, but yes, allowing streams running at arbitrary samples rates (not just "a" or "k") to be invisibly handled by the API and runtime is a concept that's deeply baked into the design.
  - sriku 8y ago
    Sounds interesting. Btw SAOL (structured audio orchestration language) was proposed as part of the mpeg7 initiative, which is itself quite long dead. SAOL was based on csound concepts. Back then, we had "csound cards" that ran csound units directly on them in real time with a near-zero (read sub ms) control-to-sound latency. Btw iirc even a control signal sent to a Bluetooth device can suffer a 3ms delay, which is low, but not "essentially zero". I get about that much from trigger to sound on the ios. BT Streams have 100x more delay than that (also noted in the keynote). PS: finally watched the video.
IshKebab 8y ago
He does say a lot about how C++ as hard and you can write these "audio shaders" in the nice easy SOUL language instead, but that beep generation function looks exactly like C++ to me! It even uses operator<< to output samples... Apart from that, this seems like a fantastic idea. My only concern would be about memory - what if you're writing a sample-based synthesizer for example? DSPs don't have enough memory for that sort of thing.
- julesrms 8y ago
  (Author here) Yeah - the design requirements of the language were for it to be instantly understandable by anyone who's done a bit of coding before, whilst enforcing the graph/node/safety architecture that's needed for it to be portable to heterogenous CPUs/DSPs. Looking exactly like C++/Java/Javascript/C# isn't an accident! And yep, some existing DSPs can be tight on memory, but it's a chicken-and-egg situation - unless people want to run complex synths on them, there's no incentive for the manufacturers to put more memory on there! If we're successful with this, we'd hope that it'll start to create a market for DSPs which DO have the oompf you need for that kind of thing.

70 points by ArtWomb 8y ago

SOUL Announcement – keynote at ADC 2018 https://www.youtube.com/watch?v=-GhleKNaPdk https://www.youtube.com/watch?v=-GhleKNaPdk Sorry. No release yet. But in development by Roli. Makers of cool beat making machines. Seeks to be "OpenGL API for audio" Found via the wonderful "This Is LINES" HN-style forum for sound technology ;) https://llllllll.co/t/soul-sound-language-juce-roli/17752 https://llllllll.co/t/soul-sound-language-juce-roli/17752

IshKebab 8y ago
So OpenAL?
- shittyadmin 8y ago
  I assume they're more referring to the shader language and such. An easy way to achieve effects in a performance friendly manner.
orblivion 8y ago
I thought code-defined synths have existed for some time now. At some point he sort of makes it sound like he's inventing that idea. Am I understanding right that the language _per se_ is not that special, but the integrations with hardware and other languages are what's special?
- emsy 8y ago
  That's my takeaway from the keynote. I've done a bit of audio coding and couldn't believe that all the processing is done on the CPU. Afaik, existing audio languages don't care much about latency (as in trying to improve it) or don't aim to become a portable standard. The ones I tried (Collider and Faust) are also not supposed to be used to implement your audio code (for media apps/games), whereas SOUL aims to fulfil these roles as well.
  - jungler 8y ago
    Audio apps have been the red-headed stepchildren of consumer operating systems since forever: the APIs aren't tuned towards latency, and they often stop providing functionality once "decode an MP3" works. And audio programming languages themselves have lingered in a research/prototyping/hobbyist zone for decades. CSound, one of the oldest examples, dates back to the 70's, but production hardware and software tends to ends up with funky embedded-code style toolchains because the higher level provisions aren't good enough for professional apps. That said I have seen a few games(indie scale projects) do something like run an instance of Pure Data rather than code up their own custom audio system. It's all a bit related to the difficulty of doing low-level, time-sensitive I/O in a computing world that has towers of abstraction. Graphics have been ushered up that abstraction path while audio is still living in the 90's, so you can see cool modern shader effects in your phone browser, but the same browser can't play Amiga tracker music without stuttering and popping. The attractive part of the SOUL pitch is in making the base tooling experience better, and then down the line using that to boost performance.
    - jancsika 8y ago
      > That said I have seen a few games(indie scale projects) do something like run an instance of Pure Data rather than code up their own custom audio system. Spore used Pure Data to do algorithmic music. IIRC somebody there went through Fux's Gradus Ad Parnassum and reduced it to a set of branches to automatically generate common practice harmonic treatment of pseudo-randomly generated melodies. :) > but the same browser can't play Amiga tracker music without stuttering and popping. You're on a much tighter schedule for the Amiga audio emulator, and the failures are more glaring. A video stream that moshes for awhile is kind of beautiful as long as the audio remains pristine IMO. On the other hand, pristine video with stuttering audio starts a countdown for canceling the streaming service. But even given that, yes-- the webaudio interface has been quite lacking.

CyberDildonics 8y ago

I really like what the Juce people have done but I still question most approaches to problems take the path of creating a new language. It seems to me something that could just as easily be a library is now constrained into sandbox where it can't be used nearly as easily or flexibly.

teilo 8y ago
This is not something that can be replaced with a library, anymore than a library can replace OpenGL itself. Keep in mind that this is more analagous to a GL shader. It is not a general purpose language.
- CyberDildonics 8y ago
  There are already shading languages and DSLs that work on buffers like Halide. Halide especially seems very good at transformations on buffers.
stabbles 8y ago
Maybe a better approach is to take an existing language and build a DSL for it
- kjeetgill 8y ago
  Respectfully disagree. DSLs embedded into another language are doubly terrible. Independent DSLs like GLSL or SQL are tricky to get right, but work; but they're just their own languages. With embedded DSLs you don't get the portability of a library and reusability of a library, but instead get a horribly crooked, compromised, limited grammar based on the meta-programming limitations of the hosting language (looking at you gradle!).

doitLP 8y ago

I'd never heard of audio programming as such so what I expected this to be was a kind of talk-to-text keyboard-less programming.

aaaaaaaaaab 8y ago

TLDR: audio “shaders”.

grawprog 8y ago

I'm not too sure I really understand what this is and how it's different than say the Jack2 api? http://jackaudio.org/api/ http://jackaudio.org/api/ >SOUL unlocks native-level speed, even when used within slower, safer languages. The SOUL language makes audio coding more accessible and less error-prone, improving productivity for beginners and expert professionals. This part confuses me. Is it a language or a library or both? I'm sorry if they explain this in the keynote i'm at work and can't watch it and their site seems to be lacking in information. From the comments here I understand it's trying to be the glsl of audio...but that doesn't really make sense to me. Audio doesn't really work like video shaders.

charlesism 8y ago
Audio doesn't really work like video shaders. I get the impression it will include a grab bag of common DSP functions, transcoding, etc. So if you have hardware that supports a function, it runs accelerated. The project looks fantastic to me. I don't know why there is so much negativity in the comments here.
- awalton 8y ago
  > I don't know why there is so much negativity in the comments here. Probably because as of right now it's vaporware with a marketing budget. Drumming up hype for something you haven't released yet is a pretty good way to get mixed reactions.
  - julesrms 8y ago
    That's true, but actually we've seen very little negativity, which given the expected grumpiness and cynicism level of developers is a good sign! (I wish we had a marketing budget!)
- grawprog 8y ago
  I wasn't trying to be negative. I was genuinely curious. As I said the website has very little information. It sounds like something i'd be interested in. I just don't entirely understand the purpose of it and what features it will have over existing solutions. Or even whether it's a language or a library. It says it's a language but that it can speed up higher level languages. I don't understand that statement. Will they provide bindings for other languages to link to or use? Will you have to write your own for your language of choice? Do you compile your SOUL binary first and link to it like a .so or .dll? Do you write standalone programs with it? It's fairly unclear.

pixelpoet 8y ago

How can a language itself be low latency? "Less CPU" than what? The whole page is full of nothing but nonsensical hype, I was half expecting it to say "blockchain" shortly after "paradigm shift". Also, if it's GPU based (shaders?), that's going to increase latency not decrease it, as anyone who understands bus transfers and that GPUs are optimised for throughput not latency will know.

deleted 8y ago
[deleted]
charlesism 8y ago
I think he mentioned GPU just as an example of acceleration via dedicated hardware. In the keynote, one of his example use-cases was sending SOUL code to execute on a set of digital studio monitors (ie: speakers with their own sound card).
- pixelpoet 8y ago
  I like how you deleted your "did you even watch the video" comment before replacing it with this one, effectively turning the question back on yourself :) With that in mind, did you read my comment? I explicitly mentioned how GPU "acceleration" is going to increase latency, which contradicts one of the very few definite statements they make.
  - charlesism 8y ago
    I was hoping you wouldn't read that. I hit delete as soon as I realized I was being rude. Sorry about that.
    - pixelpoet 8y ago
      It was much more ironic than it was rude, because yes I was referring to the video, as you would surely know because you also watched it... right? Anyway, I can understand the supposition that GPUs are good for audio processing because they have oodles of processing power and memory bandwidth, however this will hurt latency because there's at least one extra bus hop (assuming the GPU can DMA to the sound card, which I doubt, and if not it's two extra hops!) and it's not like modern multicore CPUs lack throughput for audio purposes. tl;dr They claim some latency advantage and then use GPU acceleration as an example; as someone doing lots of GPU coding I find that contradictory because of the extra bus transfers.

cyberferret 8y ago

Unable to play the video on my current sketchy internet connection, but I am assuming that this language is something similar to Pure Data? [0] [0] - https://puredata.info/ https://puredata.info/

julesrms 8y ago
No, not really. It looks a bit like C++/Java syntax but is very different in structure

yellowapple 8y ago

Wasn't this just posted here a couple weeks ago? Same issue then as now: I ain't slogging through some keynote or giving these people my email address just to maybe learn about the actual language. No code examples, no VCS repositories, no documentation, nothing.

chineseGoogle 8y ago
It's trash. Dump spammy emails into the form. They have nothing to show and clearly are building a mailing list for sale to the highest bidders.

nn3 8y ago

Everything that calls itself "paradigm shift" in the first paragraph isn't.

IshKebab 8y ago
Well this seems to be an exception then - having watched the video it really is a new way of doing things.

jancsika 8y ago

> Its architecture improves latency and performance in ways that are impossible using current techniques, and it opens up the use of new computing platforms for audio acceleration. "Latency" may actually be the only term more subject to confusion than the term "blockchain." Is the page claiming that this language can achieve a lower latency under a realtime kernel than can be achieved by iterating over an array of function/arg pointers and delivering at regular intervals the output to ALSA at the lowest delay supported by the machine/chipset/audio hardware combination? Or is it claiming that the architecture of the language allows the programmer to more easily do complex DSP computation reliably and safely at that same round-trip latency achievable using the design I just described above? I'd imagine the answer is obviously the latter. But I'd bet most developers who read the advert would would think this mystical language allows the users to access some kind of new "el dorado latency" with commodity hardware that is not currently achievable using Supercollider or Pd. I bet that because I know users who have implied that running their single audio-generating app in conjunction with Jack-using-ALSA-backend delivers audio with lower round-trip latency than ALSA alone. They think this because the Jack page says that Jack adds zero latency to the system, and they then confuse the concept of Jack's "system latency" with the "round-trip latency" of their use-case.

julesrms 8y ago
It doesn't sound like you watched the talk because I do justify the latency claim in quite a lot of detail, and no, it's not just about the language being safe or fast. For example, even on commodity existing hardware, doing your audio processing inside the audio driver itself with kernel-level privileged control over its threading, affinity, and ability to write directly to the hardware buffers is going to be faster than than passing buffers and task-switching between multiple user-space threads, and dealing with all the scheduling jitter issues.

sriku 8y ago

Is SOUL intentionally made to sound like SAOL? I haven't watched the video and will do. But the dominant problem with audio programming languages has been expressivity. Things like code up single sample feedback loops (they normally are at frame level due to compute costs and other factors) -- I.e. the a-rate / k-rate divide. This is changing with wasm as a target though and we have stuff like Gibber popping up.

julesrms 8y ago
No - none of us had heard of SAOL, but it looks like it's so long defunct that I couldn't even find a very clear description of what it did. The website for it seems to have been bit-rotting since 2002. Obviously there's a ton of technical detail that we've not released yet, but yes, allowing streams running at arbitrary samples rates (not just "a" or "k") to be invisibly handled by the API and runtime is a concept that's deeply baked into the design.
- sriku 8y ago
  Sounds interesting. Btw SAOL (structured audio orchestration language) was proposed as part of the mpeg7 initiative, which is itself quite long dead. SAOL was based on csound concepts. Back then, we had "csound cards" that ran csound units directly on them in real time with a near-zero (read sub ms) control-to-sound latency. Btw iirc even a control signal sent to a Bluetooth device can suffer a 3ms delay, which is low, but not "essentially zero". I get about that much from trigger to sound on the ios. BT Streams have 100x more delay than that (also noted in the keynote). PS: finally watched the video.

IshKebab 8y ago

He does say a lot about how C++ as hard and you can write these "audio shaders" in the nice easy SOUL language instead, but that beep generation function looks exactly like C++ to me! It even uses operator<< to output samples... Apart from that, this seems like a fantastic idea. My only concern would be about memory - what if you're writing a sample-based synthesizer for example? DSPs don't have enough memory for that sort of thing.

julesrms 8y ago
(Author here) Yeah - the design requirements of the language were for it to be instantly understandable by anyone who's done a bit of coding before, whilst enforcing the graph/node/safety architecture that's needed for it to be portable to heterogenous CPUs/DSPs. Looking exactly like C++/Java/Javascript/C# isn't an accident! And yep, some existing DSPs can be tight on memory, but it's a chicken-and-egg situation - unless people want to run complex synths on them, there's no incentive for the manufacturers to put more memory on there! If we're successful with this, we'd hope that it'll start to create a market for DSPs which DO have the oompf you need for that kind of thing.