4 ms·

I am hoping for a plugin standard that will take advantage of unused GPU cycles. Haven't see that yet.

by oidar 4y ago

digitalslr 4y ago
GPU Audio is doing it with VST 3. And it's probably possible with CLAP too? Processing audio on a GPU is a lower level issue than a plugin standard which describes the interface between the DAW and the plugin (how audio, parameters, notes, modulation data, etc. are exchanged).
- deleted 4y ago
  [deleted]
duped 4y ago
Pro audio plugins have lower latency and harder constraints than graphics processing. You cannot drop a frame, and roundtrip latency is ideally below 5ms. GPU Audio (the company) has a lot of hype but it's not really clear if they gain a ton. The existing analogs to GPUs are expensive pieces of special purpose DSPs like Waves Grid, Universal Audio's UAD, and the biggest which are Avid PT HDX accelerators. It turns out modern CPUs are plenty fast for heavy audio DSP. The point of friction is synchronization and the memory wall, most of the time. It turns out that audio DSP algorithms don't generally benefit from the same kind of parallelism you get on a GPU, since even the basic stuff you'd like to do (filtering, algorithmic reverb, etc) is a bunch of recursive/sequential algorithms that have a non-trivial cost to convert to a parallel form for SIMD evaluation, and the gains are not linear due to repeated computation (classic example, converting an exponential moving average which is used everywhere to N lanes of SIMD is a lot less than an N times speed up). That's not to say there isn't exciting work being done (GPU audio, for example). It's just really hard and not a magic bullet. It's not an Audio Processing Unit, after all. It also has video output, but not high fidelity audio i/o.
- PaulDavisThe1st 4y ago
  Wow. A low level, technical, detailed post about audio processing on HN that leaves me without feeling the need to correct any of it. Congratulations, and thanks!
- colinsane 4y ago
  GPUs have some use for physical modeling (e.g. convolutional reverb — which at least 5 years ago was not practical to do in realtime on a CPU). i toyed around with such a plugin a while back but i think it takes more expertise to program any reverb to sound obviously superior to the CPU-optimized algorithms we’re more practically familiar with. also additive synthesis greatly benefited from GPU acceleration at one point (particularly if you’re making full use of panning or detuning individual harmonics), but with SIMD/CPU improvements it’s much less relevant now. latency as a barrier to GPU use is sometimes overstated. you can reach sub-ms latency overhead on a GPU roundtrip. maybe relevant if you’re really targeting 5 ms through the whole pipeline, but not for amateur audio. of course, one issue is that these latencies stack if your host architecture uses something like VST/LV2 which only knows how to pass frames via the CPU. it’s one reason why i’d love for some declarative approach to plugins instead of shipping binaries which assume so much about their environment (even just shipping LLVM IR could let you batch GPU calls). but it seems that approach may forever be niche.
  - duped 4y ago
    I haven't delved into it too deep. Some things are embarrassingly parallel, like additive synthesis. Some things can be made quite parallel like convolution. But a lot of stuff is not. The distinction I think that needs to be made between GPUs for graphics vs audio is that all GPU algorithms are built on the assumption that there exists a reasonable compute pipeline that every kind of processing can use or exploit. That is not true for audio processing (your compute kernels for convolution and IIR filtering are completely different, for example, even static IIRs and IIRs that undergo real time modulation have very different topologies). The thing I don't yet know about GPU Audio (the company) is how they deal with routing and process chains. Sub millisecond latency is fine, but if you have two dozen plugins in a chain (or god help you, a real DAG) then how do they handle the delay compensated I/O without adding extra latency for each sequential edge in the graph without the host being aware? It's not big things that hurt latency, it's more a death by a thousand cuts in practice.
  - nyanpasu64 4y ago
    How practical is a "split reverb" with the low-latency parts computed on CPU and the high-latency parts computed on GPU (like how partitioned convolution is already performed)? Or is reverb/convolution not processor-intensive enough to reduce CPU workloads significantly?
pier25 4y ago
https://www.gpu.audio/ https://www.gpu.audio/ They did this demo at NAMM recently: https://www.youtube.com/watch?v=IFdgymosszA https://www.youtube.com/watch?v=IFdgymosszA

I am hoping for a plugin standard that will take advantage of unused GPU cycles. Haven't see that yet.

by oidar 4y ago

I am hoping for a plugin standard that will take advantage of unused GPU cycles. Haven't see that yet.

GPU Audio is doing it with VST 3. And it's probably possible with CLAP too? Processing audio on a GPU is a lower level issue than a plugin standard which describes the interface between the DAW and the plugin (how audio, parameters, notes, modulation data, etc. are exchanged).

deleted 4y ago
[deleted]

duped 4y ago

Pro audio plugins have lower latency and harder constraints than graphics processing. You cannot drop a frame, and roundtrip latency is ideally below 5ms. GPU Audio (the company) has a lot of hype but it's not really clear if they gain a ton. The existing analogs to GPUs are expensive pieces of special purpose DSPs like Waves Grid, Universal Audio's UAD, and the biggest which are Avid PT HDX accelerators. It turns out modern CPUs are plenty fast for heavy audio DSP. The point of friction is synchronization and the memory wall, most of the time. It turns out that audio DSP algorithms don't generally benefit from the same kind of parallelism you get on a GPU, since even the basic stuff you'd like to do (filtering, algorithmic reverb, etc) is a bunch of recursive/sequential algorithms that have a non-trivial cost to convert to a parallel form for SIMD evaluation, and the gains are not linear due to repeated computation (classic example, converting an exponential moving average which is used everywhere to N lanes of SIMD is a lot less than an N times speed up). That's not to say there isn't exciting work being done (GPU audio, for example). It's just really hard and not a magic bullet. It's not an Audio Processing Unit, after all. It also has video output, but not high fidelity audio i/o.

PaulDavisThe1st 4y ago
Wow. A low level, technical, detailed post about audio processing on HN that leaves me without feeling the need to correct any of it. Congratulations, and thanks!
colinsane 4y ago
GPUs have some use for physical modeling (e.g. convolutional reverb — which at least 5 years ago was not practical to do in realtime on a CPU). i toyed around with such a plugin a while back but i think it takes more expertise to program any reverb to sound obviously superior to the CPU-optimized algorithms we’re more practically familiar with. also additive synthesis greatly benefited from GPU acceleration at one point (particularly if you’re making full use of panning or detuning individual harmonics), but with SIMD/CPU improvements it’s much less relevant now. latency as a barrier to GPU use is sometimes overstated. you can reach sub-ms latency overhead on a GPU roundtrip. maybe relevant if you’re really targeting 5 ms through the whole pipeline, but not for amateur audio. of course, one issue is that these latencies stack if your host architecture uses something like VST/LV2 which only knows how to pass frames via the CPU. it’s one reason why i’d love for some declarative approach to plugins instead of shipping binaries which assume so much about their environment (even just shipping LLVM IR could let you batch GPU calls). but it seems that approach may forever be niche.
- duped 4y ago
  I haven't delved into it too deep. Some things are embarrassingly parallel, like additive synthesis. Some things can be made quite parallel like convolution. But a lot of stuff is not. The distinction I think that needs to be made between GPUs for graphics vs audio is that all GPU algorithms are built on the assumption that there exists a reasonable compute pipeline that every kind of processing can use or exploit. That is not true for audio processing (your compute kernels for convolution and IIR filtering are completely different, for example, even static IIRs and IIRs that undergo real time modulation have very different topologies). The thing I don't yet know about GPU Audio (the company) is how they deal with routing and process chains. Sub millisecond latency is fine, but if you have two dozen plugins in a chain (or god help you, a real DAG) then how do they handle the delay compensated I/O without adding extra latency for each sequential edge in the graph without the host being aware? It's not big things that hurt latency, it's more a death by a thousand cuts in practice.
- nyanpasu64 4y ago
  How practical is a "split reverb" with the low-latency parts computed on CPU and the high-latency parts computed on GPU (like how partitioned convolution is already performed)? Or is reverb/convolution not processor-intensive enough to reduce CPU workloads significantly?

pier25 4y ago

https://www.gpu.audio/ https://www.gpu.audio/ They did this demo at NAMM recently: https://www.youtube.com/watch?v=IFdgymosszA https://www.youtube.com/watch?v=IFdgymosszA