4 ms·

GPUs have some use for physical modeling (e.g. convolutional reverb — which at least 5 years ago was not practical to do in realtime on a CPU). i toyed around w

GPUs have some use for physical modeling (e.g. convolutional reverb — which at least 5 years ago was not practical to do in realtime on a CPU). i toyed around with such a plugin a while back but i think it takes more expertise to program any reverb to sound obviously superior to the CPU-optimized algorithms we’re more practically familiar with. also additive synthesis greatly benefited from GPU acceleration at one point (particularly if you’re making full use of panning or detuning individual harmonics), but with SIMD/CPU improvements it’s much less relevant now. latency as a barrier to GPU use is sometimes overstated. you can reach sub-ms latency overhead on a GPU roundtrip. maybe relevant if you’re really targeting 5 ms through the whole pipeline, but not for amateur audio. of course, one issue is that these latencies stack if your host architecture uses something like VST/LV2 which only knows how to pass frames via the CPU. it’s one reason why i’d love for some declarative approach to plugins instead of shipping binaries which assume so much about their environment (even just shipping LLVM IR could let you batch GPU calls). but it seems that approach may forever be niche.

duped 4y ago
I haven't delved into it too deep. Some things are embarrassingly parallel, like additive synthesis. Some things can be made quite parallel like convolution. But a lot of stuff is not. The distinction I think that needs to be made between GPUs for graphics vs audio is that all GPU algorithms are built on the assumption that there exists a reasonable compute pipeline that every kind of processing can use or exploit. That is not true for audio processing (your compute kernels for convolution and IIR filtering are completely different, for example, even static IIRs and IIRs that undergo real time modulation have very different topologies). The thing I don't yet know about GPU Audio (the company) is how they deal with routing and process chains. Sub millisecond latency is fine, but if you have two dozen plugins in a chain (or god help you, a real DAG) then how do they handle the delay compensated I/O without adding extra latency for each sequential edge in the graph without the host being aware? It's not big things that hurt latency, it's more a death by a thousand cuts in practice.
nyanpasu64 4y ago
How practical is a "split reverb" with the low-latency parts computed on CPU and the high-latency parts computed on GPU (like how partitioned convolution is already performed)? Or is reverb/convolution not processor-intensive enough to reduce CPU workloads significantly?
- duped 4y ago
  I don't think there's a meaningful difference between high and low latency segments of reverb algorithms except for some exotic hybrid models that use a mix of convolution for early reflections and FDNs for the longer tail, but the synchronization would absolutely suck to write and test
  - nyanpasu64 4y ago
    Naive convolution takes time proportional to the number of input samples multiplied by the length of the reverb impulse response (kernel), which becomes rather impractical for seconds-long reverb tails with >100k samples, convolved by ~50k samples per second. The usual solution is overlap-{add/discard} block convolution, where computational power decreases as the amount of input data processed at once increases (to a degree). But if you want short latency, you must process input data in small chunks, which increases CPU load. I think the trick of block convolution is to convolve the beginning of the reverb kernel in small chunks, and the later portions in large chunks (buffering small chunks of input into large chunks, convolving large chunks once available, and playing the result back in small chunks), and adding the two reverb tails (though I haven't fully studied or tried implementing it myself, and IIRC it's patented anyway).
- namibj 4y ago
  Well, in principle, you can operate with single-digit-microseconds latency via CUDA at least, it just involves breaking free of the normal kernel grid approach and using dedicated pre-spawned threads that run as persistent hardware threads on the GPU cores. It's just that Windows resets the GPU driver if anything "hangs" for more than 5-10 seconds (like a persistent audio processing thread), and GPUs aren't always good at running multiple applications at the same time concurrently, so it may prevent graphics from happening while that audio thread is running (easily solved by not using that GPU to run any displays). Edit: if anything, you can probably run harder real time code on a normal Nvidia GPU than on an Intel/AMD CPU, all of recent generation(s). Reason being that GPUs don't run management work on the application cores and have manually managed local scratchpad memory with bounded latency, but x86 CPUs have a system management mode for firmware/"BIOS" hooks that can interrupt the kernel at any time.
ratww 4y ago
Convolution Reverb was definitely practical on CPUs for much longer than 10 years, and a lot of people were using it, even on underpowered computers (I was using a base model Macbook Air myself since 2011, and yes, LOTS of long reverbs). Physical modelled reverb has been possible on consumer CPUs since the 90s of course (convolution reverb isn't in the category of "physical modelling" btw). If you're talking about physical modelling synthesis, it uses lots of recursion, so it's less interesting for GPUs. I'll also dispute the claim that low latency isn't necessary for amateur audio. Recording with low latency is necessary for everyone performing on an instrument. One can achieve low latency with external DSP or external sound processors, but most amateurs are only relying on plugins instead of more expensive hardware, and that's where their monitoring comes from. People programming instruments rather than playing (and not exclusively using samplers, which are I/O bound rather than CPU bound) still need real time processing for other tasks. Try mixing with Bluetooth: it's hilarious how much latency can slow you down even when you're not performing.
klodolph 4y ago
I used convolutional reverb in real-time on a Mac from 2001, with single-core G4 processor running at less than 1 GHz.

duped 4y ago

I haven't delved into it too deep. Some things are embarrassingly parallel, like additive synthesis. Some things can be made quite parallel like convolution. But a lot of stuff is not. The distinction I think that needs to be made between GPUs for graphics vs audio is that all GPU algorithms are built on the assumption that there exists a reasonable compute pipeline that every kind of processing can use or exploit. That is not true for audio processing (your compute kernels for convolution and IIR filtering are completely different, for example, even static IIRs and IIRs that undergo real time modulation have very different topologies). The thing I don't yet know about GPU Audio (the company) is how they deal with routing and process chains. Sub millisecond latency is fine, but if you have two dozen plugins in a chain (or god help you, a real DAG) then how do they handle the delay compensated I/O without adding extra latency for each sequential edge in the graph without the host being aware? It's not big things that hurt latency, it's more a death by a thousand cuts in practice.

nyanpasu64 4y ago

How practical is a "split reverb" with the low-latency parts computed on CPU and the high-latency parts computed on GPU (like how partitioned convolution is already performed)? Or is reverb/convolution not processor-intensive enough to reduce CPU workloads significantly?

duped 4y ago
I don't think there's a meaningful difference between high and low latency segments of reverb algorithms except for some exotic hybrid models that use a mix of convolution for early reflections and FDNs for the longer tail, but the synchronization would absolutely suck to write and test
- nyanpasu64 4y ago
  Naive convolution takes time proportional to the number of input samples multiplied by the length of the reverb impulse response (kernel), which becomes rather impractical for seconds-long reverb tails with >100k samples, convolved by ~50k samples per second. The usual solution is overlap-{add/discard} block convolution, where computational power decreases as the amount of input data processed at once increases (to a degree). But if you want short latency, you must process input data in small chunks, which increases CPU load. I think the trick of block convolution is to convolve the beginning of the reverb kernel in small chunks, and the later portions in large chunks (buffering small chunks of input into large chunks, convolving large chunks once available, and playing the result back in small chunks), and adding the two reverb tails (though I haven't fully studied or tried implementing it myself, and IIRC it's patented anyway).
namibj 4y ago
Well, in principle, you can operate with single-digit-microseconds latency via CUDA at least, it just involves breaking free of the normal kernel grid approach and using dedicated pre-spawned threads that run as persistent hardware threads on the GPU cores. It's just that Windows resets the GPU driver if anything "hangs" for more than 5-10 seconds (like a persistent audio processing thread), and GPUs aren't always good at running multiple applications at the same time concurrently, so it may prevent graphics from happening while that audio thread is running (easily solved by not using that GPU to run any displays). Edit: if anything, you can probably run harder real time code on a normal Nvidia GPU than on an Intel/AMD CPU, all of recent generation(s). Reason being that GPUs don't run management work on the application cores and have manually managed local scratchpad memory with bounded latency, but x86 CPUs have a system management mode for firmware/"BIOS" hooks that can interrupt the kernel at any time.

ratww 4y ago

Convolution Reverb was definitely practical on CPUs for much longer than 10 years, and a lot of people were using it, even on underpowered computers (I was using a base model Macbook Air myself since 2011, and yes, LOTS of long reverbs). Physical modelled reverb has been possible on consumer CPUs since the 90s of course (convolution reverb isn't in the category of "physical modelling" btw). If you're talking about physical modelling synthesis, it uses lots of recursion, so it's less interesting for GPUs. I'll also dispute the claim that low latency isn't necessary for amateur audio. Recording with low latency is necessary for everyone performing on an instrument. One can achieve low latency with external DSP or external sound processors, but most amateurs are only relying on plugins instead of more expensive hardware, and that's where their monitoring comes from. People programming instruments rather than playing (and not exclusively using samplers, which are I/O bound rather than CPU bound) still need real time processing for other tasks. Try mixing with Bluetooth: it's hilarious how much latency can slow you down even when you're not performing.

klodolph 4y ago

I used convolutional reverb in real-time on a Mac from 2001, with single-core G4 processor running at less than 1 GHz.