3 ms·
The "lots of different hardware" side of it is where audio hardware was 20 years ago. Devices with multiple streams in hardware, and without. Devices with built
by PaulDavisThe1st 6y ago
The "lots of different hardware" side of it is where audio hardware was 20 years ago. Devices with multiple streams in hardware, and without. Devices with builtin processing effect X, and without. Devices that could use huge buffer sizes and ones that could only use small ones. Devices that could use just 2 buffers, and devices that could use many. Devices that supported all sample rates in hardware, devices that only supported one.
Things are not totally cleared up, but the variation has been much reduced, and as a result the design of the driver API and then user-space APIs above that have become more straightforward and more consistent even across platforms.
>20-100x performance advantage.
Yes, for sure this is one aspect of the comparison that really breaks down. Sure, there are audio interfaces (e.g. those from UAD) that have a boatload of audio processing capabilities on board, and offload that DSP to the hardware saves oodles of CPU cycles. But these are unusual, far from the norm, and not widely used. Audio interfaces have mostly converged on a massively less capable hardware design than they had 20 years ago.
- modeless 6y agoYeah, most audio processing is well within CPU capabilities even with a 100x performance disadvantage, so there's just no need to reach for hardware acceleration and all the complexity that comes with it. Rendering cutting edge graphics, especially on modern 4k 120Hz displays, is well outside CPU capabilities so that 100x boost is unfortunately essential. Interestingly, for really advanced audio processing like sound propagation in 3D environments or physics-based sound synthesis [1], GPUs will probably be necessary. [1] https://graphics.stanford.edu/projects/wavesolver/ https://graphics.stanford.edu/projects/wavesolver/
- PaulDavisThe1st 6y agophysically modelled synthesis is no more advanced than anything going on in a modern game engine for video, and in some senses, significantly less. so to restate something i said just below, the fact the "most audio processing is well within CPU capabilities" is only really saying "audio processing hasn't been pushed in the way that video has because there's been no motivating factor for it". if enough people were actually interested in large scale physically modelled synthesis, nobody would see "audio processing [as] well within CPU capabilities". why aren't there many people who want/do this? Well ... classic chicken and egg. The final market is no doubt smaller than the game market, so I'd probably blame lack of the hardware for preventing the growth of the user community. ironically, sound propagation in 3D environments is more or less within current CPU capabilities.
- modeless 6y agoI agree that physically modeled synthesis could easily push audio processing requirements far beyond CPU limits and require hardware acceleration similar to graphics. But I disagree that it's a chicken and egg problem. It's a cost/benefit tradeoff and the cost of realtime physically modelled synthesis is higher than its benefit for most applications. Mostly because the alternative of just mixing prerecorded samples works so well, psychoacoustically. (Though I think VR may be the point where this starts to break down; I'd love to try a physically modeled drum set in VR.) I also doubt that specialized audio hardware could provide a large benefit over a GPU for physically modeled synthesis. Ultimately what you need is FLOPS and GPUs these days are pretty much FLOPS maximizing machines with a few fixed function bits bolted on the side. Swapping the graphics fixed function bits for audio fixed function bits is not going to make a huge difference. Certainly nothing like the difference between CPU and GPU.
- PaulDavisThe1st 6y agoI suppose that I broadly agree with most of this. Only caveat would be that you also a very fast roundtrip from where all those FLOPS happen to the audio interface buffer, and my understanding is even today in 2020, current GPUs do not provide that (data has to come back to the CPU, and for understandable reasons, this hasn't been optimized as a low latency path). Re: physically modelled drums ... watch this (from 2008) !!! https://vimeo.com/2433260 https://vimeo.com/2433260 Consider the sound made right before the 3:00 mark when Randy just wipes his hand over the "surface". You cannot do this with samples :) It says something about the distance between my vision of the world and reality that even though I thought this was the most exciting thing I saw in 2008, there's almost no trace of it having any effect on music technology down here in 2020.
- modeless 6y agoVery cool video! I agree that it's a little disappointing that stuff like that hasn't taken off. I would say that it's not impossible to have a low latency path from the GPU to the audio buffers on a modern system. It would take some attention to detail but especially with the new explicit graphics APIs you should be able to do it quite well. In some cases the audio output actually goes through the GPU so in theory you could skip readback, though you probably can't with current APIs. Prioritization of time critical GPU work has also gotten some attention recently because VR compositors need it. In VR you do notice every missed frame, in the pit of your stomach...