7 ms·

I'm still patiently waiting for future digital mixing consoles to do all processing in software on inexpensive x86 or ARM processors. Currently due to latency a

I'm still patiently waiting for future digital mixing consoles to do all processing in software on inexpensive x86 or ARM processors. Currently due to latency and reliability requirements all DSP work is done on dedicated chips or FPGA which brings up the BOM and engineering cost. They often have a small ARM/Linux module which is used for the displays and network control. The CPU tech is here today, and modern general purpose processors do a good job of handling low-latency audio. Someone just needs to put all that together in a unified and stable package...

TylerE 4y ago
Modern CPUs are great if your watt (and thus thermal) profile is unlimited. Not so great when they aren't.
- strangetortoise 4y ago
  Does this form a problem for digital mixing consoles? As far as I know, these already have fairly beefy fans for heat exhaust, so I don't know if they couldn't just add more airflow?
  - TylerE 4y ago
    Not especially. I'm more thinking of things that have a potential of being near a live mic, where a cooling fan is a no go, at least without a defeat switch.
- digitallyfree 4y ago
  Modern consoles are plugged into the wall and are large enough for active cooling. For a 1U rack unit they can be cooled with small server fans; noise is not an issue as the amp fans are just as loud and the music will drown them all out anyways.
- londons_explore 4y ago
  Most processing of audio isn't CPU performance limited... For most realtime audio mixing, it doesn't matter if you use 1 watt or 10 watts for your CPU - the big speakers will easily be drawing far more, and the performers time will be costing far more than the electric bill anyway.
  - PaulDavisThe1st 4y ago
    > Most processing of audio isn't CPU performance limited Processing, mostly true (though there are still some reverbs that can chew an awful lot of cycles). Synthesis, however, is a different story. That is definitely CPU performance limited once you get into substantial numbers of voices/tracks/synths.
  - TylerE 4y ago
    It's CPU limited in the sense that it's extraordinarily latency/jitter sensitive.
huehehue 4y ago
Not exactly what you're describing, but I've been running my band directly into Logic Pro on an M1 (both for recording, and live shows). Dry signals go through amp sims and effects processors, and then route to both a FOH mix and an in-ear mix all on the laptop. Wish I had seen the OP's project months ago, but one bonus of the setup I describe is the ability to swap effects after the fact (by virtue of having the dry signal) and the ability to automate effects (so I can engage distortion etc as soon as we hit measure XYZ instead of having to click a pedal)
- digitallyfree 4y ago
  I've done live mixing with a laptop, DAW, and interface in the past and it does work but it's not something I would be comfortable with for an important show. Even with something like SAC (which is specifically designed for the task) the chance of hangs, crashes, etc. go up as at the end of the day it's just a program running on top of your OS. The setup and config also gets a bit hacky and you'll be the only one able to use it. As far as I know the only specialized system that does this is the Waves LV1 which has a dedicated OS running on top of x86 hardware for processing. While I haven't tried it apparently it works quite well. However I was more thinking of mixers like the QSC Touchmix/X32/etc. where the DSP probably eats up quite a bit of the unit cost, and how the price could be significantly brought down if the innards merely contained analog I/O and converters all tying into a powerful SoC.
  - kimburgess 4y ago
    It's interesting you mentioned QSC. They're one of the few vendors in the installed audio space that have made the jump to COTS hardware: https://www.qsc.com/resource-files/productresources/dn/dsp_cores/core_5200/q_dn_core_5200_specs.pdf https://www.qsc.com/resource-files/productresources/dn/dsp_c....
bob1029 4y ago
Modern CPUs are surprisingly good at low-latency video as well. SIMD on something like a Zen4 core is a really big deal if used properly. I've got some prototypes in C# that can draw a 1080p bitmap and encode to JPEG in under 10ms. Using single threading, socket mux servers and aggressive multimedia timers means my network delay is usually right at 1ms. I feel like if you are just worried about audio, there is definitely enough bandwidth here to do what you need to per unit time.
- bluGill 4y ago
  While video needs more CPU power, it can tolerate higher latency than sound.
  - bob1029 4y ago
    Fair point. Audio also has a tendency to require more serialized throughput in complex signal chains. Video is more trivial to chunk out and process in parallel.
  - kimburgess 4y ago
    Spot on. For context: 60fps gives you ~17ms of wiggle room before you start dropping/delaying frames. With 96kHz audio you have 0.01ms between samples. Drift above 0.05ms and you'll start introducing time domain issues in the human hearing range. In other words, 'realtime' audio processing needs to happen 1700x faster that 'realtime' image work. Bandwidth isn't limiting factor, deterministic and uniform latency is that challenge for any signal as the sample rate goes up.
    - rerdavies 4y ago
      Drift really isn't a problem with audio cards. Latency is. To put things in perspective, it takes sound 1ms to travel 1 foot. The general consensus is that guitar effects have to have no more than 10ms latency.
      - kimburgess 4y ago
PaulDavisThe1st 4y ago
> I'm still patiently waiting for future digital mixing consoles to do all processing in software on inexpensive x86 or ARM processors. Harrison Consoles have done this for more than 10 years. I cannot confirm this in the same way, but I think it also likely that both Lawo and Studer digital consoles do this, and also possibly Allen & Heath. All 4 companies run Linux internally on their consoles.

TylerE 4y ago

Modern CPUs are great if your watt (and thus thermal) profile is unlimited. Not so great when they aren't.

strangetortoise 4y ago
Does this form a problem for digital mixing consoles? As far as I know, these already have fairly beefy fans for heat exhaust, so I don't know if they couldn't just add more airflow?
- TylerE 4y ago
  Not especially. I'm more thinking of things that have a potential of being near a live mic, where a cooling fan is a no go, at least without a defeat switch.
digitallyfree 4y ago
Modern consoles are plugged into the wall and are large enough for active cooling. For a 1U rack unit they can be cooled with small server fans; noise is not an issue as the amp fans are just as loud and the music will drown them all out anyways.
londons_explore 4y ago
Most processing of audio isn't CPU performance limited... For most realtime audio mixing, it doesn't matter if you use 1 watt or 10 watts for your CPU - the big speakers will easily be drawing far more, and the performers time will be costing far more than the electric bill anyway.
- PaulDavisThe1st 4y ago
  > Most processing of audio isn't CPU performance limited Processing, mostly true (though there are still some reverbs that can chew an awful lot of cycles). Synthesis, however, is a different story. That is definitely CPU performance limited once you get into substantial numbers of voices/tracks/synths.
- TylerE 4y ago
  It's CPU limited in the sense that it's extraordinarily latency/jitter sensitive.

huehehue 4y ago

Not exactly what you're describing, but I've been running my band directly into Logic Pro on an M1 (both for recording, and live shows). Dry signals go through amp sims and effects processors, and then route to both a FOH mix and an in-ear mix all on the laptop. Wish I had seen the OP's project months ago, but one bonus of the setup I describe is the ability to swap effects after the fact (by virtue of having the dry signal) and the ability to automate effects (so I can engage distortion etc as soon as we hit measure XYZ instead of having to click a pedal)

digitallyfree 4y ago
I've done live mixing with a laptop, DAW, and interface in the past and it does work but it's not something I would be comfortable with for an important show. Even with something like SAC (which is specifically designed for the task) the chance of hangs, crashes, etc. go up as at the end of the day it's just a program running on top of your OS. The setup and config also gets a bit hacky and you'll be the only one able to use it. As far as I know the only specialized system that does this is the Waves LV1 which has a dedicated OS running on top of x86 hardware for processing. While I haven't tried it apparently it works quite well. However I was more thinking of mixers like the QSC Touchmix/X32/etc. where the DSP probably eats up quite a bit of the unit cost, and how the price could be significantly brought down if the innards merely contained analog I/O and converters all tying into a powerful SoC.
- kimburgess 4y ago
  It's interesting you mentioned QSC. They're one of the few vendors in the installed audio space that have made the jump to COTS hardware: https://www.qsc.com/resource-files/productresources/dn/dsp_cores/core_5200/q_dn_core_5200_specs.pdf https://www.qsc.com/resource-files/productresources/dn/dsp_c....

bob1029 4y ago

Modern CPUs are surprisingly good at low-latency video as well. SIMD on something like a Zen4 core is a really big deal if used properly. I've got some prototypes in C# that can draw a 1080p bitmap and encode to JPEG in under 10ms. Using single threading, socket mux servers and aggressive multimedia timers means my network delay is usually right at 1ms. I feel like if you are just worried about audio, there is definitely enough bandwidth here to do what you need to per unit time.

bluGill 4y ago
While video needs more CPU power, it can tolerate higher latency than sound.
- bob1029 4y ago
  Fair point. Audio also has a tendency to require more serialized throughput in complex signal chains. Video is more trivial to chunk out and process in parallel.
- kimburgess 4y ago
  Spot on. For context: 60fps gives you ~17ms of wiggle room before you start dropping/delaying frames. With 96kHz audio you have 0.01ms between samples. Drift above 0.05ms and you'll start introducing time domain issues in the human hearing range. In other words, 'realtime' audio processing needs to happen 1700x faster that 'realtime' image work. Bandwidth isn't limiting factor, deterministic and uniform latency is that challenge for any signal as the sample rate goes up.
  - rerdavies 4y ago
    Drift really isn't a problem with audio cards. Latency is. To put things in perspective, it takes sound 1ms to travel 1 foot. The general consensus is that guitar effects have to have no more than 10ms latency.
    - kimburgess 4y ago

PaulDavisThe1st 4y ago

> I'm still patiently waiting for future digital mixing consoles to do all processing in software on inexpensive x86 or ARM processors. Harrison Consoles have done this for more than 10 years. I cannot confirm this in the same way, but I think it also likely that both Lawo and Studer digital consoles do this, and also possibly Allen & Heath. All 4 companies run Linux internally on their consoles.