4 ms·

Does a realtime kernel actually have any benefits for low latency in audio? I would have thought that givig the audio process full priority would have the grea

by peepee1982 4y ago

Does a realtime kernel actually have any benefits for low latency in audio? I would have thought that givig the audio process full priority would have the greatest benefit.

creshal 4y ago
Stock linux kernels have too much jitter even with highest priority, you really need the RT patches.
- peepee1982 4y ago
  Okay, that makes sense.
bluGill 4y ago
Audio is one of the most real time applications. Most real time applications have much lesser time controls. If you are running a motor the inertia of the whole system means that you really only need to adjust the speed every few seconds. If you are running a stepper motor you need much better time controls on the steps, but that is typically done by a dedicated CPU, leaving the logic of what speed to go to something else. Note, audio is about latency. Other real time applications need much more CPU power, but they can accept more jitter and latency.
- creshal 4y ago
  > Audio is one of the most real time applications. Most real time applications have much lesser time controls. If you are running a motor the inertia of the whole system means that you really only need to adjust the speed every few seconds. Anecdotally that's one of the "eureka" moments behind SpaceX's industry-disrupting low production costs. They realized they only really needed ±20ms precision for communication between different sensors/actuators, which made heavily multiplexed ethernet busses feasible, and got rid of massive amounts of dedicated serial lines and related hardware. 13 years later, competitors are just about ready to catch up.
  - flas9sd 4y ago
    interesting, can I go on reading about this? I found https://www.cnbc.com/2018/05/11/full-elon-musk-transcript-about-spacex-falcon-9-block-5.html https://www.cnbc.com/2018/05/11/full-elon-musk-transcript-ab... but it didn't speak on the tolerable latency to go all ethernet
- peepee1982 4y ago
  I would have thought that the operating system interfering with the scheduling of my DAW would increase latency. But since I know too little about how DAWs handle the scheduling I can only make some naive assumptions. I want my audio processes to have full priority over any other task on my OS. I don't mind if my DAW increases the latency of other processes running on my PC.
yobbo 4y ago
It depends on the application. More serious music applications have separate "render mode" which works in non-realtime. Also "real-time" apps will use a rendering buffer of say 100 ms. Someone would need to benchmark how small the buffer needs to be for RT to matter. Time-resolution for midi-input is not high enough for it to matter; it will be quantized to midi-time anyway. It is critical for audio-effects and synthesizers that are taking live input and emitting live output. These exist, but I'm not sure they are commonly used on linux or if it's wise running with low resources.
- creshal 4y ago
  > or if it's wise running with low resources. It's not really a matter of how many resources a system has, without proper realtime scheduling a thread might still be randomly delayed for just a few milliseconds too many due to scheduling screwups.
  - thomastjeffery 4y ago
    Yes, but that may or may not be important. For real-time performance, yes; but that doesn't represent the whole, or even the majority, of audio work.
  - peepee1982 4y ago
    Yes, but I want my DAW to have as much control over the scheduling, not the OS. I don't want the OS to take away resources from my DAW at any point in time, exactly for the reason to avoid dropouts.
- sublinear 4y ago
  > More serious music applications I don't understand this perspective. I've seen musicians at all levels use all sorts of crazy instruments. The output of live audio processing may not be used in the final result, but it's very important for capturing a performance. > not sure they are commonly used on linux or if it's wise running with low resources Linux-based studios are the norm for many hobbyists and in educational settings where the goal is actually teaching and not just indoctrinating a new cohort of commercial software users. Also, who said anything about "low resources" and what does that even mean?
gnulinux 4y ago
> I would have thought that givig the audio process full priority would have the greatest benefit. I think you're conflating two separate things. Giving full priority to an audio processing thread will ensure it will be scheduled as much as possible which increases throughput. This is crucial for things like rendering (audio or video) where the process takes in the order of minutes or hours and needs to be completed as fast possible. In the other application, such as synthesizers, virtual instruments (MIDI keyboard -> MIDI signal -> DAW -> sound wave), music notation etc you care only about latency. Human brain is hard-wired to observe even the minutest latency in music, especially in bass register, any millisecond delay will be perceptible to highly trained professional musicians. To fix this, you need to tell kernel, when it receives a signal, the thread cannot be scheduled out. Here, you don't care about throughput, but have extreme latency requirements. I, as a hobbyist composer, personally use a traditional archlinux setup for my studio. It's not real time at all. It honestly sucks, the latency is extremely noticable, which makes some applications, such as playing on my keyboard and hearing soundfonts through DAW impossible; as the latency makes me confused and prevents me from playing the keyboard properly. This is ok for me, as it doesn't distrupt my personal workflow (I have a very slow, calculated approach to composing, and when I need to improvise, I do it on my piano) so I didn't care about optimizing it. But if you're a professional musician and need good latency, something like this is can be crucial.
- peepee1982 4y ago
  Yeah, but that's not what I meant. I want to give my DAW full priority and let it handle the scheduling. I want any other process on my operating system to take a backseat.
  - gnulinux 4y ago
    Yeah, I don't think that'll work in practice. Unless your OS is real-time, what you described cannot work because audio latency has to do with more than your DAW's threads. It also has to do with OS polling MIDI data, or sending data to audio driver. These need to be completed on a deadline. Realtime scheduling can be less efficient than fair scheduling which is why if your DAW has all the power to set its own scheduling you'll only do compute, and not enough IO. This means poor latency.

Does a realtime kernel actually have any benefits for low latency in audio? I would have thought that givig the audio process full priority would have the greatest benefit.

creshal 4y ago

Stock linux kernels have too much jitter even with highest priority, you really need the RT patches.

peepee1982 4y ago
Okay, that makes sense.

bluGill 4y ago

Audio is one of the most real time applications. Most real time applications have much lesser time controls. If you are running a motor the inertia of the whole system means that you really only need to adjust the speed every few seconds. If you are running a stepper motor you need much better time controls on the steps, but that is typically done by a dedicated CPU, leaving the logic of what speed to go to something else. Note, audio is about latency. Other real time applications need much more CPU power, but they can accept more jitter and latency.

creshal 4y ago
> Audio is one of the most real time applications. Most real time applications have much lesser time controls. If you are running a motor the inertia of the whole system means that you really only need to adjust the speed every few seconds. Anecdotally that's one of the "eureka" moments behind SpaceX's industry-disrupting low production costs. They realized they only really needed ±20ms precision for communication between different sensors/actuators, which made heavily multiplexed ethernet busses feasible, and got rid of massive amounts of dedicated serial lines and related hardware. 13 years later, competitors are just about ready to catch up.
- flas9sd 4y ago
  interesting, can I go on reading about this? I found https://www.cnbc.com/2018/05/11/full-elon-musk-transcript-about-spacex-falcon-9-block-5.html https://www.cnbc.com/2018/05/11/full-elon-musk-transcript-ab... but it didn't speak on the tolerable latency to go all ethernet
peepee1982 4y ago
I would have thought that the operating system interfering with the scheduling of my DAW would increase latency. But since I know too little about how DAWs handle the scheduling I can only make some naive assumptions. I want my audio processes to have full priority over any other task on my OS. I don't mind if my DAW increases the latency of other processes running on my PC.

yobbo 4y ago

It depends on the application. More serious music applications have separate "render mode" which works in non-realtime. Also "real-time" apps will use a rendering buffer of say 100 ms. Someone would need to benchmark how small the buffer needs to be for RT to matter. Time-resolution for midi-input is not high enough for it to matter; it will be quantized to midi-time anyway. It is critical for audio-effects and synthesizers that are taking live input and emitting live output. These exist, but I'm not sure they are commonly used on linux or if it's wise running with low resources.

creshal 4y ago
> or if it's wise running with low resources. It's not really a matter of how many resources a system has, without proper realtime scheduling a thread might still be randomly delayed for just a few milliseconds too many due to scheduling screwups.
- thomastjeffery 4y ago
  Yes, but that may or may not be important. For real-time performance, yes; but that doesn't represent the whole, or even the majority, of audio work.
- peepee1982 4y ago
  Yes, but I want my DAW to have as much control over the scheduling, not the OS. I don't want the OS to take away resources from my DAW at any point in time, exactly for the reason to avoid dropouts.
sublinear 4y ago
> More serious music applications I don't understand this perspective. I've seen musicians at all levels use all sorts of crazy instruments. The output of live audio processing may not be used in the final result, but it's very important for capturing a performance. > not sure they are commonly used on linux or if it's wise running with low resources Linux-based studios are the norm for many hobbyists and in educational settings where the goal is actually teaching and not just indoctrinating a new cohort of commercial software users. Also, who said anything about "low resources" and what does that even mean?

gnulinux 4y ago

> I would have thought that givig the audio process full priority would have the greatest benefit. I think you're conflating two separate things. Giving full priority to an audio processing thread will ensure it will be scheduled as much as possible which increases throughput. This is crucial for things like rendering (audio or video) where the process takes in the order of minutes or hours and needs to be completed as fast possible. In the other application, such as synthesizers, virtual instruments (MIDI keyboard -> MIDI signal -> DAW -> sound wave), music notation etc you care only about latency. Human brain is hard-wired to observe even the minutest latency in music, especially in bass register, any millisecond delay will be perceptible to highly trained professional musicians. To fix this, you need to tell kernel, when it receives a signal, the thread cannot be scheduled out. Here, you don't care about throughput, but have extreme latency requirements. I, as a hobbyist composer, personally use a traditional archlinux setup for my studio. It's not real time at all. It honestly sucks, the latency is extremely noticable, which makes some applications, such as playing on my keyboard and hearing soundfonts through DAW impossible; as the latency makes me confused and prevents me from playing the keyboard properly. This is ok for me, as it doesn't distrupt my personal workflow (I have a very slow, calculated approach to composing, and when I need to improvise, I do it on my piano) so I didn't care about optimizing it. But if you're a professional musician and need good latency, something like this is can be crucial.

peepee1982 4y ago
Yeah, but that's not what I meant. I want to give my DAW full priority and let it handle the scheduling. I want any other process on my operating system to take a backseat.
- gnulinux 4y ago
  Yeah, I don't think that'll work in practice. Unless your OS is real-time, what you described cannot work because audio latency has to do with more than your DAW's threads. It also has to do with OS polling MIDI data, or sending data to audio driver. These need to be completed on a deadline. Realtime scheduling can be less efficient than fair scheduling which is why if your DAW has all the power to set its own scheduling you'll only do compute, and not enough IO. This means poor latency.