6 ms·

is that not what the linux realtime patches do? https://wiki.archlinux.org/index.php/Realtime_kernel_patchset https://wiki.archlinux.org/index.php/Realtime_kern

is that not what the linux realtime patches do? https://wiki.archlinux.org/index.php/Realtime_kernel_patchset https://wiki.archlinux.org/index.php/Realtime_kernel_patchse...

rectang 6y ago
I've been researching this for a long time, and it's unclear to me whether Linux with PREEMPT_RT patches would meet that sub-ms requirement. I see numbers all over the place from various sources, from sub-millisecond to more than 10 ms. 10 milliseconds is often considered the threshold of human perception with regards to audio latency. (It's possible that it's lower under some circumstances, but 10 ms is a good enough approximation for my purposes.) My goal is to create a sound/music tool which runs lots of DSP and which has a total system latency of under 10 milliseconds, including OS, application (including intrinsic latency of DSP procedures), hardware, and sound in air (about 1 ms per 3 meters). When I read about latency, I often see "we're at 6-9 ms, that's good enough because it's not perceptible". Unfortunately, that's not good enough if there are several components which contribute to total system latency and they are all pushing 10 ms. Hence, my sub-ms requirement for the OS. Committing to a platform will be a costly choice. I don't want to invest in writing for realtime Linux only to find that I really need to run a hardcore RTOS, or run dedicated DSP chips, etc. That's why I'm interested in whether running a unikernel can offer stronger guarantees.
- PaulDavisThe1st 6y ago
  I write realtime audio software for Linux, and have done so for 20 years. While your goal is admirable - it certainly is possible to come up with scenarios where "sub-ms" latency is desirable - it's really not relevant. Your stated goal ("...lots of DSP ... under 10 msec") is already entirely achievable on Linux, assuming you're close enough to your speakers (or wearing headphones). But sub-msec can only make sense here if it describes scheduler latency, since there's no audio hardware that can function in the sub-msec range. The linux scheduler is way, way below that threshold and SCHED_FIFO threads will see that performance barring hardware issues (c.f. https://manual.ardour.org/setting-up-your-system/the-right-computer-system-for-digital-audio/ https://manual.ardour.org/setting-up-your-system/the-right-c...) Finally .... writing audio software is a lot of fun. But please don't just jump in without seeing if you can instead contribute to an existing project first. The field is littered with the dead and half-dead corpses representing the discarded work of developers who thought it would be fun, and then moved on.
  - rectang 6y ago
    As for contributing to existing projects, I hear you — I've been highly active in Open Source for well over a decade. Conceptually, the closest thing to what I want to do is Pure Data — and I have in fact contributed to it. But I'm also extremely motivated and willing to go all the way down and write the entire thing from scratch if I have to. Or to learn enough about every last step in the chain that I can actually control for latency — and it may be that it takes about the same amount of work. Finding numbers I can trust on latency seems hopeless. Everybody fudges rather than fails.
    - PaulDavisThe1st 6y ago
      The word "latency" covers many things, many subtly different from each other. One of its meanings, unrelated to the one you're talking about, is the delay in signal flow caused by algorithms (many digital filters, for example). Often called "plugin delay compensation", or more generically "latency compensation". Let me just point out that it has taken 20 years and a guy whose PhD thesis was about latency compensation in a DAW to finally "fix" this sort of "latency" in Ardour. This is a massively harder problem from a design perspective than the scheduling latency issues you've referred to. [ EDIT: to be fair, the actual correct solution to latency compensation didn't take 20 years to implement, more like a year or so when taking place within the context of a large existing code base. ]
      - rectang 6y ago
        The complexity of solving such problems generally is actually what drives me to start from scratch: rather than solve an intractable general problem, instead limit the scope of where the project must run and what it must do. My perception is that I am not capable of reliably tuning a general purpose OS for a total system latency of under 10 msec. I can't give the total system a hard number and believe that it will obey; instead, I need to perform a lot of esoteric tweakery of subsystems I probably don't understand. The system won't alert me reliably when it fails, but will instead either drop out or just give me more latency than I asked for — and there are innumerable factors outside my control that could cause it to fail. However, the composition tool I want to create has a pretty small set of requirements, if I accept that it only need serve my particular use case. So how about I ensure that my app is the only thing running on the hardware, via unikernel, or RTOS, or even bare metal? Implementing all of my compositional requirements is probably easier and certainly more rewarding than tweaking latency parameters without having confidence that my results will be enduring or predictable. If it turns out that the knowledge I gain during that exercise allows me to control latency well enough and I can return to mainline operating systems and contribute to existing projects, all the better. I don't really want to go down this path, but I'm not willing to accept a tool that maybe-kinda-sorta-sometimes meets my absolute requirements.

is that not what the linux realtime patches do? https://wiki.archlinux.org/index.php/Realtime_kernel_patchset https://wiki.archlinux.org/index.php/Realtime_kernel_patchse...

rectang 6y ago

I've been researching this for a long time, and it's unclear to me whether Linux with PREEMPT_RT patches would meet that sub-ms requirement. I see numbers all over the place from various sources, from sub-millisecond to more than 10 ms. 10 milliseconds is often considered the threshold of human perception with regards to audio latency. (It's possible that it's lower under some circumstances, but 10 ms is a good enough approximation for my purposes.) My goal is to create a sound/music tool which runs lots of DSP and which has a total system latency of under 10 milliseconds, including OS, application (including intrinsic latency of DSP procedures), hardware, and sound in air (about 1 ms per 3 meters). When I read about latency, I often see "we're at 6-9 ms, that's good enough because it's not perceptible". Unfortunately, that's not good enough if there are several components which contribute to total system latency and they are all pushing 10 ms. Hence, my sub-ms requirement for the OS. Committing to a platform will be a costly choice. I don't want to invest in writing for realtime Linux only to find that I really need to run a hardcore RTOS, or run dedicated DSP chips, etc. That's why I'm interested in whether running a unikernel can offer stronger guarantees.

PaulDavisThe1st 6y ago
I write realtime audio software for Linux, and have done so for 20 years. While your goal is admirable - it certainly is possible to come up with scenarios where "sub-ms" latency is desirable - it's really not relevant. Your stated goal ("...lots of DSP ... under 10 msec") is already entirely achievable on Linux, assuming you're close enough to your speakers (or wearing headphones). But sub-msec can only make sense here if it describes scheduler latency, since there's no audio hardware that can function in the sub-msec range. The linux scheduler is way, way below that threshold and SCHED_FIFO threads will see that performance barring hardware issues (c.f. https://manual.ardour.org/setting-up-your-system/the-right-computer-system-for-digital-audio/ https://manual.ardour.org/setting-up-your-system/the-right-c...) Finally .... writing audio software is a lot of fun. But please don't just jump in without seeing if you can instead contribute to an existing project first. The field is littered with the dead and half-dead corpses representing the discarded work of developers who thought it would be fun, and then moved on.
- rectang 6y ago
  As for contributing to existing projects, I hear you — I've been highly active in Open Source for well over a decade. Conceptually, the closest thing to what I want to do is Pure Data — and I have in fact contributed to it. But I'm also extremely motivated and willing to go all the way down and write the entire thing from scratch if I have to. Or to learn enough about every last step in the chain that I can actually control for latency — and it may be that it takes about the same amount of work. Finding numbers I can trust on latency seems hopeless. Everybody fudges rather than fails.
  - PaulDavisThe1st 6y ago
    The word "latency" covers many things, many subtly different from each other. One of its meanings, unrelated to the one you're talking about, is the delay in signal flow caused by algorithms (many digital filters, for example). Often called "plugin delay compensation", or more generically "latency compensation". Let me just point out that it has taken 20 years and a guy whose PhD thesis was about latency compensation in a DAW to finally "fix" this sort of "latency" in Ardour. This is a massively harder problem from a design perspective than the scheduling latency issues you've referred to. [ EDIT: to be fair, the actual correct solution to latency compensation didn't take 20 years to implement, more like a year or so when taking place within the context of a large existing code base. ]
    - rectang 6y ago
      The complexity of solving such problems generally is actually what drives me to start from scratch: rather than solve an intractable general problem, instead limit the scope of where the project must run and what it must do. My perception is that I am not capable of reliably tuning a general purpose OS for a total system latency of under 10 msec. I can't give the total system a hard number and believe that it will obey; instead, I need to perform a lot of esoteric tweakery of subsystems I probably don't understand. The system won't alert me reliably when it fails, but will instead either drop out or just give me more latency than I asked for — and there are innumerable factors outside my control that could cause it to fail. However, the composition tool I want to create has a pretty small set of requirements, if I accept that it only need serve my particular use case. So how about I ensure that my app is the only thing running on the hardware, via unikernel, or RTOS, or even bare metal? Implementing all of my compositional requirements is probably easier and certainly more rewarding than tweaking latency parameters without having confidence that my results will be enduring or predictable. If it turns out that the knowledge I gain during that exercise allows me to control latency well enough and I can return to mainline operating systems and contribute to existing projects, all the better. I don't really want to go down this path, but I'm not willing to accept a tool that maybe-kinda-sorta-sometimes meets my absolute requirements.