7 ms·

TL;DR the Linux layer (ALSA) and the Java layer (Audio Flinger) use widely compatible but high latency techniques, whereas Apple designed their API's and hardwa

by ynniv 12y ago

TL;DR the Linux layer (ALSA) and the Java layer (Audio Flinger) use widely compatible but high latency techniques, whereas Apple designed their API's and hardware such that these layers can be optimized to almost nothing. (From the article: http://superpowered.com/wp-content/uploads/2015/04/Android-Audio-Path-Latency-Superpowered-Audio700px.gif http://superpowered.com/wp-content/uploads/2015/04/Android-A...)

szantog 12y ago
Audio Flinger is not Java. On the graph you see absolutely no part which runs on Java. Android's media architecture with it's "push" design is not more compatible than anything else with a "pull". The many layers doesn't add any flexibility, they are "only" the result of many years hacking.
likeclockwork 12y ago
I don't know about Android but you can get quite good audio latency with a full Linux install, using ALSA(via JACK), provided your audio interface is up to the task.
nileshtrivedi 12y ago
ALSA with JACK has been shown to give latencies less than 2ms.
cnvogel 12y ago
ALSA is not the problem, it works very fine for the low-latency case, I can reliably run soundcards with light processing load at 96 kHz @ 64 frames/period (20.7ms --> 1.4 ms latency) on a quad core i5, e.g. for running a reverb effect, but most of the time I only record and will settle for 1024 frames/period or so. (210ms --> 20ms). The period size, just for completeness, is the number of samples recorded on each block that is forwarded to the audio processing application. If whatever audio framework you use doesn't allow to run processing with a input-to-output delay (latency) of two times the period size, it's broken (probably the case for Audio Flinger at Android, don't know much about it). ➜ ~ jackd -d alsa -p 64 -r 96000 jackdmp 1.9.10 Copyright 2001-2005 Paul Davis and others. Copyright 2004-2014 Grame. (...) creating alsa driver hw:0|hw:0|64|2|96000|0|0|nomon|swmeter|-|32bit configuring for 96000Hz, period = 64 frames (0.7 ms), buffer = 2 periods ALSA: final selected sample format for capture: 32bit integer little-endian ALSA: use 2 periods for capture ALSA: final selected sample format for playback: 32bit integer little-endian ALSA: use 2 periods for playback (this is on my laptop, just for illustration purposes)
- jblow 12y ago
  But ... why is there a period size? Isn't that a broken design that can only introduce latency? What is wrong with "however much audio data is ready when the application asks, send it"?
  - cnvogel 12y ago
    Well... it's like all audio chipsets work nowadays. Only some DSPs will be able to efficiently handle single-frame data processing, but they have the help of dedicated address generators and lightweight interrupts synchronized to the digital interface. If you write "process however much audio data is ready", then you already imply that your CPU will not be up to speed to process 48000 interrupts/second reliably and you need some buffering. And if you have to assume that sometimes you'll miss 100 samples (which, then, you'll process en-block), this means that to work reliably, you'll have to start at least 100 samples early so that you don't miss the deadline of the DAC, because the DAC will, with intractably output one sample every 48000th of a second. This already implies some kind of periodic processing of blocks, doesn't it? (and yes, such a scheme will theoretically allow you to half the latency from something to 2period-size to 1period-size + the time for processing) Third, a lot of the algorithms for processing audio can be implemented much more efficiently if you have a known block size and don't have to calculate your filters or convolutions with constantly changing number of samples for every step. Also efficiency of processing will decrease (reloading the cache after each interrupt when switching from processing plugin to processing plugin), so the time spent on calculating per frame will go up if your period size gets smaller. At one point you'll need exactly one "period size" to calculate one period size worth of samples: That's the maximum your machine can handle, and at that point you'll have a latency of your "period size"*2, which is exactly the same as running with a fixed period-size ;-). And as you can choose the period-size rather freely (maybe completely arbitrary, maybe 2^n, depends on the chipset/hardware) there's no disadvantage left.
    - jblow 12y ago
      As someone who has actually done a good amount of soft-real-time audio programming, I can tell that you probably haven't. Everything you are saying about CPU speeds is made-up nonsense. Look into how these things are done on systems where folks actually care about latency (for example, commercial audio hardware, game consoles, etc).

by ynniv 12y ago

szantog 12y ago

Audio Flinger is not Java. On the graph you see absolutely no part which runs on Java. Android's media architecture with it's "push" design is not more compatible than anything else with a "pull". The many layers doesn't add any flexibility, they are "only" the result of many years hacking.

likeclockwork 12y ago

I don't know about Android but you can get quite good audio latency with a full Linux install, using ALSA(via JACK), provided your audio interface is up to the task.

nileshtrivedi 12y ago

ALSA with JACK has been shown to give latencies less than 2ms.

cnvogel 12y ago

ALSA is not the problem, it works very fine for the low-latency case, I can reliably run soundcards with light processing load at 96 kHz @ 64 frames/period (20.7ms --> 1.4 ms latency) on a quad core i5, e.g. for running a reverb effect, but most of the time I only record and will settle for 1024 frames/period or so. (210ms --> 20ms). The period size, just for completeness, is the number of samples recorded on each block that is forwarded to the audio processing application. If whatever audio framework you use doesn't allow to run processing with a input-to-output delay (latency) of two times the period size, it's broken (probably the case for Audio Flinger at Android, don't know much about it). ➜ ~ jackd -d alsa -p 64 -r 96000 jackdmp 1.9.10 Copyright 2001-2005 Paul Davis and others. Copyright 2004-2014 Grame. (...) creating alsa driver hw:0|hw:0|64|2|96000|0|0|nomon|swmeter|-|32bit configuring for 96000Hz, period = 64 frames (0.7 ms), buffer = 2 periods ALSA: final selected sample format for capture: 32bit integer little-endian ALSA: use 2 periods for capture ALSA: final selected sample format for playback: 32bit integer little-endian ALSA: use 2 periods for playback (this is on my laptop, just for illustration purposes)

jblow 12y ago
But ... why is there a period size? Isn't that a broken design that can only introduce latency? What is wrong with "however much audio data is ready when the application asks, send it"?
- cnvogel 12y ago
  Well... it's like all audio chipsets work nowadays. Only some DSPs will be able to efficiently handle single-frame data processing, but they have the help of dedicated address generators and lightweight interrupts synchronized to the digital interface. If you write "process however much audio data is ready", then you already imply that your CPU will not be up to speed to process 48000 interrupts/second reliably and you need some buffering. And if you have to assume that sometimes you'll miss 100 samples (which, then, you'll process en-block), this means that to work reliably, you'll have to start at least 100 samples early so that you don't miss the deadline of the DAC, because the DAC will, with intractably output one sample every 48000th of a second. This already implies some kind of periodic processing of blocks, doesn't it? (and yes, such a scheme will theoretically allow you to half the latency from something to 2period-size to 1period-size + the time for processing) Third, a lot of the algorithms for processing audio can be implemented much more efficiently if you have a known block size and don't have to calculate your filters or convolutions with constantly changing number of samples for every step. Also efficiency of processing will decrease (reloading the cache after each interrupt when switching from processing plugin to processing plugin), so the time spent on calculating per frame will go up if your period size gets smaller. At one point you'll need exactly one "period size" to calculate one period size worth of samples: That's the maximum your machine can handle, and at that point you'll have a latency of your "period size"*2, which is exactly the same as running with a fixed period-size ;-). And as you can choose the period-size rather freely (maybe completely arbitrary, maybe 2^n, depends on the chipset/hardware) there's no disadvantage left.
  - jblow 12y ago
    As someone who has actually done a good amount of soft-real-time audio programming, I can tell that you probably haven't. Everything you are saying about CPU speeds is made-up nonsense. Look into how these things are done on systems where folks actually care about latency (for example, commercial audio hardware, game consoles, etc).