9 ms·

Android Audio Latency

102 points by kawera 11y ago

vlaskovits 11y ago
Hi all -- Patrick from Superpowered.com. We're working on an update to our Android 10 ms Problem article with new data --- quick question: What's not clear about Android latency? What questions would you like answered that you haven't seen answered online?
wawi 11y ago
It is really an interesting aspect of the whole Android versus iOS "war", that indeed Google did not up-front plan to make the OS actually useful for realtime Audio/synthesis/processing; but instead, Android has had more of a 'data-center'-like design, wherein the kernel and its onboard assets is more of a 'managed host' than it is 'realtime embedded' sort of system .. That the Android designers are having to play catchup with this speaks a lot, I think, of how significant design-decisions were influenced by the territory from which the originated - the web, managed systems - and the realm which they hoped to capture - the embedded system in your pocket. Android was not designed up-front to be an amazing media system; iOS, was. Thus, iOS has more synth-plugins and a vibrant, thriving Audio-Application eco-system (seriously, its million-dollar) whereas Android has a lot of new players, but so far: not such a great playground, really, for "Musicians". In fact, while iOS has really impinged upon the Music-Instrument (Digital) industry, Android has yet to make a dent. So it is an interesting time to be having new solutions to what have been, relatively, frustrating and troublesome aspects of Android realtime Audio, as a developer.
- agumonkey 11y ago
  Was iOS evolved from the iPod OS ? a lot ? a little ?
  - on_and_off 11y ago
    I have no idea of the internals of the iPod. As I understand it, iOS benefited immensely from the work already done on OsX. I don't even know whether they planned for it or just benefited from a mature module already in place.
    - sirkneeland 11y ago
      IIRC the very first iPods actually had licensed much of the under the hood software. I could be confusing them with Palm OS, which did the same for the kernel of their first Pilots...
  - milkytron 11y ago
    iOS is just a rebranding of what used to be iPhone OS.
  - S_A_P 11y ago
    No- iOS is a stripped down OSX. iPod classic os was not related to that. The Core Audio framework is what makes iOS low latency. It was developed after the iPod.
- 6stringmerc 11y ago
  Yeah right out of the gate iOS was the clear leader, but I think that's the legacy of Core Audio from their main OS? To me it's not surprising that iOS was the go-to for audio, based on historical prescedent that both MS and, from what I understand, Linux platforms didn't really have strong native audio processing for low latency. I mean, yes, it was totally do-able (MS via ASIO4ALL, I know there were a few Linux based music systems even that ran on Atom netbooks), but plug-and-play wasn't quite the same. Even audio cards had some driver issues in the MS realm, speaking from personal experience. It's rather sad because the MS environment allowed for a lot of homebrew plugins - the VSTs. Those won't run in Apple's ecosystem. Other plugins do (ex. UAD) but for the longest time those required (still do?) their own hardware processing cards. $$$$$$$. iOS bridged the gap in a lot of ways between the 'average' and 'pro' musician being able to access nice digital sounds on reasonably priced hardware. I'm a genuine PC guy (Ableton Live, Reaper) but get a lot of mileage out of a second gen iPad 2 - specifically Propellerhead's Figure software, which is flat out fantastic. I've made an entire EP in GarageBand while traveling. Sure, some devices won't work with iOS devices (my LPK25 will run on the iPad 2's port via camera connector USB dongle, but the MPD8 won't), and yet some are outstanding (looking at the Line6 / iK media products for guitarists and amp simulation). There's so much of a head start with iOS that honestly I kind of think there's little to no point in Android or MS trying to play catch up in the mobile sphere. MS has a winner with the Surface line because they run real software, but in the 'pure' mobile space, iOS has all the goods in my opinion.
  - sirkneeland 11y ago
    Given the eye-watering disparity in gross revenue generated from iOS vs OS X, you really have to wonder which one is their "main OS" at this point...
    - 6stringmerc 11y ago
      Oh no doubt, I probably should have termed it 'legacy' OS considering the priorities these past years. After the 'prosumer' pivot of Final Cut Pro that angered a vocal group who had spent tens of thousands on workcenter setups, to the 'single port' type design on the new laptop, I'm pretty sure a lot of the die-hard creative types aren't going to feel really comfortable they're being considered for future releases. I honestly don't think Apple cares, because those users are a minority anyway.
- kllrnohj 11y ago
  I would more say this was just an aspect that Google inhereted by going with Linux ( see also https://xkcd.com/619/ https://xkcd.com/619/ ) iOS inhereted a strong foundation for this as OSX has a history of strong support for this sort of content creation. Linux does not.
  - wawi 11y ago
    I don't personally think this can be blamed on Linux - Audio works on Linux as well as OSX does, when the hardware is selected and configured, as it is on OSX. Linux has alternative and competing - and thriving - audio subsystems, and this means that systems integration has to pay attention to what its doing. This didn't happen in Androids' case; where other Linux distros had rock-solid and professional-quality audio subsystems and software integration, Android had an NIH implementation that ended up having to re-invent the wheel, to boot.
    - Zigurd 11y ago
      This is basically correct. Android replacing the Linux graphics and UI stack turned out to be the right thing. Multimedia, not so much.
    - kllrnohj 11y ago
      > where other Linux distros had rock-solid and professional-quality audio subsystems and software integration Such as? Remember we're talking Linux in the 2005-era (~2.6.10-ish), not Linux today. JACK was still fairly new at that time. ALSA was still mostly broken at the time. Most stuff still wanted OSS. Basic "does sound come out of the speakers?" was often broken, and god help you if you wanted two things to play audio at the same time. And you still need to solve the driver problem, which JACK, etc... don't help with at all.
      - simoncion 11y ago
        > And you still need to solve the driver problem... Um. If we're talking about 2005, we're talking about Android. If we're talking about Android, we're talking about brand-spanking new hardware on a brand-spanking new phone. This means that the audio driver didn't exist, which means that it was being written from scratch. This makes the "driver problem" a non-issue. > Remember we're talking Linux in the 2005-era (~2.6.10-ish), not Linux today. ... ALSA was still mostly broken at the time. Most stuff still wanted OSS. I never tried to do any "pro audio", [0] but that's not how I remember it. ALSA worked just fine. Anything that wanted OSS worked well enough with ALSA's OSS-compat API. I can't remember if the software mixer was around or not at the time, but I remember that when it did arrive, it eliminated that problem with single-stream sound cards. [0] But -as I remember it- most folks doing pro audio used specialized, standalone hardware for it back then.
- pjmlp 11y ago
  You also see that in the effort that Apple makes for providing game development tools and libraries, whereas Google doesn't even provide proper editor support for shaders.
- Zigurd 11y ago
  You are reading a lot into it. Yes, Android's designers had a very different mentality from the traditional RTOS-for-a-handset designers. But that does not mean they had a "data center' mindset. Indeed one of the most common error or people who do have a "data center mindset" is that they load up on libraries that do various useful things for their Android development and then blow out the method and heap limits. Understanding Android requires a unique design and implementation mentality that is neither RTOS oriented, nor "small *nix in a phone" oriented, nor server Java oriented. It's a clever multi-processing managed language runtime with aggressive memory recovery that's capable of running many managed runtime processes, none of which can hog memory globally. Without having been designed for real time audio from the beginning, retrofitting that capability was hard to do. Apple made a lot of hay out of lack of real time audio processing. But only in one domain of apps. These are the kinds of trade-offs that get products out the door.
exDM69 11y ago
Given the title, I would have liked to see some in-depth analysis on why the latency is as bad as it is? Which parts of the software stack are causing it? Even though the Linux kernel is less than ideal for audio, it can still provide a latency low enough for music, ie. 10 ms or less. This is an unbearable situation. There are lots of interesting music apps on the iOS that I would like to use, but only a handful are available on Android. And even if they are available, they are not necessarily usable.
- rogerbinns 11y ago
  > Which parts of the software stack are causing it? There is a diagram lower down in the article with the components (ALSA, audioflinger etc). http://www.androidpolice.com/wp-content/uploads/2015/11/nexus2cee_Android-Audio-Path-Latency-Superpowered-Audio700px.gif http://www.androidpolice.com/wp-content/uploads/2015/11/nexu...
  - dgregd 11y ago
    Actually Android standard doesn't include ALSA. There is a thin sound HAL above ALSA, OSS and other Linux sound technologies. Of course many phone vendors use ALSA to implement Android HAL layer. It seems that Samsung uses its own sound drivers and therefore they have small audio latency compared to other Android vendors.
  - exDM69 11y ago
    The infographic looked so much like an ad that I skipped over it. So this is one of the "good" Android audio stacks, getting 36ms latency. But half of that is "wasted" in user space, in AudioFlinger as well as the app ring buffer (which seems pretty big to me). I'd like to see the comparison to a "bad" Android device. Doing some internet searches, I found someone who had plugged in PulseAudio to an Android phone and got 20 ms latency instead of the 170+ ms latency from AudioFlinger on the same device [0]. That's pretty huge. Ideally, there should be no user space middleware component processing audio in between, this could half the latency (according to the diagram), bringing the latency to acceptable levels. But ALSA isn't really intended for routing audio on the fly conveniently (e.g. when plugging/unplugging headphones), so there is some room for improvement too. Overall, the situation is pretty sad. Linux can do better than that but a modern multimedia device (desktop or mobile) has several audio outputs and hot-plugging, which makes the situation rather difficult using ALSA only. And the userspace audio components just aren't great. http://arunraghavan.net/2012/01/pulseaudio-vs-audioflinger-fight/ http://arunraghavan.net/2012/01/pulseaudio-vs-audioflinger-f...
on_and_off 11y ago
If you want an in-depth explanation (AP mostly skims the surface) : https://www.youtube.com/watch?v=d3kfEeMZ65c https://www.youtube.com/watch?v=d3kfEeMZ65c is a must-watch. Obviously, it does not cover the work accomplished these past two years, but it gives a good idea of the challenges of audio latency on Android
deleted 11y ago
[deleted]
rcthompson 11y ago
OK, what about the absurd latency for responding to hardware buttons on headphones and the like? My experience across 3 different Android phones has been that when I press the button on my headphones to pause whatever I'm listening to, it will respond about half the time, and when it does respond, the latency is randomly somewhere between 1 second and ten minutes. This is consistent across several sets of headphones, both wired and Bluetooth, and even true of the pause button that appears on the lock screen, which I'm assuming uses the same mechanism to signal the app.
- ultramancool 11y ago
  Fuck, what is up with this? I don't care about the audio latency this article is talking about and I never used to have this problem either. But recently, maybe just since I flashed latest CyanogenMod my phone sometimes responds instantly, but sometimes doesn't respond to the headphone button press until I press the power button or wait 30s+. It's incredibly annoying. I'm using CM on a OnePlus One with Xiaomi Piston 3 headphones. Considering the power button works, I suspect this may have something to do with sleep states?
- untog 11y ago
  Yep, this is beyond frustrating. Especially as on numerous occasions I've hit the earphone pause button, had it do nothing, pull out my phone, hit pause, unplugged my earphones and THEN the signal finally arrives, UNpausing my music for all to hear.
  - wlesieutre 11y ago
    Same here. Whenever it acts like it's not responding to audio controls, I make sure to unplug headphones and turn the speaker volume down to nothing before I walk into work in the morning. Similarly stupid response when unplugging headphones: 1) Oh, he unplugged the headphones! 2) Let's play that audio out of the speakers instead! At whatever volume the speakers happened to be at last time he used them! 3) And keep playing it for a half second or so! 4) I guess he might have wanted it to stop when the headphones came out? Fiiiiine.
mikecb 11y ago
Given that the android tv devices are (imo) more important in this aspect than a phone or tablet, I would have liked to see what latency was on things like the nexus player, shield tv, sony tv (baked in), apple tv, and roku maybe. I assume they would be similar, but perhaps they made some special modifications that will later be incorporated into AOSP?
- WalterSear 11y ago
  I don't understand why you think that round trip latency matters at all on tvs and media consumption devices.
  - mikecb 11y ago
    I misunderstood that the issue was input-to-output latency, and was instead thinking of audio/visual sync issues.
- valdiorn 11y ago
  wrong. Latency matters when you're playing/interacting with the device. watching TV is a passive activity with no user input, latency does not matter in that case.
  - mikecb 11y ago
    Ahh, I see. They are speaking of a specific type of latency.
yeureka 11y ago
I have been writing a game on my spare time that has an audio sequencer for interactive music. It currently runs on iOS and OSX. Initially I thought about doing an Android version as the code is all in C++ and it would be easy to port, but if the latency is as bad as described here I might not bother.
vanous 11y ago
Direct link for the superpowered.com latency testing app: http://superpowered.com/latency http://superpowered.com/latency
SCdF 11y ago
> The fact of the matter is that sound (along with everything else we know of in the Universe) travels at a finite speed. In regular conditions, a sound wave propagates at around 340 meters (or 1100 feet) through the air in a single second. This means that even for plain old physical instruments, there exists an audio delay between the moment the instrument is played and the instant the sound reaches the musician's ear. You heard it here first folks, in a few years Apple is going to come out with the revolutionary "Cochlea sound", which has such a small delay your ears are unable to perceive that there is any delay at all (while listening from 10-12 inches from your ears of course)! It will be magical and delight us, and something we never realised we needed until we were told we did. It will become the new standard and a household name, and we'll all have to catch ourselves when we say it and try to use some obnoxious generic term instead (LoALPS perhaps: Low Audio Latency Per Sound).
deleted 11y ago
[deleted]
ZenoArrow 11y ago
IMO the biggest reason to be optimistic that this will get this fixed is VR. GearVR had to rely on its own audio stack, but if VR takes off in a big way then I'm sure Google will want to support in stock Android, and that'll only work if they re-architect the Android audio stack to support low latency audio. This could end up fixing the issue for desktop Linux too.
ericmo 11y ago
> (from the article) [...] It did unquestionably require more time to code in lots of low-level functionalities that were simply missing from Android's SDK — and even after all that work, the end result was still far from ideal. That being said, it's not clear whether the improved latency on newer platforms can be achieve without using Android NDK. I haven't used it, but I know Android 4.0+ supports OpenSL through NDK, and I'm guessing that if you want low latency you need OpenSL. By skimming through the docs, it looks like that's the case, isn't that so? http://source.android.com/devices/audio/latency_app.html http://source.android.com/devices/audio/latency_app.html So, maybe it does solve the latency issue, and I would gladly use NDK because I like C++ stuff, but I think most developers wouldn't want to do it.
SwellJoe 11y ago
The overall latency (not audio, but the responsiveness, in general) of my Nexus 7 2013 got much worse in Marshmallow. It now takes a second or two after clicking anything to get a response. I'm not sure what happened, but battery life also dropped immediately after the OTA update to 6.0. I don't have a lot of confidence that audio latency has improved, given the overall increased sluggishness of the device. I have a few music apps (Caustic, several DJ things, a couple of classic style trackers), but haven't tried them lately...partly because latency has always been such an issue that it wasn't worth trying to do stuff on it. Nonetheless, I avoid Apple products on principle, and also enjoy playing with audio stuff (so much so that I started an audiovisual company a few years ago, which I only recently extricated myself from, leaving it in the hands of one of the co-founders), so I'm pleased to see the issue getting attention. I'd love to be able to make real music on my tablet. I have a Gameboy that I use for "on the train" music tinkering, because it has an immediacy that is lost in most modern devices (not merely latency...just the general composition process on something so simple it's impossible to get bogged down in bullshit like selecting the perfect snare sample or something). It's be cool to be able use the Nexus 7 for that purpose.
- 6stringmerc 11y ago
  Uh, while I think it's perfectly reasonable to like using a Gameboy for music tinkering, that whole "immediacy" thing you cite about composition process is essentially what got solved with Propellerhead's Figure. I've had it since the first version and it's grown into my personal, portable drum/bass/synth sketchpad and recommend it to anybody serious about composing on the go. Just because you don't know what's out there doesn't mean it's not out there, for what it's worth.
  - SwellJoe 11y ago
    Figure is an iPhone/iPad app. There is no Android version.
    - 6stringmerc 11y ago
      I know, which means that until Android gets its garbage latency issues sorted out, having principles a la "No iDevices" simply cuts off the nose to spite the face.
coupdejarnac 11y ago
I have a virtual hearing aid app for iPhone(Enhanced Ears) that simply is not possible right now with Android. I've had potential customers clamoring for an Android port, but I don't want to develop an app that has crippled performance. The audio coming in to the user's ears needs to match moving lips.
Polyphonie 11y ago
You can watch this I/O session from a few years back: https://youtu.be/d3kfEeMZ65c https://youtu.be/d3kfEeMZ65c It probably provides the best insight to the problem Google engineers were and probably is still facing in regards to high performance audio on Android. Even when the latency is brought down, there's still other audio related issues/glitches (like buffer underruns). Even now that Android is measurably better than it was previously, things like buffer underruns can still affect the overall usability. Increasing the buffer (to fix the underruns) render the device less suitable for such things as using it with external MIDI controller. You can see from Superpowered's audio latency measurements: http://superpowered.com/latency#cta http://superpowered.com/latency#cta that the buffer for iOS is half that of the best Android (the dual-core Nexus 9)- 64 instead of 128 for Android. I
kabdib 11y ago
A few years ago I got into a project that involved pretty tight control of audio latency, in a system with a fair amount of legacy. "How hard can it be?" At the end of a year, imagine someone who has spent a year in the Alaska wilderness, strangled three grizzly bears with his bear hands, lived on berries and nuts and fish caught in frigid glacial streams, and generally been up and down the audio stack in the OS and a couple pieces of firmware, involving audios APIs, a USB software stack that we re-wrote, a close examination of USB hardware and how chipsets are fucked up, and how exactly I2C microphones really work. My wife described me as "broken". It took a while to recover. How hard can it be? Isochronous, low-latency audio can be pretty hard.

102 points by kawera 11y ago

Hi all -- Patrick from Superpowered.com. We're working on an update to our Android 10 ms Problem article with new data --- quick question: What's not clear about Android latency? What questions would you like answered that you haven't seen answered online?

wawi 11y ago

It is really an interesting aspect of the whole Android versus iOS "war", that indeed Google did not up-front plan to make the OS actually useful for realtime Audio/synthesis/processing; but instead, Android has had more of a 'data-center'-like design, wherein the kernel and its onboard assets is more of a 'managed host' than it is 'realtime embedded' sort of system .. That the Android designers are having to play catchup with this speaks a lot, I think, of how significant design-decisions were influenced by the territory from which the originated - the web, managed systems - and the realm which they hoped to capture - the embedded system in your pocket. Android was not designed up-front to be an amazing media system; iOS, was. Thus, iOS has more synth-plugins and a vibrant, thriving Audio-Application eco-system (seriously, its million-dollar) whereas Android has a lot of new players, but so far: not such a great playground, really, for "Musicians". In fact, while iOS has really impinged upon the Music-Instrument (Digital) industry, Android has yet to make a dent. So it is an interesting time to be having new solutions to what have been, relatively, frustrating and troublesome aspects of Android realtime Audio, as a developer.

agumonkey 11y ago
Was iOS evolved from the iPod OS ? a lot ? a little ?
- on_and_off 11y ago
  I have no idea of the internals of the iPod. As I understand it, iOS benefited immensely from the work already done on OsX. I don't even know whether they planned for it or just benefited from a mature module already in place.
  - sirkneeland 11y ago
    IIRC the very first iPods actually had licensed much of the under the hood software. I could be confusing them with Palm OS, which did the same for the kernel of their first Pilots...
- milkytron 11y ago
  iOS is just a rebranding of what used to be iPhone OS.
- S_A_P 11y ago
  No- iOS is a stripped down OSX. iPod classic os was not related to that. The Core Audio framework is what makes iOS low latency. It was developed after the iPod.
6stringmerc 11y ago
Yeah right out of the gate iOS was the clear leader, but I think that's the legacy of Core Audio from their main OS? To me it's not surprising that iOS was the go-to for audio, based on historical prescedent that both MS and, from what I understand, Linux platforms didn't really have strong native audio processing for low latency. I mean, yes, it was totally do-able (MS via ASIO4ALL, I know there were a few Linux based music systems even that ran on Atom netbooks), but plug-and-play wasn't quite the same. Even audio cards had some driver issues in the MS realm, speaking from personal experience. It's rather sad because the MS environment allowed for a lot of homebrew plugins - the VSTs. Those won't run in Apple's ecosystem. Other plugins do (ex. UAD) but for the longest time those required (still do?) their own hardware processing cards. $$$$$$$. iOS bridged the gap in a lot of ways between the 'average' and 'pro' musician being able to access nice digital sounds on reasonably priced hardware. I'm a genuine PC guy (Ableton Live, Reaper) but get a lot of mileage out of a second gen iPad 2 - specifically Propellerhead's Figure software, which is flat out fantastic. I've made an entire EP in GarageBand while traveling. Sure, some devices won't work with iOS devices (my LPK25 will run on the iPad 2's port via camera connector USB dongle, but the MPD8 won't), and yet some are outstanding (looking at the Line6 / iK media products for guitarists and amp simulation). There's so much of a head start with iOS that honestly I kind of think there's little to no point in Android or MS trying to play catch up in the mobile sphere. MS has a winner with the Surface line because they run real software, but in the 'pure' mobile space, iOS has all the goods in my opinion.
- sirkneeland 11y ago
  Given the eye-watering disparity in gross revenue generated from iOS vs OS X, you really have to wonder which one is their "main OS" at this point...
  - 6stringmerc 11y ago
    Oh no doubt, I probably should have termed it 'legacy' OS considering the priorities these past years. After the 'prosumer' pivot of Final Cut Pro that angered a vocal group who had spent tens of thousands on workcenter setups, to the 'single port' type design on the new laptop, I'm pretty sure a lot of the die-hard creative types aren't going to feel really comfortable they're being considered for future releases. I honestly don't think Apple cares, because those users are a minority anyway.
kllrnohj 11y ago
I would more say this was just an aspect that Google inhereted by going with Linux ( see also https://xkcd.com/619/ https://xkcd.com/619/ ) iOS inhereted a strong foundation for this as OSX has a history of strong support for this sort of content creation. Linux does not.
- wawi 11y ago
  I don't personally think this can be blamed on Linux - Audio works on Linux as well as OSX does, when the hardware is selected and configured, as it is on OSX. Linux has alternative and competing - and thriving - audio subsystems, and this means that systems integration has to pay attention to what its doing. This didn't happen in Androids' case; where other Linux distros had rock-solid and professional-quality audio subsystems and software integration, Android had an NIH implementation that ended up having to re-invent the wheel, to boot.
  - Zigurd 11y ago
    This is basically correct. Android replacing the Linux graphics and UI stack turned out to be the right thing. Multimedia, not so much.
  - kllrnohj 11y ago
    > where other Linux distros had rock-solid and professional-quality audio subsystems and software integration Such as? Remember we're talking Linux in the 2005-era (~2.6.10-ish), not Linux today. JACK was still fairly new at that time. ALSA was still mostly broken at the time. Most stuff still wanted OSS. Basic "does sound come out of the speakers?" was often broken, and god help you if you wanted two things to play audio at the same time. And you still need to solve the driver problem, which JACK, etc... don't help with at all.
    - simoncion 11y ago
      > And you still need to solve the driver problem... Um. If we're talking about 2005, we're talking about Android. If we're talking about Android, we're talking about brand-spanking new hardware on a brand-spanking new phone. This means that the audio driver didn't exist, which means that it was being written from scratch. This makes the "driver problem" a non-issue. > Remember we're talking Linux in the 2005-era (~2.6.10-ish), not Linux today. ... ALSA was still mostly broken at the time. Most stuff still wanted OSS. I never tried to do any "pro audio", [0] but that's not how I remember it. ALSA worked just fine. Anything that wanted OSS worked well enough with ALSA's OSS-compat API. I can't remember if the software mixer was around or not at the time, but I remember that when it did arrive, it eliminated that problem with single-stream sound cards. [0] But -as I remember it- most folks doing pro audio used specialized, standalone hardware for it back then.
pjmlp 11y ago
You also see that in the effort that Apple makes for providing game development tools and libraries, whereas Google doesn't even provide proper editor support for shaders.
Zigurd 11y ago
You are reading a lot into it. Yes, Android's designers had a very different mentality from the traditional RTOS-for-a-handset designers. But that does not mean they had a "data center' mindset. Indeed one of the most common error or people who do have a "data center mindset" is that they load up on libraries that do various useful things for their Android development and then blow out the method and heap limits. Understanding Android requires a unique design and implementation mentality that is neither RTOS oriented, nor "small *nix in a phone" oriented, nor server Java oriented. It's a clever multi-processing managed language runtime with aggressive memory recovery that's capable of running many managed runtime processes, none of which can hog memory globally. Without having been designed for real time audio from the beginning, retrofitting that capability was hard to do. Apple made a lot of hay out of lack of real time audio processing. But only in one domain of apps. These are the kinds of trade-offs that get products out the door.

exDM69 11y ago

Given the title, I would have liked to see some in-depth analysis on why the latency is as bad as it is? Which parts of the software stack are causing it? Even though the Linux kernel is less than ideal for audio, it can still provide a latency low enough for music, ie. 10 ms or less. This is an unbearable situation. There are lots of interesting music apps on the iOS that I would like to use, but only a handful are available on Android. And even if they are available, they are not necessarily usable.

rogerbinns 11y ago
> Which parts of the software stack are causing it? There is a diagram lower down in the article with the components (ALSA, audioflinger etc). http://www.androidpolice.com/wp-content/uploads/2015/11/nexus2cee_Android-Audio-Path-Latency-Superpowered-Audio700px.gif http://www.androidpolice.com/wp-content/uploads/2015/11/nexu...
- dgregd 11y ago
  Actually Android standard doesn't include ALSA. There is a thin sound HAL above ALSA, OSS and other Linux sound technologies. Of course many phone vendors use ALSA to implement Android HAL layer. It seems that Samsung uses its own sound drivers and therefore they have small audio latency compared to other Android vendors.
- exDM69 11y ago
  The infographic looked so much like an ad that I skipped over it. So this is one of the "good" Android audio stacks, getting 36ms latency. But half of that is "wasted" in user space, in AudioFlinger as well as the app ring buffer (which seems pretty big to me). I'd like to see the comparison to a "bad" Android device. Doing some internet searches, I found someone who had plugged in PulseAudio to an Android phone and got 20 ms latency instead of the 170+ ms latency from AudioFlinger on the same device [0]. That's pretty huge. Ideally, there should be no user space middleware component processing audio in between, this could half the latency (according to the diagram), bringing the latency to acceptable levels. But ALSA isn't really intended for routing audio on the fly conveniently (e.g. when plugging/unplugging headphones), so there is some room for improvement too. Overall, the situation is pretty sad. Linux can do better than that but a modern multimedia device (desktop or mobile) has several audio outputs and hot-plugging, which makes the situation rather difficult using ALSA only. And the userspace audio components just aren't great. http://arunraghavan.net/2012/01/pulseaudio-vs-audioflinger-fight/ http://arunraghavan.net/2012/01/pulseaudio-vs-audioflinger-f...

on_and_off 11y ago

If you want an in-depth explanation (AP mostly skims the surface) : https://www.youtube.com/watch?v=d3kfEeMZ65c https://www.youtube.com/watch?v=d3kfEeMZ65c is a must-watch. Obviously, it does not cover the work accomplished these past two years, but it gives a good idea of the challenges of audio latency on Android

deleted 11y ago

[deleted]

rcthompson 11y ago

OK, what about the absurd latency for responding to hardware buttons on headphones and the like? My experience across 3 different Android phones has been that when I press the button on my headphones to pause whatever I'm listening to, it will respond about half the time, and when it does respond, the latency is randomly somewhere between 1 second and ten minutes. This is consistent across several sets of headphones, both wired and Bluetooth, and even true of the pause button that appears on the lock screen, which I'm assuming uses the same mechanism to signal the app.

ultramancool 11y ago
Fuck, what is up with this? I don't care about the audio latency this article is talking about and I never used to have this problem either. But recently, maybe just since I flashed latest CyanogenMod my phone sometimes responds instantly, but sometimes doesn't respond to the headphone button press until I press the power button or wait 30s+. It's incredibly annoying. I'm using CM on a OnePlus One with Xiaomi Piston 3 headphones. Considering the power button works, I suspect this may have something to do with sleep states?
untog 11y ago
Yep, this is beyond frustrating. Especially as on numerous occasions I've hit the earphone pause button, had it do nothing, pull out my phone, hit pause, unplugged my earphones and THEN the signal finally arrives, UNpausing my music for all to hear.
- wlesieutre 11y ago
  Same here. Whenever it acts like it's not responding to audio controls, I make sure to unplug headphones and turn the speaker volume down to nothing before I walk into work in the morning. Similarly stupid response when unplugging headphones: 1) Oh, he unplugged the headphones! 2) Let's play that audio out of the speakers instead! At whatever volume the speakers happened to be at last time he used them! 3) And keep playing it for a half second or so! 4) I guess he might have wanted it to stop when the headphones came out? Fiiiiine.

mikecb 11y ago

Given that the android tv devices are (imo) more important in this aspect than a phone or tablet, I would have liked to see what latency was on things like the nexus player, shield tv, sony tv (baked in), apple tv, and roku maybe. I assume they would be similar, but perhaps they made some special modifications that will later be incorporated into AOSP?

WalterSear 11y ago
I don't understand why you think that round trip latency matters at all on tvs and media consumption devices.
- mikecb 11y ago
  I misunderstood that the issue was input-to-output latency, and was instead thinking of audio/visual sync issues.
valdiorn 11y ago
wrong. Latency matters when you're playing/interacting with the device. watching TV is a passive activity with no user input, latency does not matter in that case.
- mikecb 11y ago
  Ahh, I see. They are speaking of a specific type of latency.

yeureka 11y ago

I have been writing a game on my spare time that has an audio sequencer for interactive music. It currently runs on iOS and OSX. Initially I thought about doing an Android version as the code is all in C++ and it would be easy to port, but if the latency is as bad as described here I might not bother.

vanous 11y ago

Direct link for the superpowered.com latency testing app: http://superpowered.com/latency http://superpowered.com/latency

SCdF 11y ago

> The fact of the matter is that sound (along with everything else we know of in the Universe) travels at a finite speed. In regular conditions, a sound wave propagates at around 340 meters (or 1100 feet) through the air in a single second. This means that even for plain old physical instruments, there exists an audio delay between the moment the instrument is played and the instant the sound reaches the musician's ear. You heard it here first folks, in a few years Apple is going to come out with the revolutionary "Cochlea sound", which has such a small delay your ears are unable to perceive that there is any delay at all (while listening from 10-12 inches from your ears of course)! It will be magical and delight us, and something we never realised we needed until we were told we did. It will become the new standard and a household name, and we'll all have to catch ourselves when we say it and try to use some obnoxious generic term instead (LoALPS perhaps: Low Audio Latency Per Sound).

deleted 11y ago

[deleted]

ZenoArrow 11y ago

IMO the biggest reason to be optimistic that this will get this fixed is VR. GearVR had to rely on its own audio stack, but if VR takes off in a big way then I'm sure Google will want to support in stock Android, and that'll only work if they re-architect the Android audio stack to support low latency audio. This could end up fixing the issue for desktop Linux too.

ericmo 11y ago

> (from the article) [...] It did unquestionably require more time to code in lots of low-level functionalities that were simply missing from Android's SDK — and even after all that work, the end result was still far from ideal. That being said, it's not clear whether the improved latency on newer platforms can be achieve without using Android NDK. I haven't used it, but I know Android 4.0+ supports OpenSL through NDK, and I'm guessing that if you want low latency you need OpenSL. By skimming through the docs, it looks like that's the case, isn't that so? http://source.android.com/devices/audio/latency_app.html http://source.android.com/devices/audio/latency_app.html So, maybe it does solve the latency issue, and I would gladly use NDK because I like C++ stuff, but I think most developers wouldn't want to do it.

SwellJoe 11y ago

The overall latency (not audio, but the responsiveness, in general) of my Nexus 7 2013 got much worse in Marshmallow. It now takes a second or two after clicking anything to get a response. I'm not sure what happened, but battery life also dropped immediately after the OTA update to 6.0. I don't have a lot of confidence that audio latency has improved, given the overall increased sluggishness of the device. I have a few music apps (Caustic, several DJ things, a couple of classic style trackers), but haven't tried them lately...partly because latency has always been such an issue that it wasn't worth trying to do stuff on it. Nonetheless, I avoid Apple products on principle, and also enjoy playing with audio stuff (so much so that I started an audiovisual company a few years ago, which I only recently extricated myself from, leaving it in the hands of one of the co-founders), so I'm pleased to see the issue getting attention. I'd love to be able to make real music on my tablet. I have a Gameboy that I use for "on the train" music tinkering, because it has an immediacy that is lost in most modern devices (not merely latency...just the general composition process on something so simple it's impossible to get bogged down in bullshit like selecting the perfect snare sample or something). It's be cool to be able use the Nexus 7 for that purpose.

6stringmerc 11y ago
Uh, while I think it's perfectly reasonable to like using a Gameboy for music tinkering, that whole "immediacy" thing you cite about composition process is essentially what got solved with Propellerhead's Figure. I've had it since the first version and it's grown into my personal, portable drum/bass/synth sketchpad and recommend it to anybody serious about composing on the go. Just because you don't know what's out there doesn't mean it's not out there, for what it's worth.
- SwellJoe 11y ago
  Figure is an iPhone/iPad app. There is no Android version.
  - 6stringmerc 11y ago
    I know, which means that until Android gets its garbage latency issues sorted out, having principles a la "No iDevices" simply cuts off the nose to spite the face.

coupdejarnac 11y ago

I have a virtual hearing aid app for iPhone(Enhanced Ears) that simply is not possible right now with Android. I've had potential customers clamoring for an Android port, but I don't want to develop an app that has crippled performance. The audio coming in to the user's ears needs to match moving lips.

Polyphonie 11y ago

You can watch this I/O session from a few years back: https://youtu.be/d3kfEeMZ65c https://youtu.be/d3kfEeMZ65c It probably provides the best insight to the problem Google engineers were and probably is still facing in regards to high performance audio on Android. Even when the latency is brought down, there's still other audio related issues/glitches (like buffer underruns). Even now that Android is measurably better than it was previously, things like buffer underruns can still affect the overall usability. Increasing the buffer (to fix the underruns) render the device less suitable for such things as using it with external MIDI controller. You can see from Superpowered's audio latency measurements: http://superpowered.com/latency#cta http://superpowered.com/latency#cta that the buffer for iOS is half that of the best Android (the dual-core Nexus 9)- 64 instead of 128 for Android. I

kabdib 11y ago

A few years ago I got into a project that involved pretty tight control of audio latency, in a system with a fair amount of legacy. "How hard can it be?" At the end of a year, imagine someone who has spent a year in the Alaska wilderness, strangled three grizzly bears with his bear hands, lived on berries and nuts and fish caught in frigid glacial streams, and generally been up and down the audio stack in the OS and a couple pieces of firmware, involving audios APIs, a USB software stack that we re-wrote, a close examination of USB hardware and how chipsets are fucked up, and how exactly I2C microphones really work. My wife described me as "broken". It took a while to recover. How hard can it be? Isochronous, low-latency audio can be pretty hard.