14 ms·
An update on Android's audio latency
- Jeff_Brown 6y agoThe figure of most interest to someone playing an electronic instrument is "tap-to-touch latency". The article indicates a minimum of 43 ms for that (28 round trip latency - 5 audio input latency + 20 touch latency). That's like ten times what you'd hope for.
- Drakim 6y agoEmulators of old retro games also care about audio latency, as the emulated audio is generated on the fly by a turning machine, which makes it very hard to match visuals to audio if there is inherent audio latency.
- TheRealSteel 6y agoI was very sad when I got a Galaxy Note (R.I.P. Note line) and Elite Beat Agents was unplayable due to audio lag. It makes sense, but it is a shame.
- Lucasoato 6y agoDamn, Elite Beat Agents was such a good game, I cried like a baby in the stage with You're My Inspiration.
- _flux 6y agoIf just matching is desired then you can just buffer video frames; all audio frameworks provide means to synchronize video and audio. Of course, it probably makes the experience just worse if the delay is silly long..
- Drakim 6y agoBy doing that you create input lag for the game.
- hnuser123456 6y agodeleted
- thomasahle 6y agoIn case of audio it doesn't really matter how fast the display updates though? Unless your numbers are for the digitizer and not the screen.
- fireattack 6y agoWe're talking about tap-to-audio-output latency though, so it has nothing to do with display part of the screen. And there is no "touch" on PC generally speaking.. input latency would be on mouse or keyboard. I think you're talking different thing from the topic.
- fistynuts 6y agoThe article specifically says that tap to tone latency is typically around 20ms.
- morsch 6y agoThe 20 ms tap-to-touch reference is from 2017, maybe this has gotten better in the past 4 years? But maybe not. And per the source, it's possible to get down to 10 ms round trip latency if you buy the right phone.
- fomine3 6y ago60Hz touch scan rate (16.67ms) was common in 2017, but now we can easily buy a phone with 240Hz(4.17ms) touch scan rate. It should be much improvement.
- AceJohnny2 6y agoUsually, "an update on [blank]" is euphemism for said product getting cancelled. More to the point, one of PulseAudio's claims was that they consumed less CPU than Android's audio pipeline. With the recent hyped PipeWire, I wonder how that compares to Android's Oboe.
- raphlinus 6y agoThat's actually the joke here, which got a chuckle out of me. The idea is that excessively large audio latency itself has gotten cancelled. I haven't played with it myself (it's been a few years since I was actively involved in Android audio latency), but from what I've seen, AAudio is pretty good, probably close to what the hardware is capable of. I'd very much encourage people to do empirical measurements against, for example, PipeWire (which also looks good). Do keep in mind that constraints are different on mobile, and things like power management can and do get in the way of down-to-the-wire latency.
- jancsika 6y ago> I'd very much encourage people to do empirical measurements against, for example, PipeWire (which also looks good). A rule of thumb from reading linux audio mailing lists but which probably works generally-- completely disregard any claim of measurement of latency that isn't prefixed. For example, the Google blog labels the Y-axis with "round-trip time in milliseconds." They're clearly and explicitly measuring round-trip latency. They then piggy-back on that well-understood concept to introduce the concept of "tap-to-tone latency" and show measurements for that. Almost to the case, the times I've read a back-and-forth conversation where the word "latency" is used unprefixed, the commenters might as well be describing the warmth of vinyl recordings. Often they are simply describing an integer they typed into a widget or config file. On several occasions I've witnessed users insert Jack between a single application and ALSA because "that's what the professionals use." These are knowledgeable users who want as little latency between the system and their ear as possible.
- cout 6y agoAre you saying there is still a use case for going straight to ALSA without something in the middle?
- Kuinox 6y ago"Popular smartphone" list only samsung smartphone.
- bitmapbrother 6y agoPerhaps you should read the article in full. It clearly says the most popular Android smartphones in 2021 are: Samsung Redmi Oppo Huawei Vivo Which probably account for the majority of Android smartphone sales.
- Kuinox 6y agoPerhaps you should read the article, "Latency of the most popular android phones in Jan 2017" list only 20 samsungs.
- bitmapbrother 6y agoWhy would you reference a 4 year old graph when the 2021 graph was right below with current latency measurements of the Most Popular Android Phones.
- wffurr 6y agoThat’s true for the 2017 chart and probably was true in 2017. The 2020 chart lists Huawei, Oppo, Xiaomi.
- Kuinox 6y agoIn 2017 huawei was huge (in popularity), even bigger than samsung.
- karlding 6y agoAndrew Huang recently made a YouTube video [0] that provides some insight from the perspective of a music producer and someone who recently created a music production app. It's not overly technical, but it highlights some of the challenges one might face. As the article says, there is room for improvement, so I guess it's good to see that the long term goal is 10ms round trip. [0] https://youtube.com/watch?v=-sPbcTcUmcI https://youtube.com/watch?v=-sPbcTcUmcI
- Naac 6y agoThat was a strange video. It seemed like he started talking about latency between Android and IOS devices, which makes sense given that this article talks about response time of just under 40 ms like you said. The article says this is "well within the range required for real-time applications" if you define real-time applications as non-pro-audio stuff. If you're playing an instrument and it takes 40 ms for you to hear back the sound, that's Not Good. But then the video takes a strange turn and starts talking about "Stability" of mac vs PC, and I had to turn it off. It made the argument that "I haven't seen any professional producers use anything except macs" as its main argument. It would have been nice to hear some more technical information like mentioning core-audio, or something else. Regarding the article itself. I'm excited that Android is target less than 20 ms round-trip latency. I'm hoping that this work can be brought back to the Linux Desktop. Messing around with Jack and dealing with xruns is something I wish I could stop doing.
- Daho0n 6y ago>"I haven't seen any professional producers use anything except macs" as its main argument. Yeah that is both totally irrelevant to the topic at hand and also a bit strange. I have only seen PCs in the pro studios I have used but that doesn't make me believe nothing else is used.
- smoldesu 6y agoSame here. I've even seen an increased number of Linux-based studios, oftentimes running Bitwig or Reaper. I don't doubt that audio latency can be a little high on some workflows, but I've honestly had a less laggy experience with Pulse than Coreaudio. Out of the box, Bitwig has a latency of ~8ms on my (Linux) machine, as opposed to 20ms on my Macbook.
- adamnemecek 6y ago28ms is still like twice as much than one would want. And accounting for fluctuations, it’s like four times as much.
- fireattack 6y agoI chuckled at the fact that their figures still have spell checker wavy underline :p
- donturner 6y agoDamn that spellchecker! :D Thanks for spotting, I'll fix it shortly.
- apecat 6y agoCan't wait for Roland, Yamaha et al. to build awful Android-based touchscreen UIs for all upscale music gear with built-in control dashboards
- nkozyra 6y agoVersus the bad custom linux flavors & UIs they currently use? I'll take it.
- apecat 6y agoYou have a point, but my main concern is just the inevitable internet-connected app ecosystem for expensive and cool devices that are going to have a security patch lifecycle of 18-36 months with an expected lifespan of 15 years for second and third owners
- AndrewDucker 6y agoWhat's the equivalent latency on an iPhone?
- leecb 6y agoThe most common results on this page indicate that it varies with buffer size, but could be 9-11 ms for iPhones. https://superpowered.com/latency https://superpowered.com/latency
- fireattack 6y agoI feel like their app (or method) is flawed in some way. It can't be that many phones have 300~400ms+ latency!?
- jefftk 6y agoThis is way better than it used to be (I measured half a second latency in 2012: https://www.jefftk.com/p/android-sound-synthesis-and-latency https://www.jefftk.com/p/android-sound-synthesis-and-latency) but it is still too high to let you use an Android device as a musical instrument. This is something iPhones have always been able to do, and one of my biggest disappointments with Android. (Disclosure: I work for Google, speaking only for myself)
- thomasahle 6y agoWhat are the equivalent numbers for iPhone? Touch to audio and round-trip times?
- jefftk 6y agohttps://superpowered.com/latency https://superpowered.com/latency has messy data, but it looks like typically single-digit latency.
- S_A_P 6y agoI think typical newish (2014+) devices have between 5-9ms latency. Having core audio from the start was a huge leg up for iOS for sure. I am mostly platform agnostic unless I am working with audio where I pretty much only use Apple products.
- breakfastduck 6y agoCore Audio is the most under appreciated part of the apple ecosystem. It is lightyears ahead of windows / linux audio drivers.
- kall 6y agoThis data [0] doesn't have the latest models, and it's meassuring latency internaly, not "touch-to-sound". The ones they tested average around 7-8ms. I've heard of <10ms as the benchmark for "instant" in audio. Edit: I've been thinking about this and a touch-to-tone latency below 10ms seems imposible, even with the 120hz digitizer on newer iPhones. I would be very interested in seeing some experimental data as well. [0] https://juce.com/discover/stories/Mobile%20performance%20index%20 https://juce.com/discover/stories/Mobile%20performance%20ind...
- AdmiralAsshat 6y agoToo little, too late, I'm afraid. The improvement is great, but I think the audio professionals jumped ship long ago to Apple, and they're not going back. I know of several music professionals plugging into their Macbooks or iPads as part of the pre-amp, but I don't know if any using Android. Google sat on this for way too long while Apple courted the creative professionals, and for any that have already made the upfront buy-in for Apple hardware, I don't see them taking the risk of switching ecosystems to save a few dollars for performance that still won't match what they're used to.
- deadmutex 6y agoPerhaps. Though I wonder if every musician afford Macbooks or iPads. These things are not cheap, especially if you consider earnings of non-US based musicians, etc. Disclaimer: Googler, but opinions are my own.
- ocdtrekkie 6y agoThis is where Apple's far superior product lifecycle comes in: Buying a four year old Apple product is affordable, still runs the current OS, unlike Androids that old, and has the audio performance down.
- zachruss92 6y agoThis is a valid point, but it’s hard to place the blame on Google/Android. I think the challenge is less of Android itself but a refusal of the OEMs to support newer OS versions. This makes business sense, but sucks for consumers. I think Google is doing a better job with their Pixel devices. I honestly hope they try to lead the Android industry more with them.
- micv 6y agoA four year old Android is years out of support to Googlew. You get two, maybe three, years from them at best and then security patches until they get bored.
- com2kid 6y agoIt'd be nice if something was done about the extra 100-200ms (!!) of latency that Bluetooth headsets can add[1]. I understand this is largely a problem from the manufacturer problem, but I'd love to see Google lead the way with some sort of certification program, or even make their own low cost BT chipset for 3rd party headphones to use, to improve the situation. Of course Android is already world's better than desktop, where 200ms is seemingly closer to average than an outlier. It is also kind of cool that perusing that chart, it seems headsets connected to Android have, on average, lower latency than when connected to iOS! https://www.rtings.com/headphones/tests/connectivity/bluetooth https://www.rtings.com/headphones/tests/connectivity/bluetoo...
- kevingadd 6y agoThere are low latency bluetooth protocols, if you sort by latency on rtings for aptx-LL you can see some headsets are down below 40ms. Of course, this relies on both the host and the headset having support for the protocol and being engineered well. SBC and aptX just aren't designed for low latency, and low latency is something that can't be hacked into every codec after the fact.
- summm 6y agoapt-x is designed for ransom extraction via "intellectual property" licensing.
- AndriyKunitsyn 6y agoHow is this different from any other licensing?
- com2kid 6y agoI'll argue that an device meant for phone calls (which is where bluetooth audio started out!) should aim for low latency by default. I also understand that the CPU/Latency/Memory trade offs made over a decade ago are not the same one's we'd make now. Doesn't change the fact that the same device can have a 3x latency difference between platforms with the same codec. Google went and developed VP9 as a royalty free codec in the AV space, they should do something similar for Bluetooth. Right now if I'm on a video call with someone and we both have BT headsets, it is very possible that there is a 500ms audio latency!
- mellosouls 6y agoOff topic, but would be nice if they could do something - presumably far simpler - about the lack of fine-grained volume control. It might be a niche case, but it's really frustrating using the various sleep-related audio apps at night with earphones and not being able to select a minimum volume that is both loud enough to hear but not so loud it keeps you awake (example: Audible with a sleep timer). I'm sure there are other uses for the finer control as well.
- kwijibob 6y agoTotally agree. Fine volume control is a pain.
- exikyut 6y agoThat sounds involved but 100% doable using synthesized volume limiting in-app. Have you tried pestering Audible? Would probably take a while, but it feels like the kind of thing that would be hard to "un-justify" once you started arguing for it.
- mellosouls 6y agoThat's a nice idea - there are apps that purport (unsuccessfully in my experience) to work around the issue using software though I don't know they use that approach. Obviously any major app supplier may be reductant to invest significant dev time to offer a workaround for an OS limitation (and Audible is just an example), but yes it's worth raising I suppose, thanks.
- exikyut 6y agoI had a quick poke around to see if it's possible to implement something like audio filtering on Android, and found just about exactly that question asked over at https://stackoverflow.com/questions/15385797/hook-simple-audio-filter-to-music-playing-under-android; https://stackoverflow.com/questions/15385797/hook-simple-aud... TL;DR no, for obvious (albeit annoying) security reasons. I'm not even sure if it would be straightforward with a rooted device either.
- 6y ago
- matchbok 6y ago5 years of work, and still bad compared to what iOS had 10 years ago. Impressive.
- Robotbeat 6y agoResponsiveness and low input-to-result latency was why I chose iPhone over android 10 years ago even though I prefer a more open environment like android. Sounds like things haven’t changed a whole lot on that front. Sometimes I wonder if these increased levels of abstraction and even digitization itself (or at least the Internet-ization of things) is a mistake for human-machine interface. Watching drone racing pilots use analog controls and analog video headsets in spite of the potato resolution because WiFi/digital/etc just add too much latency is kind of eye opening and makes you sort of reconsider the last couple decades. (Improvements have been made so digital isn’t so high latency and VR has served to address a lot of these latency concerns, but it’s still kind of interesting...)
- thefourthchime 6y agoWhat's funny is that this “open” environment of android now is nearly the opposite. With any android device openly spying on you while locking down all the open source apps you might want to install.
- idonotknowwhy 6y agoIt's spying on me, but I can still install whatever i want. Same as Windows 10.
- andrepd 6y agoWhat does this mean? I can choose not to install google services in my phone, and use an open app store like F-Droid. Not that it counts for much in the grand scheme of things, seeing as 99% of users will never install a custom rom.
- BoorishBears 6y agoI like how the second line of your comment answers the first.
- foobar33333 6y agoIt used to be possible to dip your toes in with light modifications while still keeping everything working. Now with verified boot chains, safety net and hardware backed verification, you either have everything fully locked down or you go full Richard Stallman and have everything foss.
- fullstop 6y agohttps://www.youtube.com/watch?v=PnDK17zP9BI https://www.youtube.com/watch?v=PnDK17zP9BI Here's a corny video from Google about audio latency and pops from several years ago.
- bla3 6y agoIt's nice that some things get faster sometimes. Let's celebrate that, instead of whining about all the things that don't get faster, or complaining how they aren't getting faster enough.
- throwaway888abc 6y agoDoes it make active noise canceling on the phone by application more possible ? Last time checked it was not possible due to high latency.
- google234123 6y agoHow does Apple compare?
- Blueskytech 6y agoIf android can get its audio latency act together there is some real opportunity here to steal market share from Apple. Since the release of the 2nd iPad Pro Apple has forced audio processing onto the slower low power CPU core, this means that for the last three years I can run all my iOS synth applications at 96kHZ with a low buffer sample size without any drop outs on my first gen iPad Pro and an iPad mini 3. What I cannot do is get similar performance on last year's iPad Pro. It is absolute insanity that Apple has no work around to direct audio processing to the high powered cores for apps made for musicians. Presumably at some point they will allow you to do this with a documented programing call but it has been three years of bad performance on synth applications due to the necessity to have high sample buffers (ie high latency).
- aeontech 6y agoIs there more details about this published somewhere? I’m very curious to learn more
- Tade0 6y agoI've noticed an increase in measured latencies, but didn't know about that slower CPU core thing. Sad to see Apple take such a direction - musicians appreciated having proper hardware/software and Apple definitely made money on all the paid content. Why let this arrangement fall by the wayside?
- layoutIfNeeded 6y agoThe 10x engineers have either left Apple or were reorganized to gimmick projects like SwiftUI.
- xenadu02 6y ago> Since the release of the 2nd iPad Pro Apple has forced audio processing onto the slower low power CPU core What makes you believe this?
- pjmlp 6y agoOn Android is even worse, specially if one gets an average model.
- mxmilkiib 6y agohttps://github.com/atsushieno/aap-lv2 https://github.com/atsushieno/aap-lv2 - LV2 plugin backend and ports to AAP
- Zhenya 6y agoIt's pretty wild that none of the popular phones from either charts are Pixels...
- The_rationalist 6y agoHow much does it compare to linux pulseaudio ?
- RedNifre 6y agoFor comparison, the Nintendo Switch cardboard piano works by having you press a cardboard key with a highly reflective sticker on the other end, to lift that sticker above a barrier, so it can be seen by the game controller's integrated IR camera, which streams the image over to the switch over bluetooth, which then runs some image analysis to figure out which key sticker just became visible and THEN plays a note and the latency is STILL so much lower than on Android, that you can actually play it like a musical instrument.
- fistynuts 6y agoDo you have any figures to corroborate that? Because it sounds incredibly unlikely.
- cma 6y agoThe wiimote has built in IR blob extraction from the sensor on device, so I think he is wrong unless Switch/joycon changed things. In some modes, which piano may use, it basically is only sending sparse location data of 4 brightest detected light blobs.
- flipcoder 6y agoThis has been a problem for years. I can't believe they're just now addressing it.
- userbinator 6y agoUp to now I've been referring to round-trip audio latency. Round-trip latency involves three components in the audio chain: audio input, audio processing and audio output. I read through the whole article and that's the most detail given about any testing methodology; I don't see any more description of how these numbers are being measured. My rule of thumb for computer speed measurements and efficiency has always been in terms of CPU instructions; the goal of 10ms, while seeming very tiny, is still roughly ten million instructions of a 1GHz CPU. The other numbers mentioned are proportionally more, which then makes one wonder where all the time is being spent. Relatedly, I've actually written audio drivers (HD audio for Windows 3.x/9x, long story but I won't digress...) and the output latency measured from the app writing a buffer to the signal appearing at the headphone jack is probably in the dozens to hundreds of microseconds range (thousands of CPU instructions, including virtualisation overhead) at most --- but those OSs have extremely thin abstraction layers, and the call stack between the app and writing to the hardware registers is very shallow. The fact that newer hardware/software seems to have more audio latency reminds me of this: https://danluu.com/input-lag/ https://danluu.com/input-lag/
- klodolph 6y ago> I read through the whole article and that's the most detail given about any testing methodology; I don't see any more description of how these numbers are being measured. The details usually aren’t that interesting. What you can do is record an input signal, pass the signal through to the output, and wire the output back in to another input. You end up with two copies of the input signal, one delayed by the total round-trip time relative to the other. It’s fairly easy.
- userbinator 6y agoIt's certainly interesting because they can have a huge effect on the measurement results. For example, with HDA and I suspect other audio codec systems, what you described can be done with essentially 0 delay by configuring the codec to do the in/out mix itself. If you are measuring output from application buffers to input to application buffers, then how big they are and at what sample rates and bit depths will also affect the latency you see, etc.
- snvzz 6y ago40ms is way too much. To put it in perspective, at 60Hz, that's almost 3 frames of video. 10ms would start to be useful for music, but I wouldn't call it good until it was below 2ms, which is what I run my jackd setup (which requires linux-rt) at. I'm afraid we won't see good latency until Android moves away from Linux. The good news is that this is bound to happen, sooner or latter, as Fuchsia exists.
- mhh__ 6y agoIs 10ms really the bounds of usefulness - especially on a phone?
- bsder 6y agoSpeed of sound is one foot per millisecond. 40ms is 40 feet. Bands play with that kind of separation all the time.
- jjcon 6y agoThere is a huge difference between the latency you need to feasibly pull of a live performance - and the latency you need for high quality audio production. Why you would be considering doing that on an android though I'm not sure. Though I guess it may be a reinforcing problem, there isn't an ecosystem because the tech isn't there.
- snvzz 6y ago>Bands play with that kind of separation all the time. Shortcutting this latency is one of many reasons musicians wear earplugs on stage.
- anaisbetts 6y agoThe latency that matters is the discrepancy between what you feel with your fingers and what you hear. Your finger nerves respond extremely quickly, and if what you hear is delayed from what you feel by more than ~10ms, it is extremely jarring and basically unusable
- bambax 6y agoIs there a way to know the audio latency of a specific device before buying it? Is this data published somewhere?
- baybal2 6y ago> The average latency of the most popular Android phones has dropped to under 40ms, which is well within the range required for real-time applications. Uh.... It needs to be at least 10 times faster to claim realtime use
- bondolo 6y agoIt is welcome to hear that Google is working on this because sound latency is critical for visually impaired users using text to speech on their phones. The TTS latency is on top of the sound latency and can be significant, to the point where the UI is unresponsive or falls behind. Any improvements they can make will be greatly appreciated.