6 ms·

The Care and Feeding of the Android GPU

57 points by buymorechuck 16y ago

wccrawford 16y ago
This isn't the first thing I've seen that Google devs really just didn't get. I think it's from the rapid expansion that Google has done. They get people working on things that aren't really their area of expertise because they wanted to try something in their 20% time. In this case, it's obvious that we've got server people working on mobile code. Future hardware will resolve these inefficiencies? UGH. That's definitely someone from the server world speaking.
- tgflynn 16y ago
  I'm talking out of my hat here a bit because I haven't actually worked with Android yet but it sounds to me that these problems might be related to the choice of Java as the core programming environment. I doubt that was a decision made by somebody working on a 20% project.
  - buymorechuck 16y ago
    Although Java has its problems, I think the core of Android's issue is a bad utilization of computing resources. One can only hope that Google will eventually fix this problem. Palm's WebOS put much of its core logic in JavaScript, which had a slower implementation (at the time) than Dalvik VM does. However, Palm carefully made sure to take advantage of the GPU to achieve many of the effects in the UI.
- jrockway 16y ago
  But the thing is, nobody actually cares about this. I have used an Android phone since the beginning. Touch responsiveness is not something I have ever noticed. Any delay in rendering web pages is due to the network. I think people that write articles about touch responsiveness or animation latency are suffering from the "benchmarking hello world" problem. Yeah, you could speed that stuff up. But 99% of the time, the phone is waiting for data on a socket, which is not going to get fast any time soon.
  - buymorechuck 16y ago
    Perhaps, but for me, it's absolutely about usability. Whenever I see new Android phones or builds, the first thing I try out is how responsive the scrolling or zooming is. Often times, I'm waiting for Android to respond to my touch, other times my touches are missed completely. These issues cause unpredictability for me and make the phone harder to use. As phones got faster, the situation improved, until a larger screen was added, which brought things back down again as the CPU burns to keep up. The milliseconds latency here and the stuttering FPS there decreases the usability significantly in my opinion. I think that there is a group of users who don't notice these latency and animation issues. But I do, and I think many users still unconsciously attribute their dislike for Android to these issues. (example: my brother in law, used several Android Froyo phones extensively before settling on a BlackBerry because it felt "faster".)
    - jrockway 16y ago
      I have never noticed any latency. I blame the manufacturer-specific Android "improvements", as I use plain AOSP and don't have any trouble. My EVO 4G gets about 24 hours battery life, with moderate usage during the day. (But again, this is because I don't use HTC's build. With their build, the battery life is barely long enough to boot the phone.)
      - potatolicious 16y ago
        I've used vanilla Froyo on the Nexus One. It's dog slow. I don't think this problem is isolated to OEM-specific Android modifications (though they certainly don't help). Responsiveness and framerate-wise, even a simple cursory examination will tell you: WP7 and iOS run laps around poor old Android. I got to play with a Nexus S recently, and the responsiveness is greatly improved over Froyo on N1... but is still noticeably less smooth than either Apple or MS's platforms. It's an odd day when MS's UX is more polished than somebody else's... One area where Android's lacklustre performance really hurts usability is diving into menus. iPhone came up with the novel idea of sliding menus - a simple animation that communicates a lot of context to the user (A belongs under B triggers modal window C, all based solely on the direction of the swipe animation). On Froyo+N1 this animation is jerky enough, frequently enough, that this relationship can easily be missed, resulting in more user confusion and worse UI comprehension overall. Google needs to realize that we are in the year 2011, not 1995. "UI design" is no longer about placing the right buttons in the right places, and animation is often now the defining line between an easily comprehensible user experience and an obtuse one. IMHO graphics/animation performance is the #1 obstacle in Android's way, and the main advantage everyone else seems to have on it (and it's a huge advantage). When I use a 2.2 phone, my impression is one of annoyance. I don't know if my touches will be detected, and even if they are, I don't know how long I'll have to wait for the phone to respond. Heck, I enjoy using WP7 phones more than Android because of the performance, despite the fact that WP7 is significantly less featureful than Android! Navigating WP7 is a pleasure, and it never feels like the phone stands in your way. I have yet to get this feeling from Android.
- rektide 16y ago
  i'd throw ANDROID-1147 and ANDROID-1376 in the Android technical management "absolutely really just doesn't get it" bucket. otoh, the issue Charles Ying of Satine.org rages aobut, ANDROID-6914, seems much more questionable to me because the issue itself is really complex. the two tickets, "Support for external keyboard, mouse" and "Browser doesn't render SVG images" both strike me as absolutely essential to making smartphones competitive with PC's. people have been happily attaching fold up keyboards to Palm Pilots for a decade now, SVG is perhaps the only sane route forwards for resolution independent apps and animation, and Apple has found religion on both these topics-- yet Android is obstinately behind. otoh composited gpu acceleration is not nearly the huge obvious win C. Ying boasts it as. the gpu is more power efficient than the cpu for intensive applications, but a smartphones natural state is not intensive use. mr guy of android points out a number of complexities-- opengl drivers only supporting a single context, opengl acceleration proving less speedy than software implementations. it places huge emphasis on driver support, which any embedded wonk will tell you is a scary prospect. although gpu acceleration is becoming better supported, keep in mind that the smartphone market expands, both up and down, with broadcom & others putting out chips designed for sub $75 smartphones. android is definitely not winning the race for the top atm, but performance has always been a rather lonely market-- that said, unsmooth experiences dont fly. software graphics focus insures a base level of performance for everyone, and strikes me as the right place to start to insure a speedy experience not subject to external interrupts. the current silkiness situation is a little sad. it'll get better. more relevant to me is Android not playing in the ecosystem-- external peripherials, UPnP/DLNA, SVG-- these are the real things that will make or break not Android, but the open digital web, and Android is failing, badly, even in compare with non-web-enthusiasts Apple.
- tarandeep 16y ago
  Have you guys tried MIUI on Froyo ? It gives me 60FPS for scrolling, panning and almost everything on a Nexus One: http://www.youtube.com/watch?v=Or-9T44Bt7M http://www.youtube.com/watch?v=Or-9T44Bt7M It has got NO GPU acceleration. That PWNs it
wzdd 16y ago
Apparently touch responsiveness is improved in 2.3. However this article doesn't really explain what's wrong with it. I've been doing latency benchmarks for touch recently, and the highest latency I've seen for touch delivery is 10ms. Now 10ms is pretty terrible for what should be a very straightforward thing, but on the other hand it's only 2/3 of a frame, at 60 FPS. The argument in favour of a GPU-accellerated GUI which is not mentioned here is battery life. I haven't been testing this side of things, but just because a CPU can do a 60 FPS GUI doesn't mean it should. Deferring it to the GPU would certainly reduce power consumption while navigating a UI. Whether this is non-negligible in terms of overall battery life, I'm not sure.
- tgflynn 16y ago
  Are those 10ms with a realistic app/usage scenario ? I wouldn't think users would complain about 10ms, isn't that below the threshold of perception ?
  - wtallis 16y ago
    A stutter 2/3 of a frame long means there's a 2/3 chance that you'll draw a frame that doesn't match the current finger position, followed by a frame that catches up more than it should have had to. The ~40ms input lag shown by most desktop IPS LCD monitors is noticeable even though it falls below the reaction time. On a touch display, it should be even easier to notice input lag, because you can see your finger moving and the screen around it not keeping up. 10ms sensor lag may be below the threshold of perception for circumstances like this, but probably not by much. If you add even a few milliseconds of processing lag after your app receives the touch event, you'll be behind by a frame.
  - pvaibhav 16y ago
    I work with digital audio/MIDI and a latency of 15-20 ms between pressing a key and hearing the sound is easily audible making it near impossible to play expressive music. 10ms can be felt and is 'annoying', under that is acceptable. I usually keep latency down to 5ms, at most 7ms, only letting it increase, e.g. during mixing when responsiveness is not utterly essential.
- 16y ago
yoden 16y ago
I think the android bug linked in the article is much more informative than the article itself. Basically: 1) Android phones can already animate the screen @ 60 fps in software in most cases. 2) When they can't, it's not because the CPU is too slow, it's because there is other stuff happening (GC, etc.). A GPU can't fix that.
- buymorechuck 16y ago
  The GPU can offload most of the graphics intensive tasks from the CPU, utilizing the GPU instead of letting it sit idle. freeing the CPU to focus on GC tasks. In addition, most of the animation work done with Dalvik code can be moved into a dedicated thread / task with highly optimized code backing it.
comex 16y ago
Anyone who thinks Android's animation is "fast enough" is either using a faster device than I have ever tried, or has never tried to compare, say, scrolling lists versus the iPhone. I'm not saying the GPU is the answer, and maybe 2.3 is the answer, but ignoring the problem is not. (Edit: So why does a GC pause stop animation, and not just by taking up CPU time, since moving to the GPU would help in that case? Is the animation code itself written in Java and waiting on the GC? If so, I bet rewriting it in C would be a massive help. Also, yuck.)
- eru 16y ago
  Or use a GC that doesn't need to stop the world. (Or at least only does so for very small intervals.)
- ulrich 16y ago
  Gingerbread has a concurrent GC, so animation should not stop completely. However, on a single core system I wouldn't expect this to have a huge impact. But this is just my personal guess.
dave1619 16y ago
I think the author nails it. Without GPU acceleration Android really is a notch below iOS in terms of UX. In my opinion, even WebOS and Windows 7 have much smoother scrolling and touch response than Android. But if Google wakes up and adds GPU acceleration and better eye-candy, watch out Apple.
asdasdrasdasdr 16y ago
What is the main performance difference between Vista SP1 and Windows 7? Hardware accelerated UI.
simonh 16y ago
Purely by chance, I was trying out a galaxy tab at a store today. I loaded up the Financial Times web site, which took an age because the store wifi was crap and tried scrolling the web page and it stuttered quite a bit at first then smoothed up, zooming as juddery as well. Did the same thing on an ipad - slow loading about the same, but the instant the page was rendered it scrolled and zoomed perfectly. Also the widget for editing the web page URL on the Tab was a horrible usability car crash. What's up with that? Disclosure - iPhone user, but Android curious.
typo_pl 16y ago
Android's VM can improve perf-wise - see http://blogs.oracle.com/javaseembedded/2010/11/how_does_android_22s_performance_stack_up_against_java_se_embedded.html http://blogs.oracle.com/javaseembedded/2010/11/how_does_andr...
michaelward82 16y ago
It doesn't matter what the solution is, but Google need to prioritise UI responsiveness above all else. Until that happens, the UI on Android will suck.
deleted 16y ago
[deleted]

57 points by buymorechuck 16y ago

This isn't the first thing I've seen that Google devs really just didn't get. I think it's from the rapid expansion that Google has done. They get people working on things that aren't really their area of expertise because they wanted to try something in their 20% time. In this case, it's obvious that we've got server people working on mobile code. Future hardware will resolve these inefficiencies? UGH. That's definitely someone from the server world speaking.

tgflynn 16y ago
I'm talking out of my hat here a bit because I haven't actually worked with Android yet but it sounds to me that these problems might be related to the choice of Java as the core programming environment. I doubt that was a decision made by somebody working on a 20% project.
- buymorechuck 16y ago
  Although Java has its problems, I think the core of Android's issue is a bad utilization of computing resources. One can only hope that Google will eventually fix this problem. Palm's WebOS put much of its core logic in JavaScript, which had a slower implementation (at the time) than Dalvik VM does. However, Palm carefully made sure to take advantage of the GPU to achieve many of the effects in the UI.
jrockway 16y ago
But the thing is, nobody actually cares about this. I have used an Android phone since the beginning. Touch responsiveness is not something I have ever noticed. Any delay in rendering web pages is due to the network. I think people that write articles about touch responsiveness or animation latency are suffering from the "benchmarking hello world" problem. Yeah, you could speed that stuff up. But 99% of the time, the phone is waiting for data on a socket, which is not going to get fast any time soon.
- buymorechuck 16y ago
  Perhaps, but for me, it's absolutely about usability. Whenever I see new Android phones or builds, the first thing I try out is how responsive the scrolling or zooming is. Often times, I'm waiting for Android to respond to my touch, other times my touches are missed completely. These issues cause unpredictability for me and make the phone harder to use. As phones got faster, the situation improved, until a larger screen was added, which brought things back down again as the CPU burns to keep up. The milliseconds latency here and the stuttering FPS there decreases the usability significantly in my opinion. I think that there is a group of users who don't notice these latency and animation issues. But I do, and I think many users still unconsciously attribute their dislike for Android to these issues. (example: my brother in law, used several Android Froyo phones extensively before settling on a BlackBerry because it felt "faster".)
  - jrockway 16y ago
    I have never noticed any latency. I blame the manufacturer-specific Android "improvements", as I use plain AOSP and don't have any trouble. My EVO 4G gets about 24 hours battery life, with moderate usage during the day. (But again, this is because I don't use HTC's build. With their build, the battery life is barely long enough to boot the phone.)
    - potatolicious 16y ago
      I've used vanilla Froyo on the Nexus One. It's dog slow. I don't think this problem is isolated to OEM-specific Android modifications (though they certainly don't help). Responsiveness and framerate-wise, even a simple cursory examination will tell you: WP7 and iOS run laps around poor old Android. I got to play with a Nexus S recently, and the responsiveness is greatly improved over Froyo on N1... but is still noticeably less smooth than either Apple or MS's platforms. It's an odd day when MS's UX is more polished than somebody else's... One area where Android's lacklustre performance really hurts usability is diving into menus. iPhone came up with the novel idea of sliding menus - a simple animation that communicates a lot of context to the user (A belongs under B triggers modal window C, all based solely on the direction of the swipe animation). On Froyo+N1 this animation is jerky enough, frequently enough, that this relationship can easily be missed, resulting in more user confusion and worse UI comprehension overall. Google needs to realize that we are in the year 2011, not 1995. "UI design" is no longer about placing the right buttons in the right places, and animation is often now the defining line between an easily comprehensible user experience and an obtuse one. IMHO graphics/animation performance is the #1 obstacle in Android's way, and the main advantage everyone else seems to have on it (and it's a huge advantage). When I use a 2.2 phone, my impression is one of annoyance. I don't know if my touches will be detected, and even if they are, I don't know how long I'll have to wait for the phone to respond. Heck, I enjoy using WP7 phones more than Android because of the performance, despite the fact that WP7 is significantly less featureful than Android! Navigating WP7 is a pleasure, and it never feels like the phone stands in your way. I have yet to get this feeling from Android.
rektide 16y ago
i'd throw ANDROID-1147 and ANDROID-1376 in the Android technical management "absolutely really just doesn't get it" bucket. otoh, the issue Charles Ying of Satine.org rages aobut, ANDROID-6914, seems much more questionable to me because the issue itself is really complex. the two tickets, "Support for external keyboard, mouse" and "Browser doesn't render SVG images" both strike me as absolutely essential to making smartphones competitive with PC's. people have been happily attaching fold up keyboards to Palm Pilots for a decade now, SVG is perhaps the only sane route forwards for resolution independent apps and animation, and Apple has found religion on both these topics-- yet Android is obstinately behind. otoh composited gpu acceleration is not nearly the huge obvious win C. Ying boasts it as. the gpu is more power efficient than the cpu for intensive applications, but a smartphones natural state is not intensive use. mr guy of android points out a number of complexities-- opengl drivers only supporting a single context, opengl acceleration proving less speedy than software implementations. it places huge emphasis on driver support, which any embedded wonk will tell you is a scary prospect. although gpu acceleration is becoming better supported, keep in mind that the smartphone market expands, both up and down, with broadcom & others putting out chips designed for sub $75 smartphones. android is definitely not winning the race for the top atm, but performance has always been a rather lonely market-- that said, unsmooth experiences dont fly. software graphics focus insures a base level of performance for everyone, and strikes me as the right place to start to insure a speedy experience not subject to external interrupts. the current silkiness situation is a little sad. it'll get better. more relevant to me is Android not playing in the ecosystem-- external peripherials, UPnP/DLNA, SVG-- these are the real things that will make or break not Android, but the open digital web, and Android is failing, badly, even in compare with non-web-enthusiasts Apple.
tarandeep 16y ago
Have you guys tried MIUI on Froyo ? It gives me 60FPS for scrolling, panning and almost everything on a Nexus One: http://www.youtube.com/watch?v=Or-9T44Bt7M http://www.youtube.com/watch?v=Or-9T44Bt7M It has got NO GPU acceleration. That PWNs it

wzdd 16y ago

Apparently touch responsiveness is improved in 2.3. However this article doesn't really explain what's wrong with it. I've been doing latency benchmarks for touch recently, and the highest latency I've seen for touch delivery is 10ms. Now 10ms is pretty terrible for what should be a very straightforward thing, but on the other hand it's only 2/3 of a frame, at 60 FPS. The argument in favour of a GPU-accellerated GUI which is not mentioned here is battery life. I haven't been testing this side of things, but just because a CPU can do a 60 FPS GUI doesn't mean it should. Deferring it to the GPU would certainly reduce power consumption while navigating a UI. Whether this is non-negligible in terms of overall battery life, I'm not sure.

tgflynn 16y ago
Are those 10ms with a realistic app/usage scenario ? I wouldn't think users would complain about 10ms, isn't that below the threshold of perception ?
- wtallis 16y ago
  A stutter 2/3 of a frame long means there's a 2/3 chance that you'll draw a frame that doesn't match the current finger position, followed by a frame that catches up more than it should have had to. The ~40ms input lag shown by most desktop IPS LCD monitors is noticeable even though it falls below the reaction time. On a touch display, it should be even easier to notice input lag, because you can see your finger moving and the screen around it not keeping up. 10ms sensor lag may be below the threshold of perception for circumstances like this, but probably not by much. If you add even a few milliseconds of processing lag after your app receives the touch event, you'll be behind by a frame.
- pvaibhav 16y ago
  I work with digital audio/MIDI and a latency of 15-20 ms between pressing a key and hearing the sound is easily audible making it near impossible to play expressive music. 10ms can be felt and is 'annoying', under that is acceptable. I usually keep latency down to 5ms, at most 7ms, only letting it increase, e.g. during mixing when responsiveness is not utterly essential.
16y ago

yoden 16y ago

I think the android bug linked in the article is much more informative than the article itself. Basically: 1) Android phones can already animate the screen @ 60 fps in software in most cases. 2) When they can't, it's not because the CPU is too slow, it's because there is other stuff happening (GC, etc.). A GPU can't fix that.

buymorechuck 16y ago
The GPU can offload most of the graphics intensive tasks from the CPU, utilizing the GPU instead of letting it sit idle. freeing the CPU to focus on GC tasks. In addition, most of the animation work done with Dalvik code can be moved into a dedicated thread / task with highly optimized code backing it.

comex 16y ago

Anyone who thinks Android's animation is "fast enough" is either using a faster device than I have ever tried, or has never tried to compare, say, scrolling lists versus the iPhone. I'm not saying the GPU is the answer, and maybe 2.3 is the answer, but ignoring the problem is not. (Edit: So why does a GC pause stop animation, and not just by taking up CPU time, since moving to the GPU would help in that case? Is the animation code itself written in Java and waiting on the GC? If so, I bet rewriting it in C would be a massive help. Also, yuck.)

eru 16y ago
Or use a GC that doesn't need to stop the world. (Or at least only does so for very small intervals.)
ulrich 16y ago
Gingerbread has a concurrent GC, so animation should not stop completely. However, on a single core system I wouldn't expect this to have a huge impact. But this is just my personal guess.

dave1619 16y ago

I think the author nails it. Without GPU acceleration Android really is a notch below iOS in terms of UX. In my opinion, even WebOS and Windows 7 have much smoother scrolling and touch response than Android. But if Google wakes up and adds GPU acceleration and better eye-candy, watch out Apple.

asdasdrasdasdr 16y ago

What is the main performance difference between Vista SP1 and Windows 7? Hardware accelerated UI.

simonh 16y ago

Purely by chance, I was trying out a galaxy tab at a store today. I loaded up the Financial Times web site, which took an age because the store wifi was crap and tried scrolling the web page and it stuttered quite a bit at first then smoothed up, zooming as juddery as well. Did the same thing on an ipad - slow loading about the same, but the instant the page was rendered it scrolled and zoomed perfectly. Also the widget for editing the web page URL on the Tab was a horrible usability car crash. What's up with that? Disclosure - iPhone user, but Android curious.

typo_pl 16y ago

Android's VM can improve perf-wise - see http://blogs.oracle.com/javaseembedded/2010/11/how_does_android_22s_performance_stack_up_against_java_se_embedded.html http://blogs.oracle.com/javaseembedded/2010/11/how_does_andr...

michaelward82 16y ago

It doesn't matter what the solution is, but Google need to prioritise UI responsiveness above all else. Until that happens, the UI on Android will suck.

deleted 16y ago

[deleted]