4 ms·
Ask HN: Why is Android moving to AOT Java compilation?
http://anandtech.com/show/8231/a-closer-look-at-android-runtime-art-in-android-l
I'm a bit surprised to see Android move to AOT compilation, together with large claimed performance benefits. As far as I know, aside from ART, all state of the art Java Runtimes are using JIT. In theory, JIT should have the advantage of having live profile data, the ability to speculatively do optimization and back off when they're wrong, etc...
The article list a bunch of benefits, but they don't really seem to stand up to scrutiny. Overhead from repeatedly having to JIT the code? Cache it - also removing the battery life argument. AoT having the advantage of seeing all the code and do global optimizations? It's just the opposite!
I could see some advantage of AoT for first-startup, but that isn't even claimed as an advantage, because the AoT happens at install, not at compilation/packaging time.
It's even stranger as modern Android phones have 4 to even 8 cores. That leaves plenty of horsepower to do the JITing in the background and be even faster.
Now, I can see that ART is faster simply because it's a better, second iteration on Dalvik (which wasn't great). Dalvik was 2-3 times slower than standard JITs. ART seems to make up this difference, but that still leaves the question: why did Google choose to go the AoT route instead of JIT?
- mateuszf 12y agoThis is a question for stackoverflow, not HN. Also, just use google.
- barrkel 12y agoPower is a bigger constraint on mobile devices than raw speed.
- rockdoe 12y agoWhat advantage does AoT have over JIT in terms of power? Recompiling is a non-issue, if you can cache AoT code you can cache JIT code. Not to mention, faster code uses less battery.
- SigmundA 12y agoRecompiling is an issue on large code bases, that's the problem, power is secondary, slow app startup is primary in my experience. If you cache jit code it loses it live data benefit for the most part, might as well AOT on your build server instead.
- rockdoe 12y agoIf you cache jit code it loses it live data benefit for the most part As pointed out elsewhere: that's just not true.
- drdaeman 12y agoBut would AOT have significant effect in this regard? From my experience, major power consumers are display, its backlight and various connectivity interfaces. Oh, and working GPS can surely drain the phone in an hour or two. Compared to those, any possible differences AOT vs JIT CPU power consumption (if there are any and if they are somehow in favour of AOT) seem totally negligible. But maybe I'm just wrong on this.
- SigmundA 12y agoBecause making your users wait for your code to compile over and over again on their device is not what you want your users doing. .Net has the same issue and MS also went this route for Windows phone and it's now making its way to the desktop. If it can be done at build time it should, I think everyone is realizing that on many fronts. Long startup times and battery power being consumed by a jit is a worse users experience.
- rockdoe 12y agoBut why not just cache the JITed code? There's third party JIT runtimes that do it (Excelsior JET) and it's planned in the official Java 8 runtime. Caching first-run still gives you the ability to do more/new profiled-based optimization later.
- dleskov 12y agoExcelsior guy here. Actually, Excelsior JET is a proper AOT compiler. That is, a developer or build enginer runs it on their workstation (build server) and then ships a native executable to the end users. We used to have a caching JIT, and even an option to recompile the cache into a single binary, but concluded it is not worth the hassle, and deprecated it last year.
- SigmundA 12y agoIf you cache you lose most of the benefits of a jit, might as well AOT one time for everyone.
- rockdoe 12y agoArguments please. The post you are responding to already rebuffs this.
- SigmundA 12y agoSee DCKings response. You can reduce startup time of jitting but not eliminate with multistage interpret and background jit with persistent caching and how much complexity and overhead did you add to your runtime and battery power use to get that far? Jitted languages are not known for their performance, they are faster than interpreted ones and slower than the AOT ones generally speaking. So what performance benefit did jit bring compared to a good AOT compiler?
- deong 12y agoIf you cache everything, you're just doing AoT compilation. But in general, I would say that the performance advantages of JIT have never really materialized outside of a few specific types of code. In theory, there should be room for them to appear, but in theory, we're all running Itanium chips. In practice, we haven't been able to write software to take advantage of those theoretical possibilities very well. Look at it this way -- there are a lot of really smart people working at Google on Android, and despite having a completely working JIT environment and centuries of man-effort to devote to it, they went with a complete rewrite of the runtime in order to gain speed. They wouldn't have done that if there were easier paths to efficiency.
- david87600 12y agoGood Points ... I've been doing Java development for 14 years and it is just plain slower for most things I've done. Sure you can write loop which will optimize correctly but most of the code executes more slowly or suffers from class loading slowness or GC delays. This is why my android phone needs a quad-core 1.5 GHZ processor to feel as fast as an IPhone 4S.
- rockdoe 12y agoIf you cache everything, you're just doing AoT compilation. Not at all. That's a false dichotomy. You can restore the past JIT-ed code from the cache and do new profile directed optimizations. Java JITs already do this, just not (AFAIK!) HotSpot. there are a lot of really smart people working at Google on Android ...and yet their previous runtime was 3 times slower than Sun's one, requiring a full rewrite. Sorry, that's just an appeal to authority, not an argument.
- deong 12y ago"You can restore the past JIT-ed code from the cache and do new profile directed optimizations." Sure, but now you're paying for the compilation/runtime analysis stage more than once. If the JIT compiler is actively doing any profiling of your code, that's time and watts being spent not running the program the user wanted you to run. In order for it to be a net gain, you have to improve the efficiency of the user's program enough through JIT to pay for this repeated cost of doing the analysis however often you do it. You can profile more intensively and more often, thus improving the generated code, but now you need to improve it a lot. Or you can profile lightly or infrequently, but now you have way less information with which to improve it. In practice, it seems to be very hard to do significantly better than just running it through a good compiler once.
- sp332 12y agoMaybe it helps with latency? When a user presses a button, you don't want to have to compile some code before you can show them the result.
- rockdoe 12y agoThat's not really an advantage of AoT over JIT+interpreter, is it? AoT requires compiling at first run, the JIT can just use the interpreter (and preferably compile in the background).
- bane 12y agoWouldn't the background JIT thread possibly cause some of Android's notorious stuttering in the UI? Let's be honest, today's smartphones are more powerful than desktops of only a 3-5 years ago for most things, have as much RAM, decent GPUs and as many cores and near enough as much storage. It's actually pretty miraculous what's been crammed into these tiny mobile computers. When you remove the battery out of one it's pretty astonishing how much computing you can fit in a few ounces.
- rockdoe 12y agoWouldn't the background JIT thread possibly cause some of Android's notorious stuttering in the UI? No? Why would it? Unless it's a single-core device with bad scheduling and prioritization, that makes no sense.
- blinkingled 12y agoStartup time. JIT doesn't kick into action until after the app has started up and ran through some hot code paths. Up until that time things are interpreted and thus slower. Secondly there are limits to how much you can optimize at run time. For servers there is more leeway in terms of available memory and CPU power but on mobile you can only optimize so much without slowing things down with optimization and jit code generation overhead. JIT generated code has to be generated every time the app runs. Besides the pages used to store the JIT code are process private - I.e. no sharing. Compared to all of this - AOT can afford to optimize more, it only has to do it once, the framework code can be spit out to common shared files on the disk this enabling sharing of those pages and reducing memory consumption. This gives you speed, battery and memory benefits over JIT.
- rockdoe 12y agoUp until that time things are interpreted and thus slower. That's true - but not specific to mobile. For servers there is more leeway in terms of available memory and CPU power but on mobile you can only optimize so much without slowing things down with optimization and jit code generation overhead. Which makes me wonder why it's going into new versions of Android (running on 4 to 8 core devices, i.e. having plenty of free horsepower). If anything this is becoming less of an issue. JIT generated code has to be generated every time the app runs. False. The memory savings argument might be something - particularly with heavy use of system libraries. Android phones usually have relatively memory compared to their other resources.
- 51Cards 12y ago"Which makes me wonder why it's going into new versions of Android (running on 4 to 8 core devices, i.e. having plenty of free horsepower)." ... running on a very taxed battery.
- blinkingled 12y ago>>JIT generated code has to be generated every time the app runs. > False. How so? Do you know of production class JITs that cache generated native code and reuse it every time the process is re-executed?
- stcredzero 12y agoAgain and again Apple's much criticized design choices turn out to be the right ones. In a resource constrained environment, AOT compilation and reference counting get you to a fairly high level of performance and good levels of abstraction with enough simplification for the programmer's task, but without taking too much away from UX in the form of lag. Mobile is always going to be resource constrained for the same reason highways are always busy at rush hour. Such resources are too valuable to stay unused for long. (Dalvik will never go to ref counting, but the design trade offs in its GC will reflect the same goals.)
- BitMastro 12y agoYou're comparing apples and oranges, Apple had the privilege of choosing its own hardware and programming language. For android they went for java because of popularity, and everyone can use it on their own hardware. So you have a java based language running on ARM, x86 and MIPS, what would you choose? A proven approach like JIT VM or AOT?
- david87600 12y agoRe: "For android they went for java because of popularity, and everyone can use it on their own hardware." True, but only the hardware independence part is true. Android was targeted for any vendor, while Apple targeted only their A-series processor.
- stcredzero 12y ago> You're comparing apples and oranges, Apple had the privilege of choosing its own hardware and programming language. Google and Apple both used already existing languages and adapted existing kernels. If you're saying that Apple didn't use an existing language or that Google didn't have the option of being more restrictive with hardware, then you are wrong on both counts. Google made choices which turned out to be worse for users over a certain timeframe. They were more attractive to vendors and manufacturers, however. > For android they went for java because of popularity, and everyone can use it on their own hardware. Google was optimizing for programer popularity, not user experience. > So you have a java based language running on ARM, x86 and MIPS, what would you choose? A proven approach like JIT VM or AOT? If you're going to go with what's more proven with constrained resources, then you'd go AOT. If you're trying to optimize for UX and lag, but high level enough for rapid development, then AOT with reference counting GC.
- edderly 12y ago> modern Android phones have 4 to even 8 cores, That leaves plenty of horsepower to do the JITing in the background Android can't really look at the world like that, the platform is being used in wearables through to what will be console class devices. Android must be able to work in the low-end, take project Svelte for example, as being important to get Android (back) to being usuable in 512MB devices. Also typically most effective power management strategies run to idle then off, rather than keep power consumers alive at some nominal frequency. > Overhead from repeatedly having to JIT the code? Cache it Caching in memory is a limitation because of basic memory constraints, and caching to persistent storage (swap) is not a great option because of the relative speed of mobile storage and to a certain extent wear levelling effects on flash may also be a concern. I argue that what gets lost in the fog when talking about JITs and virtual machines vs static / AOT based solutions is that VMs can be very performant when you can ignore startup time and when there is a single instance of a VM that you have to manage on a system. Take an Android device and look at the number of activities resident at any one time, you'll have 10 maybe 20 plus activities present, each of which is an instance of a virtual machine spawned from Zygote. This makes memory footprint of the VM critical, because the penalty for restarting Activities is often pretty high.
- captainmuon 12y agoApart from technical reasons, there might be political or legal ones. One would be to bring the platform further away from Oracle's Java. Standard java uses the JVM, Android uses the Dalvik VM. While technically quite different, one could argue that in both cases Java is compiled to bytecode, and thus Android would be ripping of Java. ART would mean moving to machine code and thus a step away from the VM paradigm. However, I don't think this is a compelling reason, since the disputes have always been around the Java API and parts of the source, and not about the architecture. Another (more paranoid) idea is that ART would nip stuff like Cydia Substrate in the bud. Substrate is basically a framework to hook arbitrary functions in apps or in the OS. For that, it depends on rewriting Dalvik bytecodes. If Google wants to lock the platform down to disallow stuff (bypassing DRM, turning off the camera sound, installing facial-recognition software on Glass, etc.) it would make sense to stop that kind of tool.
- pjmlp 12y agoThere are certified JVMs with AOT compilers available.
- TheJCDenton 12y agoIMO, more static code means even more predictability in terms of object placement in memory, thus allowing much more aggresive memory optimization : less fragmentation, efficient pagination etc... It means also that you have static code and GC interacting with it, it's pretty new, isn't it ?
- politician 12y agoNot really, these are orthogonal concerns. One could just as easily write an abusive unpredictable memory allocator in Go (native, GC) as in Java (bytecode, GC) or C (native, no GC).
- sp332 12y agoMaybe they're not really moving away from JIT but just Dalvik. After they have the new runtime running well as AOT-compiled, they can move some of the optimizations into a tracer or other profiler, in a hybrid approach.