5 ms·
Azul introduces remote compilation for Java
- ggfgg 5y agoI can see the benefit of this. We have similar problem with JIT executing on a whole bunch of containers during a roll out. However I’d rather scrap the software and rewrite it in something with a full AOT compiler and rapid startup at this point. And without more dependencies which will fuck up mid deployment whenever cloud provider X falls over.
- mgaudet 5y agoInteresting. OpenJ9 has something similar in preview: https://blog.openj9.org/tag/jitserver/ https://blog.openj9.org/tag/jitserver/
- gavinray 5y agoSo I thought this whole thing was bullshit, then I read the blogpost from OpenJ9 I changed my mind, this image is really what they ought to be showing. It gets the point across: https://i0.wp.com/blog.openj9.org/wp-content/uploads/2021/09/image.png?w=744&ssl=1 https://i0.wp.com/blog.openj9.org/wp-content/uploads/2021/09... What you're looking at is allocation of multiple apps inside of nodes in a cluster. With a JIT Server in each node, the memory required for each instance of the app is reduced, such that the effect is more instances can be fit in the same node size than before. It reminds me of "bonuses" on equipment in RPGs. You lose 1 equipment slot to have it taken up by the item, but in return the rest of your equipment gets a bonus that more than makes up for the slot you can't use now.
- kazinator 5y agoI suppose ahead-of-time compiling everything so there is no compiler in any application node is a technique that is still some years away.
- pjmlp 5y agoThe problem is that requires a closed world approach, or constrained usage of reflection. Yes there have been AOT compilers since around 2000, however you will notice that they target specific deployment cases, and also offer JIT caches in alternative. Actually that is also how Android rebooted their AOT efforts in version 7. You can see it in GraalVM and native image as well.
- kjeetgill 5y agoThe article addresses this actually. For a language like Java, the jit is the source of a ton of it's performance. Think of it more like continuous PGO, but the delta perf improvement is much higher.
- heisenbit 5y agoSomehow the link of the OP does not work. This one does: https://www.theregister.com/2021/12/15/azul_introduces_remote_compilation_for/ https://www.theregister.com/2021/12/15/azul_introduces_remot...
- ksec 5y agoSorry, this page doesn't exist! Edit: Could @dang change the link to https://www.theregister.com/2021/12/15/azul_introduces_remote_compilation_for/ https://www.theregister.com/2021/12/15/azul_introduces_remot...
- dang 5y agoFixed now. Thanks!
- rightbyte 5y agoSeems like a rather unnecessarily complex dependency to do remote JIT compiles. "it can reduce compute resources by up to 50 per cent" How ... Do they not include their cloud service in the computations required?
- hyperman1 5y agoPresumably because it becomes a shared cost. 100 JVMs in 100 containers have to pay that cost individually. Now 1 or 2 central instances have the resources for compiling, the 100 worker bees don't need them. They also might becable to optimize faster as they have runtime statistics from other instances, skipping warmup.
- dragontamer 5y ago100 JVMs in 100 containers (on one machine) also have to pay the costs of 100x individual garbage collections, instead of centralizing those garbage collections and merging the memories of everyone together. Hmmm... maybe just having a monolith at that point is better? If you can run 100 containers on one box, why not just run one environment on one box?
- vlovich123 5y agoI know you're joking but there is an important distinction that enables this. The optimization choices you're going to make of how to lower bytecode to native are going to be largely the same across all your machines - the minute differences in profiles aren't going to make significant differences unless you have the same codebase doing drastically different things. Thus you can share cost easily that so you accumulate execution profiles across your cluster and then lower to native code efficiently. The garbage collection choices are going to be unique to the workload running on that server and have little information worth sharing that will speed up operations.
- hinkley 5y agoWe could in theory get a lot of this benefit by supporting profile guided optimization and building a simple distribution system. You could even instrument a fraction of your servers and distribute to the rest.
- coldcode 5y agoRemote inside your environment might make sense, but sending your source to an external entity to compile sounds like a security nightmare.
- cogman10 5y agoReading the article, they are only talking about "remote in your environment" because this is somewhat latency sensitive. Sending something like this over the internet would have far too many issues.
- orev 5y agoDevelopers have been fine with moving everyone else’s stuff to the cloud: Ops, data, security, networking, etc., but now when the cloud comes for your source code that’s a step too far? That horse left the barn a long time ago.
- aredplug 5y agoThink of this as an extension of the build pipeline, which already involves shipping source and build artifacts between multiple machines.
- stickfigure 5y agoConsider me extremely skeptical. One more network service to potentially fail or have security issues. And a whole new thundering herd problem on system recovery. I can only imagine this is tailored for some peculiar workload that I've never seen.
- cogman10 5y agoThe 2 usecases I see this working well for. * Anything involving autoscaling * My usecase of 1000s of servers doing ETL type work with java processes getting created and destroyed often. I don't think this is a great boon for long lived services.
- aredplug 5y agoAzul is popular in low latency financial services. A usecase might be to reduce the variance that JIT compilation introduces to transaction latency, especially at the high percentiles.
- deleted 5y ago[deleted]
- anonymousDan 5y agoInteresting. I wonder would it permit more advanced optimization passes that might have too much impact on performance otherwise?
- nradov 5y agoAny additional gains at the compilation stage are going to be marginal. The real potential is with further dynamic run time optimization within the JVM.
- anonymousDan 5y agoNot sure I understand what your disagreement is... as I said you can do more advanced compilation, which by implication should give you better performance. Is that what you meant?
- nradov 5y agoThe gains from more advanced compilation are going to be so small as to not be worth the hassle for most code bases. It's better to focus effort on other areas.
- jedberg 5y agoBack around 2004, Azul created a special machine to run Java. It had 792 small CPUs. You would put it in your datacenter and then replace the JVM on each machine with one that could use this box. There were some constraints on which calls and libraries you could use, but most Java code just worked. It would use the network to do the JIT on this custom Java box, and it was super fast as long as your machine shared a fast switch with the box. I wonder if this is basically the resurrection of that with some extra magic to make it work over the internet.
- drunkenmagician 5y agoSeems more like a marketing ’feature’, to justify jvm per node pricing, more than a feature with technical merit