5 ms·

But why not just cache the JITed code? There's third party JIT runtimes that do it (Excelsior JET) and it's planned in the official Java 8 runtime. Caching fir

by rockdoe 12y ago

But why not just cache the JITed code? There's third party JIT runtimes that do it (Excelsior JET) and it's planned in the official Java 8 runtime. Caching first-run still gives you the ability to do more/new profiled-based optimization later.

dleskov 12y ago
Excelsior guy here. Actually, Excelsior JET is a proper AOT compiler. That is, a developer or build enginer runs it on their workstation (build server) and then ships a native executable to the end users. We used to have a caching JIT, and even an option to recompile the cache into a single binary, but concluded it is not worth the hassle, and deprecated it last year.
SigmundA 12y ago
If you cache you lose most of the benefits of a jit, might as well AOT one time for everyone.
- rockdoe 12y ago
  Arguments please. The post you are responding to already rebuffs this.
  - SigmundA 12y ago
    See DCKings response. You can reduce startup time of jitting but not eliminate with multistage interpret and background jit with persistent caching and how much complexity and overhead did you add to your runtime and battery power use to get that far? Jitted languages are not known for their performance, they are faster than interpreted ones and slower than the AOT ones generally speaking. So what performance benefit did jit bring compared to a good AOT compiler?
- Twirrim 12y ago
  I'm not sure I understand how. You've gone through the JIT process of live profiling, hot spot detection, compiled down to native code etc. etc., then saved it along the way so that next launch you can start fast with everything already optimised (but also in a position to continue to learn and improve should the opportunity arrive.) Your first run or two will be slower, beyond that you should be as fast as your code will go.
  - deong 12y ago
    What magical extra CPU, memory, cache, and battery have you introduced into that third run to handle the process of "continuing to learn and improve should the opportunity arise"? Because if you're using the same one that's running the user's program, then you won't be running as fast as the code can go. You have to pay for profile guided optimization. That optimizer is another program running on your CPU. It needs memory, time slices, access to secondary storage, etc. It's worth it only if the user code can be improved so much that the total of the two is smaller than what you would have gotten by just compiling it once. And that's not easy to do for general purpose computation.
DCKing 12y ago
> But why not just cache the JITed code? What is the difference between AOT and sufficiently aggressive JIT caching? > Caching first-run still gives you the ability to do more/new profiled-based optimization later. Well you could have your runtime running with the overhead of an interpreter, a profiler, and an optimizer, and hope all will turn out well. Or you could just run the machine code without these complexities and know that it is probably close to optimal anyway.
- rockdoe 12y ago
  What is the difference between AOT and sufficiently aggressive JIT caching? AoT doesn't adapt to changing program profiles.
  - deong 12y ago
    But it also doesn't spend any CPU time, memory pressure, cache lines, etc., on trying to adapt to changing profiles.
  - DCKing 12y ago
    Well, I think the answer to you question then would have to be that not enough advantage could be gained from changing program profiles to compensate the disadvantages of JIT.
  - pjmlp 12y ago
    Yeah, but AOT + PGO does.
thejdude 12y ago
"It's planned in the official Java 8 runtime." Do you have any sources on that? Java 8 has been released for a while now, and this is the first time I hear about this. Given the dynamism of most Java code and the very very wide use of runtime-generated code via cglib or asm, dynamic proxies etc., I think this could actually be quite hard (well, the runtime could still cache the generated code, and if the dynamically generated bytecode matches the stored hash, it could just use the cached code; but maybe there are other issues as well).

by rockdoe 12y ago

dleskov 12y ago

Excelsior guy here. Actually, Excelsior JET is a proper AOT compiler. That is, a developer or build enginer runs it on their workstation (build server) and then ships a native executable to the end users. We used to have a caching JIT, and even an option to recompile the cache into a single binary, but concluded it is not worth the hassle, and deprecated it last year.

SigmundA 12y ago

If you cache you lose most of the benefits of a jit, might as well AOT one time for everyone.

rockdoe 12y ago
Arguments please. The post you are responding to already rebuffs this.
- SigmundA 12y ago
  See DCKings response. You can reduce startup time of jitting but not eliminate with multistage interpret and background jit with persistent caching and how much complexity and overhead did you add to your runtime and battery power use to get that far? Jitted languages are not known for their performance, they are faster than interpreted ones and slower than the AOT ones generally speaking. So what performance benefit did jit bring compared to a good AOT compiler?
Twirrim 12y ago
I'm not sure I understand how. You've gone through the JIT process of live profiling, hot spot detection, compiled down to native code etc. etc., then saved it along the way so that next launch you can start fast with everything already optimised (but also in a position to continue to learn and improve should the opportunity arrive.) Your first run or two will be slower, beyond that you should be as fast as your code will go.
- deong 12y ago
  What magical extra CPU, memory, cache, and battery have you introduced into that third run to handle the process of "continuing to learn and improve should the opportunity arise"? Because if you're using the same one that's running the user's program, then you won't be running as fast as the code can go. You have to pay for profile guided optimization. That optimizer is another program running on your CPU. It needs memory, time slices, access to secondary storage, etc. It's worth it only if the user code can be improved so much that the total of the two is smaller than what you would have gotten by just compiling it once. And that's not easy to do for general purpose computation.

DCKing 12y ago

> But why not just cache the JITed code? What is the difference between AOT and sufficiently aggressive JIT caching? > Caching first-run still gives you the ability to do more/new profiled-based optimization later. Well you could have your runtime running with the overhead of an interpreter, a profiler, and an optimizer, and hope all will turn out well. Or you could just run the machine code without these complexities and know that it is probably close to optimal anyway.

rockdoe 12y ago
What is the difference between AOT and sufficiently aggressive JIT caching? AoT doesn't adapt to changing program profiles.
- deong 12y ago
  But it also doesn't spend any CPU time, memory pressure, cache lines, etc., on trying to adapt to changing profiles.
- DCKing 12y ago
  Well, I think the answer to you question then would have to be that not enough advantage could be gained from changing program profiles to compensate the disadvantages of JIT.
- pjmlp 12y ago
  Yeah, but AOT + PGO does.

thejdude 12y ago

"It's planned in the official Java 8 runtime." Do you have any sources on that? Java 8 has been released for a while now, and this is the first time I hear about this. Given the dynamism of most Java code and the very very wide use of runtime-generated code via cglib or asm, dynamic proxies etc., I think this could actually be quite hard (well, the runtime could still cache the generated code, and if the dynamically generated bytecode matches the stored hash, it could just use the cached code; but maybe there are other issues as well).