4 ms·

The JIT compiler has live runtime data on how the program is actually used and it optimizes around this instead of what the source code says. For instance, the

by vips7L 4y ago

The JIT compiler has live runtime data on how the program is actually used and it optimizes around this instead of what the source code says. For instance, the JIT compiler can see that a function has never been passed a null object and optimize out null checks and other branches that are never taken at runtime. An AOT compiler doesn't have this information unless its provided PGO data at compile time.

hbrn 4y ago
Nitpicking here. > JIT compiler can see that a function has never been passed a null object It cannot. It can only see that function has never been passed a null object so far. And the cost of such optimization might be that edge cases are way slower than anticipated. But what if those edge cases are actually the only thing that provide value for your business? Imagine you're building a monitoring system that alerts very rarely, but reaction time is really critical for you. Do you really want to JIT compiler to optimize the alerting branch out because it hasn't seen any alerts yet? My bet is that JIT compiler rarely does those optimizations by default, because it cannot know which branches in your code contain business value. I don't need my code to be twice as fast on average but twice as slow on critical branches. Now, what would be great if profiler would make suggestions about how to modify your code for performance based on usage. But those should be approved and commented by humans, because humans know whether those are valuable optimizations or a fad.
- vips7L 4y ago
  > so far This is truly a nit pick. > Imagine you're building a monitoring system that alerts very rarely, but reaction time is really critical for you. Do you really want to JIT compiler to optimize the alerting branch out because it hasn't seen any alerts yet? Yes I do, because once the constraints are invalidated the the runtime system will instruct the JIT to recompile the code with the now uncommon branch [0]. [0] https://shipilev.net/jvm/anatomy-quarks/29-uncommon-traps/ https://shipilev.net/jvm/anatomy-quarks/29-uncommon-traps/
  - hbrn 4y ago
    We're talking about optimizations that JIT can do compared to AOT. The link you shared talks about skipping compilation for certain branches to speed up the JIT compilation. Such optimization makes startup faster, but it doesn't make code execution faster. AOT compiled languages don't have slow startup problem in the first place. It's like saying paper books are worse than e-ink readers because you cannot charge paper books.
    - kaba0 4y ago
      Eliding a branch makes not only compilation faster, but the resulting code as well. That’s how for example some logging statements become zero-cost when the debug level is set lower than their corresponding one. JIT compiled languages don’t care too deeply about startup, they shine with longer runtimes.
      - hbrn 4y ago
        Are you sure you're not confusing zero-cost with almost zero-cost? Even zerolog talks about negligible costs after most aggressive optimizations, not zero: https://github.com/obsidiandynamics/zerolog https://github.com/obsidiandynamics/zerolog If you're just using constants in your code to set logging level, AOT compilation can do exactly the same optimizations.
        blackoil 4y ago
        AOT would require recompilation and deployment. JIT can do it for the same code with final params. When the log level is not final, branch prediction Will ensure JIT code is faster.
  - garblegarble 4y ago
    >> JIT compiler can see that a function has never been passed a null object >>... so far. >This is truly a nit pick. It's not even a nitpick, it's just wrong - your original statement is true: you said "has never been". The "so far" reply you received mistakenly implies you said "will never be" (given that HotSpot will deoptimise+reoptimise if its assumptions are invalidated) That said, the remainder of their point I think is reasonable in the extremely specific (I think to the point of being unrealistic, given this seems like a hand-optimised assembly level of perf importance they have alluded to) scenario they mentioned - the JIT optimiser can't know that the user wants to optimise an extremely rare branch for performance to the point where the overwhelmingly common case should be degraded. Another useful jumping off point to explore for HotSpot's performance techniques is [1] 1: https://wiki.openjdk.org/display/HotSpot/PerformanceTechniques https://wiki.openjdk.org/display/HotSpot/PerformanceTechniqu...
- pron 4y ago
  JIT compilers do exactly that. Not only do they have more information, they are also allowed to -- and very frequently do -- perform speculative optimisations. If proven wrong, they incur "deoptimisations" and then recompile with more information.
deleted 4y ago
[deleted]
alserio 4y ago
I don't believe this is right. The optimization as described does not preserve the correct program behavior unless you have a guard to do the null check anyway to fallback to the slow path edit: just read kaba0 answer below and in fact it's possible via tlb miss, really cool

by vips7L 4y ago

hbrn 4y ago

Nitpicking here. > JIT compiler can see that a function has never been passed a null object It cannot. It can only see that function has never been passed a null object so far. And the cost of such optimization might be that edge cases are way slower than anticipated. But what if those edge cases are actually the only thing that provide value for your business? Imagine you're building a monitoring system that alerts very rarely, but reaction time is really critical for you. Do you really want to JIT compiler to optimize the alerting branch out because it hasn't seen any alerts yet? My bet is that JIT compiler rarely does those optimizations by default, because it cannot know which branches in your code contain business value. I don't need my code to be twice as fast on average but twice as slow on critical branches. Now, what would be great if profiler would make suggestions about how to modify your code for performance based on usage. But those should be approved and commented by humans, because humans know whether those are valuable optimizations or a fad.

vips7L 4y ago
> so far This is truly a nit pick. > Imagine you're building a monitoring system that alerts very rarely, but reaction time is really critical for you. Do you really want to JIT compiler to optimize the alerting branch out because it hasn't seen any alerts yet? Yes I do, because once the constraints are invalidated the the runtime system will instruct the JIT to recompile the code with the now uncommon branch [0]. [0] https://shipilev.net/jvm/anatomy-quarks/29-uncommon-traps/ https://shipilev.net/jvm/anatomy-quarks/29-uncommon-traps/
- hbrn 4y ago
  We're talking about optimizations that JIT can do compared to AOT. The link you shared talks about skipping compilation for certain branches to speed up the JIT compilation. Such optimization makes startup faster, but it doesn't make code execution faster. AOT compiled languages don't have slow startup problem in the first place. It's like saying paper books are worse than e-ink readers because you cannot charge paper books.
  - kaba0 4y ago
    Eliding a branch makes not only compilation faster, but the resulting code as well. That’s how for example some logging statements become zero-cost when the debug level is set lower than their corresponding one. JIT compiled languages don’t care too deeply about startup, they shine with longer runtimes.
    - hbrn 4y ago
      Are you sure you're not confusing zero-cost with almost zero-cost? Even zerolog talks about negligible costs after most aggressive optimizations, not zero: https://github.com/obsidiandynamics/zerolog https://github.com/obsidiandynamics/zerolog If you're just using constants in your code to set logging level, AOT compilation can do exactly the same optimizations.
      - blackoil 4y ago
        AOT would require recompilation and deployment. JIT can do it for the same code with final params. When the log level is not final, branch prediction Will ensure JIT code is faster.
- garblegarble 4y ago
  >> JIT compiler can see that a function has never been passed a null object >>... so far. >This is truly a nit pick. It's not even a nitpick, it's just wrong - your original statement is true: you said "has never been". The "so far" reply you received mistakenly implies you said "will never be" (given that HotSpot will deoptimise+reoptimise if its assumptions are invalidated) That said, the remainder of their point I think is reasonable in the extremely specific (I think to the point of being unrealistic, given this seems like a hand-optimised assembly level of perf importance they have alluded to) scenario they mentioned - the JIT optimiser can't know that the user wants to optimise an extremely rare branch for performance to the point where the overwhelmingly common case should be degraded. Another useful jumping off point to explore for HotSpot's performance techniques is [1] 1: https://wiki.openjdk.org/display/HotSpot/PerformanceTechniques https://wiki.openjdk.org/display/HotSpot/PerformanceTechniqu...
pron 4y ago
JIT compilers do exactly that. Not only do they have more information, they are also allowed to -- and very frequently do -- perform speculative optimisations. If proven wrong, they incur "deoptimisations" and then recompile with more information.

deleted 4y ago

[deleted]

alserio 4y ago

I don't believe this is right. The optimization as described does not preserve the correct program behavior unless you have a guard to do the null check anyway to fallback to the slow path edit: just read kaba0 answer below and in fact it's possible via tlb miss, really cool