4 ms·

I think I landed in a place where it's basically "the compiler has insufficient information to achieve ideal optimization because some things can only be known

I think I landed in a place where it's basically "the compiler has insufficient information to achieve ideal optimization because some things can only be known at runtime." Which is not exclusively an argument for runtime JIT— it can also be an argument for instrumenting your runtime environment, and feeding that profiling data back to the compiler to help it make smarter decisions the next time. But that's definitely a more involved process than just baking it into the same JavaScript interpreter used by everyone— likely well worth it in the case of things like game engines, though.

anyfoo 4y ago
That's a lesson Intel had to (re-?)learn with Itanium as well.
inkyoto 4y ago
> […] "the compiler has insufficient information to achieve ideal optimization because some things can only be known at runtime." This is where the profile guided optimisation comes in – for statically compiled languages, with a caveat being not always straightforward to come up with a set of inputs that will trigger an execution of all possible code paths. One solution is to provide the coverage specifically for the performance critical code paths and let the rest just be.
funstuff007 4y ago
> it can also be an argument for instrumenting your runtime environment Aren't JITs already self-instrumenting? What would you instrument that the JIT is not already keeping track of?
masklinn 4y ago
It's also an argument for having much more expressive and precise type systems, so the compiler has better information. Once you've managed to debug the codegen anyway (see: The Long and Arduous Story of Noalias).
- mikepurvis 4y ago
  Is it? I'd love to see a breakdown of what classes of information can be gleaned from profile data, and how much of an impact each one has in isolation in terms of optimization. Naively, I would have assumed that branch information would be most valuable, in terms of being able to guide execution toward the hot path and maximize locality for the memory accesses occurring on the common branches. And that info is not something that would be assisted by more expressive types, I don't think.
notriddle 4y ago
https://tomaszs2.medium.com/how-rust-1-64-became-10-20-faster-on-windows-3a8bb5e81d70 https://tomaszs2.medium.com/how-rust-1-64-became-10-20-faste... https://news.ycombinator.com/item?id=33306945 https://news.ycombinator.com/item?id=33306945
bluGill 4y ago
The problem with JIT is not all information known at runtime is the correct information to optimize one. In finance the performance critical code path is often the one run least often. That is you have a if(unlikely_condition) {run_time_sensitive_trade();}. In this case you need to tell the compiler to ensure the CPU will have a pipeline stall because of a branch misprediction most of the time to ensure the time that counts the pipeline doesn't stall. The above is a rare corner case for sure, but it is one of those weird exceptions you always need to keep in mind when trying to make any blanket rule.
- ridiculous_fish 4y ago
  How does the compiler arrange for the CPU to mispredict the branch most of the time? I didn't think there were any knobs for the branch predictor other than static ones (e.g. backwards-jumps statically predicted as taken, or PowerPC branch hint bit).
  - bluGill 4y ago
    CPUs have documentation, for this. I forget which one, but the one I did read it was as simple as the true case is assumed more common, and it is easy to arrange logic around that (I probably mis remember, but close enough to that). The common case also should be near the if in memory (inline code), so it is likely on the same cache line (or the next which is prefetched), while the other case is farther away and so you can stall the cache if the if goes that way.
  - mikepurvis 4y ago
    C++20 is gaining the [[likely]] attribute for guiding the predictor in a portable way: http://eel.is/c++draft/dcl.attr.likelihood http://eel.is/c++draft/dcl.attr.likelihood Prior to this it was also possible, but required compiler-specific macros like GCC's __builtin_expect, which is used all over the kernel: https://github.com/torvalds/linux/search?q=__builtin_expect https://github.com/torvalds/linux/search?q=__builtin_expect
    - wumpus 4y ago
titzer 4y ago
Darn it, replied too early. See sibling comment I just posted. The problem with dynamic languages is that you need to speculate and be ready to undo that speculation.

anyfoo 4y ago

That's a lesson Intel had to (re-?)learn with Itanium as well.

inkyoto 4y ago

> […] "the compiler has insufficient information to achieve ideal optimization because some things can only be known at runtime." This is where the profile guided optimisation comes in – for statically compiled languages, with a caveat being not always straightforward to come up with a set of inputs that will trigger an execution of all possible code paths. One solution is to provide the coverage specifically for the performance critical code paths and let the rest just be.

funstuff007 4y ago

> it can also be an argument for instrumenting your runtime environment Aren't JITs already self-instrumenting? What would you instrument that the JIT is not already keeping track of?

masklinn 4y ago

It's also an argument for having much more expressive and precise type systems, so the compiler has better information. Once you've managed to debug the codegen anyway (see: The Long and Arduous Story of Noalias).

mikepurvis 4y ago
Is it? I'd love to see a breakdown of what classes of information can be gleaned from profile data, and how much of an impact each one has in isolation in terms of optimization. Naively, I would have assumed that branch information would be most valuable, in terms of being able to guide execution toward the hot path and maximize locality for the memory accesses occurring on the common branches. And that info is not something that would be assisted by more expressive types, I don't think.

notriddle 4y ago

https://tomaszs2.medium.com/how-rust-1-64-became-10-20-faster-on-windows-3a8bb5e81d70 https://tomaszs2.medium.com/how-rust-1-64-became-10-20-faste... https://news.ycombinator.com/item?id=33306945 https://news.ycombinator.com/item?id=33306945

bluGill 4y ago

The problem with JIT is not all information known at runtime is the correct information to optimize one. In finance the performance critical code path is often the one run least often. That is you have a if(unlikely_condition) {run_time_sensitive_trade();}. In this case you need to tell the compiler to ensure the CPU will have a pipeline stall because of a branch misprediction most of the time to ensure the time that counts the pipeline doesn't stall. The above is a rare corner case for sure, but it is one of those weird exceptions you always need to keep in mind when trying to make any blanket rule.

ridiculous_fish 4y ago
How does the compiler arrange for the CPU to mispredict the branch most of the time? I didn't think there were any knobs for the branch predictor other than static ones (e.g. backwards-jumps statically predicted as taken, or PowerPC branch hint bit).
- bluGill 4y ago
  CPUs have documentation, for this. I forget which one, but the one I did read it was as simple as the true case is assumed more common, and it is easy to arrange logic around that (I probably mis remember, but close enough to that). The common case also should be near the if in memory (inline code), so it is likely on the same cache line (or the next which is prefetched), while the other case is farther away and so you can stall the cache if the if goes that way.
- mikepurvis 4y ago
  C++20 is gaining the [[likely]] attribute for guiding the predictor in a portable way: http://eel.is/c++draft/dcl.attr.likelihood http://eel.is/c++draft/dcl.attr.likelihood Prior to this it was also possible, but required compiler-specific macros like GCC's __builtin_expect, which is used all over the kernel: https://github.com/torvalds/linux/search?q=__builtin_expect https://github.com/torvalds/linux/search?q=__builtin_expect
  - wumpus 4y ago

titzer 4y ago

Darn it, replied too early. See sibling comment I just posted. The problem with dynamic languages is that you need to speculate and be ready to undo that speculation.