4 ms·

Ugh at risk of being the grumpy old programmer: jit code is just never going to touch actual handwritten native code in performance. It's been like 50 years. If

Ugh at risk of being the grumpy old programmer: jit code is just never going to touch actual handwritten native code in performance. It's been like 50 years. If we care about performance we need languages for it.

pizlonator 7y ago
Totally true. JITs are just about making the dynamic languages perform better than if they were interpreted. If you want native perf you need a statically typed and ahead of time compiled language.
hansvm 7y ago
I mostly agree, but compilers intentionally don't explore every optimization path to save compilation time, and a JIT is able to trace the program for real data and find places that could benefit from additional attention. I don't think JITs are the only way to address that trade-off in compilation time vs runtime performance metrics, and whether their benefits are worth their other costs is an interesting, program-dependant question, but they have strictly more information available than a naive compiler and can potentially use that to make better decisions.
- jjnoakes 7y ago
  Any languages explore both? Compile to native and run with a jit which profiles and optimizes further based on runtime behavior?
  - pizlonator 7y ago
    No fundamental reason why you cannot do this. Maybe Graal can even do this for Java since it can both compile to native and do profile guided optimization (but I don't know if they ever combine the two). Here's a reason why this might not be a super great thing to do, if you're thinking of it from a language design standpoint: - Profiling and rejitting costs you memory and makes all code pay a tax. The biggest tax is the safepoint/OSR tax. It costs significant memory and some small amount of time (maybe the time cost you add from comprehensive OSR support is like 1%-5% overall, but I'm not sure, because it's hard to isolate this cost and measure it). So you don't want to build a system that supports JITing unless it's going to give you big wins. That implies languages with lots of dynamic typing. - The languages that most benefit from JITs (the win they get consistently overcomes the overhead of JITing) are the ones that have so much dynamic typing features that compiling them to native is a super hard problem. Java is one of the few languages that is both dynamic enough to benefit from JITs but static enough to be possible to compile to native. And even for Java the native compilation is a lot of effort to get right. Note that "possible to compile to native" to me means: compiling to native produces something that performs well so there exists some benefit to actually doing it. Like, I would expect compiling JavaScript to native to produce something that doesn't perform well at all. So, basically, the issue is that most languages are either in the "benefit of JIT is smaller than cost of JIT" category because they have adequately static typing or in the "must have JIT and cannot compile statically" category because they don't have static typing. Not a lot of languages are in the sweet spot where doing both would help, but such languages exist (Java) and there's no reason why they can't do what you suggest and they may already do it (seems like a natural thing for Graal to do).
  - charleslmunger 7y ago
    You can get a bunch of the benefit by applying FDO: https://llvm.org/devmtg/2013-04/novillo-slides.pdf https://llvm.org/devmtg/2013-04/novillo-slides.pdf
  - jdsully 7y ago
    Its called profile guided optimization, although they don’t run with an actual JIT. The big 3 C compilers all have it.
    - The_rationalist 7y ago
      It is clearly less powerful, it analyse one sample and extrapolate from it. Every user should have it's own PGO and regularly refresh it to be competitive with the advantage of JIT profiling
      - btrask 7y ago
        This is a completely legitimate comment (which was 'dead' when I saw it). PGO really is less powerful than JIT. A compiled language with a "micro-JIT" (e.g. for v-tables) seems like an interesting idea to me.
    - tracker1 7y ago
      Just curious what the big 3 you're considering are? - gcc - clang - msvc - intel
      - jdsully 7y ago
        Yes in that order. Maybe you could say big 4, but I find Intel is only popular in smaller niches.
    - fulafel 7y ago
      PGO as done in C family compilers of today can't make speculative optimizations and fall back to recompilation, so it's much weaker.
  - voldacar 7y ago
    i think there are some common lisp implementations that do this
fulafel 7y ago
I agree that we need to design languages that take into account advanced optimizations, but we're pretty far from the limit of compiler optimizations - partly because many languages have inconvenient semantics for it. We haven't even begun to leverage data layout optimizations and there's a lot of things about feedback directed partial application style transformations that haven't made it out of academia to production compilers. In some sense we can always fully specialize a program to its current input and make a lot of radical changes to data layout and represesentation. But in languages like C/C++ where programs can "see" struct/array layout, member order, pointers as integer values that have fixed order, spacing, grouping based on the program specified data types, this is prohibitively complicated.

pizlonator 7y ago

Totally true. JITs are just about making the dynamic languages perform better than if they were interpreted. If you want native perf you need a statically typed and ahead of time compiled language.

hansvm 7y ago

I mostly agree, but compilers intentionally don't explore every optimization path to save compilation time, and a JIT is able to trace the program for real data and find places that could benefit from additional attention. I don't think JITs are the only way to address that trade-off in compilation time vs runtime performance metrics, and whether their benefits are worth their other costs is an interesting, program-dependant question, but they have strictly more information available than a naive compiler and can potentially use that to make better decisions.

jjnoakes 7y ago
Any languages explore both? Compile to native and run with a jit which profiles and optimizes further based on runtime behavior?
- pizlonator 7y ago
  No fundamental reason why you cannot do this. Maybe Graal can even do this for Java since it can both compile to native and do profile guided optimization (but I don't know if they ever combine the two). Here's a reason why this might not be a super great thing to do, if you're thinking of it from a language design standpoint: - Profiling and rejitting costs you memory and makes all code pay a tax. The biggest tax is the safepoint/OSR tax. It costs significant memory and some small amount of time (maybe the time cost you add from comprehensive OSR support is like 1%-5% overall, but I'm not sure, because it's hard to isolate this cost and measure it). So you don't want to build a system that supports JITing unless it's going to give you big wins. That implies languages with lots of dynamic typing. - The languages that most benefit from JITs (the win they get consistently overcomes the overhead of JITing) are the ones that have so much dynamic typing features that compiling them to native is a super hard problem. Java is one of the few languages that is both dynamic enough to benefit from JITs but static enough to be possible to compile to native. And even for Java the native compilation is a lot of effort to get right. Note that "possible to compile to native" to me means: compiling to native produces something that performs well so there exists some benefit to actually doing it. Like, I would expect compiling JavaScript to native to produce something that doesn't perform well at all. So, basically, the issue is that most languages are either in the "benefit of JIT is smaller than cost of JIT" category because they have adequately static typing or in the "must have JIT and cannot compile statically" category because they don't have static typing. Not a lot of languages are in the sweet spot where doing both would help, but such languages exist (Java) and there's no reason why they can't do what you suggest and they may already do it (seems like a natural thing for Graal to do).
- charleslmunger 7y ago
  You can get a bunch of the benefit by applying FDO: https://llvm.org/devmtg/2013-04/novillo-slides.pdf https://llvm.org/devmtg/2013-04/novillo-slides.pdf
- jdsully 7y ago
  Its called profile guided optimization, although they don’t run with an actual JIT. The big 3 C compilers all have it.
  - The_rationalist 7y ago
    It is clearly less powerful, it analyse one sample and extrapolate from it. Every user should have it's own PGO and regularly refresh it to be competitive with the advantage of JIT profiling
    - btrask 7y ago
      This is a completely legitimate comment (which was 'dead' when I saw it). PGO really is less powerful than JIT. A compiled language with a "micro-JIT" (e.g. for v-tables) seems like an interesting idea to me.
  - tracker1 7y ago
    Just curious what the big 3 you're considering are? - gcc - clang - msvc - intel
    - jdsully 7y ago
      Yes in that order. Maybe you could say big 4, but I find Intel is only popular in smaller niches.
  - fulafel 7y ago
    PGO as done in C family compilers of today can't make speculative optimizations and fall back to recompilation, so it's much weaker.
- voldacar 7y ago
  i think there are some common lisp implementations that do this

fulafel 7y ago

I agree that we need to design languages that take into account advanced optimizations, but we're pretty far from the limit of compiler optimizations - partly because many languages have inconvenient semantics for it. We haven't even begun to leverage data layout optimizations and there's a lot of things about feedback directed partial application style transformations that haven't made it out of academia to production compilers. In some sense we can always fully specialize a program to its current input and make a lot of radical changes to data layout and represesentation. But in languages like C/C++ where programs can "see" struct/array layout, member order, pointers as integer values that have fixed order, spacing, grouping based on the program specified data types, this is prohibitively complicated.