6 ms·

Title seems incorrect, or at least misleading. It is not a JIT for C++ code (ie. not "C++ to..."), it is a JIT library for C++ that seems focused on compiling a

Title seems incorrect, or at least misleading. It is not a JIT for C++ code (ie. not "C++ to..."), it is a JIT library for C++ that seems focused on compiling arithmetic algorithms as used in bing. Interestingly, it requires the functions called by the JIT'd code to be side-effect free, since it guarantees it will call any given function invocation at least once, since it evaluates both sides of any but top level branches. See "Design Notes and Warnings" in https://github.com/BitFunnel/NativeJIT/blob/master/Documentation/README.md https://github.com/BitFunnel/NativeJIT/blob/master/Documenta...

AstralStorm 10y ago
The real question is why would you use it instead of mature LLVM.
- taspeotis 10y ago
  It sounds like this is both high-throughput and low-latency. I trust LLVM is high-throughput but I imagine it's got a bit of overhead in starting up.
  - gliptic 10y ago
    LLVM is neither high-throughput nor low-latency. It does probably generate faster code, though.
    - taspeotis 10y ago
      Surely LLVM is high-throughput? Otherwise your dev-test cycle of edit/compile/run would suffer from long compilation/link times.
      - jdub 10y ago
        It's "acceptable-throughput" for that use case, and you can balance the tradeoffs of compilation time to optimisation (for time or code size) by adding -On parameters. That's a very different kind of throughput to "compile small functions many times per second".
      - barrkel 10y ago
        These are different metrics of throughput. I'd expect NativeJIT to be 100x to 1000x (or possibly more) faster than LLVM for its use case: temporary expressions measured in low 100s of bytes, no loops, no expression-visible side effects.
- stormbrew 10y ago
  LLVM is a fairly heavyweight tool to use for simplistic or really frequent JIT use-cases. There is definitely room for smaller and simpler tools for that purpose, imo. Like, I'm pretty sure LuaJIT runs rings around LLVM in terms of how quickly it generates code, but at the cost of doing considerably less optimization. This is sometimes a useful tradeoff.
  - haberman 10y ago
    > I'm pretty sure LuaJIT runs rings around LLVM in terms of how quickly it generates code, but at the cost of doing considerably less optimization. This is sometimes a useful tradeoff. You can't really compare LuaJIT and LLVM directly, because LuaJIT is a trace compiler. A LuaJIT trace is a linear sequence of instructions with no control flow, which makes optimization much easier. LLVM on the other hand is statically compiling code consisting of many basic blocks and potentially complicated control flow between them. So LLVM's job is much harder. LuaJIT's optimizer doesn't have to work nearly as hard to get a similar quality of code.
    - sitkack 10y ago
      Because LLVM isn't a JIT in the sense of PyPy, the JVM or Lua. It is more in the sense of the CLR, a late bound AoT compiler. It doesn't watch data access patterns, it doesn't watch arms of a switch statement, there is no feedback.
  - RossBencina 10y ago
    "but at the cost of doing considerably less optimization" That's arguable. Ravi[1] is a Lua implemented with an LLVM-based JIT. The benchmarks here[2] compare runtime performance of Ravi vs. LuaJIT. The Ravi timings don't include compilation time, the LuaJIT timings do. Sometimes LuaJIT siginficantly outperforms Ravi, sometimes it's on par, sometimes worse. But the results don't suggest that there is less (effective) optimisation going on. [1] https://github.com/dibyendumajumdar/ravi https://github.com/dibyendumajumdar/ravi [2] http://the-ravi-programming-language.readthedocs.io/en/latest/ravi-benchmarks.html http://the-ravi-programming-language.readthedocs.io/en/lates...
- paulasmuth 10y ago
  It seems to solve a different usecase. LLVM is (by their own documentation) not really intended for ad-hoc JIT compilation for queries. It doesn't put a lot of priority on making the compilation fast (but rather makes the produced code fast). On top of that, LLVM is a massive dependency. I only looked at NativeJIT a few minutes but it seems to solve both of these issues. It's pretty small in terms of API surface // total amount of code and is explicitly designed for a usecase where compilation happens as part of a user-issued query (that needs to be fast).
- jonathanstrange 10y ago
  GNU lightning would be a better choice. However, it does not optimize.
- cm3 10y ago
  The same reason regex engines have their own specific JIT emitters. Limited overhead and size constraints because it's not a strict requirement to pre-compile regexes. It's basically a native dialect of existing automaton generators, where the abstract code is replaced with machine instructions.
sctb 10y ago
Thanks, we updated the title from “NativeJIT – a C++ to x64 JIT used in Bing”.

AstralStorm 10y ago

The real question is why would you use it instead of mature LLVM.

taspeotis 10y ago
It sounds like this is both high-throughput and low-latency. I trust LLVM is high-throughput but I imagine it's got a bit of overhead in starting up.
- gliptic 10y ago
  LLVM is neither high-throughput nor low-latency. It does probably generate faster code, though.
  - taspeotis 10y ago
    Surely LLVM is high-throughput? Otherwise your dev-test cycle of edit/compile/run would suffer from long compilation/link times.
    - jdub 10y ago
      It's "acceptable-throughput" for that use case, and you can balance the tradeoffs of compilation time to optimisation (for time or code size) by adding -On parameters. That's a very different kind of throughput to "compile small functions many times per second".
    - barrkel 10y ago
      These are different metrics of throughput. I'd expect NativeJIT to be 100x to 1000x (or possibly more) faster than LLVM for its use case: temporary expressions measured in low 100s of bytes, no loops, no expression-visible side effects.
stormbrew 10y ago
LLVM is a fairly heavyweight tool to use for simplistic or really frequent JIT use-cases. There is definitely room for smaller and simpler tools for that purpose, imo. Like, I'm pretty sure LuaJIT runs rings around LLVM in terms of how quickly it generates code, but at the cost of doing considerably less optimization. This is sometimes a useful tradeoff.
- haberman 10y ago
  > I'm pretty sure LuaJIT runs rings around LLVM in terms of how quickly it generates code, but at the cost of doing considerably less optimization. This is sometimes a useful tradeoff. You can't really compare LuaJIT and LLVM directly, because LuaJIT is a trace compiler. A LuaJIT trace is a linear sequence of instructions with no control flow, which makes optimization much easier. LLVM on the other hand is statically compiling code consisting of many basic blocks and potentially complicated control flow between them. So LLVM's job is much harder. LuaJIT's optimizer doesn't have to work nearly as hard to get a similar quality of code.
  - sitkack 10y ago
    Because LLVM isn't a JIT in the sense of PyPy, the JVM or Lua. It is more in the sense of the CLR, a late bound AoT compiler. It doesn't watch data access patterns, it doesn't watch arms of a switch statement, there is no feedback.
- RossBencina 10y ago
  "but at the cost of doing considerably less optimization" That's arguable. Ravi[1] is a Lua implemented with an LLVM-based JIT. The benchmarks here[2] compare runtime performance of Ravi vs. LuaJIT. The Ravi timings don't include compilation time, the LuaJIT timings do. Sometimes LuaJIT siginficantly outperforms Ravi, sometimes it's on par, sometimes worse. But the results don't suggest that there is less (effective) optimisation going on. [1] https://github.com/dibyendumajumdar/ravi https://github.com/dibyendumajumdar/ravi [2] http://the-ravi-programming-language.readthedocs.io/en/latest/ravi-benchmarks.html http://the-ravi-programming-language.readthedocs.io/en/lates...
paulasmuth 10y ago
It seems to solve a different usecase. LLVM is (by their own documentation) not really intended for ad-hoc JIT compilation for queries. It doesn't put a lot of priority on making the compilation fast (but rather makes the produced code fast). On top of that, LLVM is a massive dependency. I only looked at NativeJIT a few minutes but it seems to solve both of these issues. It's pretty small in terms of API surface // total amount of code and is explicitly designed for a usecase where compilation happens as part of a user-issued query (that needs to be fast).
jonathanstrange 10y ago
GNU lightning would be a better choice. However, it does not optimize.
cm3 10y ago
The same reason regex engines have their own specific JIT emitters. Limited overhead and size constraints because it's not a strict requirement to pre-compile regexes. It's basically a native dialect of existing automaton generators, where the abstract code is replaced with machine instructions.

sctb 10y ago

Thanks, we updated the title from “NativeJIT – a C++ to x64 JIT used in Bing”.