As folks have mentioned, LLVM's JIT produces great code and is (relatively) easy to use. However, LLVM is extremely heavyweight. Which "latency" did you mean?

As folks have mentioned, LLVM's JIT produces great code and is (relatively) easy to use. However, LLVM is extremely heavyweight. Which "latency" did you mean? Are you going to run these functions 1M times, so that the quality of the generated code is paramount (and you can afford really long compile time) or do you care more about "I hit enter and get the answer"? You can tune LLVM (disable almost all passes, use fast instruction selection) but it's really not focused on millisecond-ish compilation. There are a lot of "simple JIT" libraries for the latter case (you just want to feed in a simple IR and get machine code out, do an okay job at register allocation, but nothing fancy). None of them has "won" and most only have C bindings (to my knowledge).