6 ms·

This seems poorly grounded. In fact almost three decades after the release of the Java HotSpot runtime we're still waiting for even one system to produce the p

• deleted 2y ago
  [deleted]
• noelwelsh 2y ago
  The benchmarks I have seen show Hotspot is ahead of V8. E.g. https://stefan-marr.de/papers/oopsla-larose-et-al-ast-vs-bytecode-interpreters-in-the-age-of-meta-compilation/ https://stefan-marr.de/papers/oopsla-larose-et-al-ast-vs-byt... What makes this very complicated is that 1) language design plays a big part in performance and 2) CPUs change as well and this anecdotally seems to have more impact on interpreter than compiler performance. With regards to 1), consider optimizing Javascript. It doesn't have machine integers, so you have to do a bunch of analysis to figure when something is being used as an integer and then you can make that code fast. There are many other cases. Python is even worse in this regard. In comparison AOT compiled languages are usually designed to be fast, so they make tradeoffs that favour performance at the cost of some level of abstraction / expressivity. The JVM is somewhere in the middle, and so is its performance. With regards to 2) this paper is an example, as is https://inria.hal.science/hal-01100647/file/InterpIBr-hal.pdf https://inria.hal.science/hal-01100647/file/InterpIBr-hal.pd...
  • ajross 2y ago
    With all respect that sounds like excuse-making. I mean, yeah, Javascript and JVM and .NET are slower runtimes than C or Rust[1]. Nonetheless that's the world we live in, and if you have a performance-sensitive problem to solve you pick up rustc or g++ and not a managed runtime. If that's wrong, someone's got to actually show that it's wrong. [1] Maybe Go or Swift would be more apples-to-apples. But even then are there clear benchmarks showing Kotlin or C# beating similar AoT code? If anything the general sense of the community is that Go is faster than Java.
    • noelwelsh 2y ago
      Excuses for what? I'm not the elected representative for JIT compiled languages, sworn to defend them. There are technical reasons they tend to be slower. I was sketching some of them.
      • mrkeen 2y ago
        I think the above comments are because JIT gets so much positive press, someone wandering in from outside could be mistaken for thinking that JIT isn't coming 2nd in a two-man race with AOT. I've been around long enough to hear that Java and JIT are gonna overtake C++ any day now. The title on this article doesn't help.
    • pca006132 2y ago
      When things are performance-sensitive, you want things to be tunable and predictable. Good luck playing with the JIT if you rely that for performance...
      • pjmlp 2y ago
        Good luck with AOT as well, unless you hardcode the target hardware, like game consoles.
    • wiseowise 2y ago
      https://devblogs.microsoft.com/oldnewthing/20060731-15/?p=30293 https://devblogs.microsoft.com/oldnewthing/20060731-15/?p=30... https://blog.codinghorror.com/on-managed-code-performance-again/ https://blog.codinghorror.com/on-managed-code-performance-ag... And that was 2005. Modern .NET is much, much faster. > If anything the general sense of the community is that Go is faster than Java. Faster where?
  • MaxBarraclough 2y ago
    > you have to do a bunch of analysis to figure when something is being used as an integer and then you can make that code fast It doesn't get much attention now that WASM exists, but asm.js essentially solves this, so a more head-to-head comparison ought to be possible. (V8 has optimisations specific to asm.js.) https://en.wikipedia.org/wiki/Asm.js https://en.wikipedia.org/wiki/Asm.js
    • IainIreland 2y ago
      asm.js solves this in the specific case where somebody has compiled their C/C++ code to target asm.js. It doesn't solve it for arbitrary JS code. asm.js is more like a weird frontend to wasm than a dialect of JS.
      • lern_too_spel 2y ago
        No, if you just use the standard JavaScript cast to integer incantation, |0, v8 will optimize it. asm.js is valid JavaScript.
      • MaxBarraclough 2y ago
        Sure, but that was essentially my point. If we're trying to compare HotSpot and V8 for similar input code, Java and asm.js seem closer than Java and full-blown JavaScript with its dynamic typing.
• ForTheKidz 2y ago
  > I guess consensus is that V8 has come closest? V8 better than the JVM? Insanity, maybe it can come to within an order of magnitude in terms of performance.
  • edflsafoiewq 2y ago
    Comes closest to realizing the concept of a JIT that is better than AOT.
    • ForTheKidz 2y ago
      I think that's completely silly framing; you can AOT compile any code better—or at least, just as well—if you already know how you want it to perform at runtime. Any efficiency gain would necessarily need to be in the context of total productivity.
      • ajross 2y ago
        > I think that's completely silly framing It's literally the framing of the linked article though, which takes as a prior that JIT compilers are already ahead of AoT toolchains. And... they aren't!
        • ForTheKidz 2y ago
          > This gives them an advantage when compared to Ahead-of-Time (AoT) compilers that must choose the code to generate once for all. I assumed they were talking about the general case, which is nearly useless to discuss. I just kind of filtered it out as internecine bickering amongst academics. The actual data are still interesting tho.
        • noelwelsh 2y ago
          They are comparing Javascript JIT to Javascript AOT, to avoid the issue of language design. "The fastest contemporary JavaScript implementations use JIT compilers [27]. ... However, JIT compilers may not be desirable or simply not available in some contexts, for instance if programs are to be executed on platforms with too limited resources or if the architecture forbids dynamic code generation. Ahead of time (AoT) compilers offer a response to these situations. Hopc [25] is an AoT JavaScript-to-C compiler. Its performance is often in the same range as that of the fastest JIT compilers but its impossibility to adapt the code executed at runtime seems a handicap for some patterns and benchmarks [27]." In the context of JS it's reasonable to think that JIT may have an advantage, as the language is difficult to statically analyse.
• IshKebab 2y ago
  I agree, the "JITs can be faster because X Y Z" arguments have never turned into "JITs are actually faster". Maybe that's because JIT is almost always used in languages that were slowed in the first place, e.g. due to GC. Is there a JITing C compiler, or something like that? Would that even make sense?
  • remexre 2y ago
    Maybe the "allocate as little as possible, use sun.misc.Unsafe a lot, have lots of long-lived global arrays" style of Java programming some high-performance Java programs use would get close to being a good stand-in.
  • pjmlp 2y ago
    C++/CLI is one example, it is C++, not C, but example holds.
    • do_not_redeem 2y ago
      Now the money question: can anyone come up with a benchmark where, due to the JIT, C++/CLI runs faster than normal C++ compiled for the same CPU?
      • bjoli 2y ago
        Writing a program where a jit version is faster than the aot version is just an exercise in knowing the limitations of AOT. People have been doing runtime code generation for a very long time for exactly this reason. A general implementation faster than, say, g++ is a completely different beast.
    • zabzonk 2y ago
      It is not C++ (or C) but a Microsoft invented language - which is OK, but don't confuse it with C++ anymore than MS have already done
      • pjmlp 2y ago
• pizlonator 2y ago
  > Java HotSpot runtime we're still waiting for even one system to produce the promised advantages. What promised advantages are you waiting on? There are lots of systems that have architectures that are similar to HotSpot, or that surpass it in some way. V8 is just one.
  • CamouflagedKiwi 2y ago
    There were many many statements made that JIT compilers could be faster than AOT compilers because they had more information to use at runtime - originally this was mostly aimed at Java/HotSpot which has not, in practice, significantly displaced languages like C or C++ (or these days Rust) from high-performance work.
    • pjmlp 2y ago
      I guess distributed systems and OS GUI frameworks aren't it then.
    • pizlonator 2y ago
      Yeah those statements were overly optimistic and I don’t think they’re representative of what most people in the JIT field think. It’s also not what I as a JIT engineer would have promised you. The actual promise is just: JITs make dynamic languages faster and they are better at doing that than AOTs. I think lots of systems have delivered on that promise.
      • CamouflagedKiwi 2y ago
        Yup, agreed, in the case of dynamic languages it's much clearer and the evidence is a lot more favourable. The linked article doesn't help here because the abstract only mentions Javascript in the context of their work to prove their concept, but the body of the paper is clearer that it is discussing JIT vs AOT in the context of Javascript specifically.
        • pizlonator 2y ago
• pjmlp 2y ago
  JVM implementations, especially those with PGO feedback loop across runs do quite well. Likewise modern Android, runs reasonably well with its mix of JIT, AOT with JIT PGO metadata, baseline profiles shared across devices via Play Store. The gold standard for anyone that actually cares about ultimate performance is hand written Assembly, naturally guided with a profilers capable to measure everything that the CPU is doing like VTune.
• titzer 2y ago
  > But the reality is that hand-optimized AoT builds remain the gold standard for performance work. It's considerably more complicated than that. After working in this area for 25 years, I have vacillated between extremes over decades-long arcs. The reality is much more nuanced than a four sentence HN comment. Profile and measure and stare at machine code. If you don't do that daily, it's hand waving and having hunches.
  • cogman10 2y ago
    I'd also point out that it's an ever-shifting landscape. What was slow yesterday might not be today. In my experience, while there are some negatives of the runtime selected, the vast majority of performance is won or lost at the algorithm level. It really doesn't matter that rust can be faster than ruby if you chose an O(n^3) algorithm. Rust will run the O(n^3) algorithm faster than ruby, for sure, but ruby will beat the pants off of rust if someone converts it into an O(n) algorithm. It only starts mattering if you've already have an O(n) algorithm. However, in my experience, a LOT of programmers are happy writing a n^3 and moving on to the next task without considering what this will do. for (i : foo) { for (j : foo) { for (k : foo) { bar(i, j, k) } } }
    • neonsunset 2y ago
      You may be underestimating the degree of difference in performance between Ruby and Rust. Here's comparison of Ruby with JS, and Rust is of course faster still: https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/node-ruby.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... If the code runs 100 times faster, it might just offset even highly inefficient implementation. > a LOT of programmers are happy writing a n^3 I have the same experience. Unfortunately, and this is an issue I keep fighting with in some .NET communities, languages like C, C++ and Rust tend to select for engineers which are more likely to care about writing reasonably efficient implementation. At the same time, higher-level languages sometimes can almost encourage the blindness to the real world model of computation, the execution implications be damned. In such languages you will encounter way more people who will write O(n^3) algorithm and will fight you tooth and nail to keep it that way because they have zero understanding of the fundamentals, wasting the heroic effort by the runtime/compiler to keep it running acceptably well.
• neonsunset 2y ago
  If you pit virtual-call-heavy code written in C++ against C#, C# will come out on top every single time, especially if you consume dynamically-linked dependencies or if you can't afford to wait until the heat death of the universe when all the LTO plugins finish their job. Or if you use SIMD-heavy path and your binary is built against, say, X86-64-v2/3 and the target supports AVX512, .NET will happily use the entirety of AVX512 thanks to JIT even when still using 256b-wide operations (i.e. bespoke path that uses Vector256) with AVX512VL. This tends to surpass what you can get out of runtime dispatch under LLVM. re: Java challenges - those stem from the JVM bytecode being a very difficult optimization target i.e. every call is virtual by default with complex dispatch strategy, everything is a heap-allocated object by default save for very few primitives, generics lose type information and are never monomorphized - PGO optimization through tiered compilation and resulting guarded devirtualization and object escape analysis is something that reclaims performance in Java and makes it acceptable. C and C++ with templates are a massively easier optimization target for GCC, and GCC does not operate under strict time constraints too. Therefore we have the results that we do. Also interesting data points here if you'd like to look at AOT capabilities of higher-level languages: https://benchmarksgame-team.pages.debian.net/benchmarksgame/box-plot-summary-charts.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
• twoodfin 2y ago
  Hand-optimized AoT builds with solid profile-based feedback, right?
• paulddraper 2y ago
  > we're still waiting for even one system to produce the promised advantages To be clear, successful JIT do runtime profiling+optimization, at significant benefit. But on net, JIT languages are slower. It is a valid question to ask whether AOT binaries can selectively use runtime optimizations, making them even faster.