5 ms·

The only "abstraction penalty" of "running on a VM" (by which I think you mean using a JIT compiler), is the warmup time of waiting for the JIT.

by pron 9mo ago

andersmurphy 9mo ago
Its a statement of our times that this is getting down voted. JIT is so underrated.
stefs 9mo ago
in my opinion, this assertion suffers from the "sufficiently smart compiler" fallacy somewhat. https://wiki.c2.com/?SufficientlySmartCompiler https://wiki.c2.com/?SufficientlySmartCompiler
- remexre 9mo ago
  c++ certainly also has and needs a similarly sufficiently smart compiler to be compiled at all…
- sswatson 9mo ago
  The linked article makes a specific carveout for Java, on the grounds that its SufficientlySmartCompiler is real, not hypothetical.
- pron 9mo ago
  No, Java's existing compiler is very good, and it generates as good code as you'd want. There is definitely still a cost due to objects not being inlined in arrays yet (this will change soon) that impacts some programs, but in practice Java performs more-or-less the same as C++. In this case, however, it appears that the Java program may have been configured in a suboptimal way. I don't know how much of an impact it has here, but it can be very big.
  - galangalalgol 9mo ago
    Even benchmarks that allow for jit warmup consistently show java roughly half the speed of c/c++/rust. Is there something they are doing wrong? I've seen people write some really unusual java to eliminate all runtime allocations, but that was about latency, not throughput.
    - pron 9mo ago
      > Is there something they are doing wrong? Yes. The most common issues are heap misconfiguration (which is more important in Java than any compiler configuration in other languages) and that the benchmarks don't simulate realistic workloads in terms of both memory usage and concurrency. Another big issue is that the effort put into the program is not the same. Low-level languages do allow you to get better performance than Java if you put significant extra work to get it. Java aims to be "the fastest" for a "normal" amount of effort at the expense of losing some control that could translate to better performance in exchange for significantly more work, bot at initial development time, but especially during evolution/maintenance. E.g. I know of a project at one of the world's top 5 software companies where they wanted to migrate a real Java program to C++ or Rust to get better performance (it was probably Rust because there's some people out there who really want to to try Rust). Unsurprisingly, they got significantly worse performance (probably because low-level languages are not good at memory management when concurrency is at play, or at concurrency in general). But they wanted the experiment to be a success, so they put in a tonne of effort - I'm talking many months - hand-optimising the code, and in the end they managed to match Java's performance or even exceed it by a bit (but admitted it was ultimately wasted effort). If the performance of your Java program doesn't more-or-less match or even exceed the performance of a C++ (or other low level language) program then the cause is one of: 1. you've spent more effort optimising the other program, 2. you've misconfigured the Java program (probably a bad heap-size setting), or 3. the program relies on object flattening, which means the Java program will suffer from costly cache misses (until Valhalla arrives, which is expected to be very soon).
      - galangalalgol 9mo ago
        Has anyone done a fork of the benchmark game or plb2 to demonstrate the impacts of jit warmup and heap settings?
  - deleted 9mo ago
    [deleted]
xigoi 9mo ago
The true penalty of Java is that product types have to be heap-allocated, as there is no mechanism for stack-allocated product types.
- pron 9mo ago
  You're right that Java lacks inline types (although it's getting them really soon, now), but the main cost of that isn't because of stack allocation (because heap allocations in Java don't cost much more than stack allocations), but because cache misses due to objects not being inlined in arrays.
  - pron 9mo ago
    P.S. Even for flattened types, the "abstraction penalty", or, more precisely, its converse, the "concreteness penalty", in Java will be low, as you don't directly pick when an object is flattened. Instead, you declare whether a class cares about identity or not, and if not, the compiler will transparently choose whether and when to flatten the object, depending on how it's used.
- Someone 9mo ago
  > product types have to be heap-allocated Conceptually, that’s true, but a compiler is free to do things differently. For example, if escape analysis shows that an object allocated in a block never escapes the block, the optimizer can replace the object by local variables, one for each field in the object. And that’s not theoretical. https://www.bettercodebytes.com/allocation-elimination-when-the-java-keyword-new-doesnt-create-an-object/ https://www.bettercodebytes.com/allocation-elimination-when-..., https://medium.com/@souvanik.saha/are-java-objects-always-created-in-heap-memory-7ba1596143a6 https://medium.com/@souvanik.saha/are-java-objects-always-cr... show that it (sometimes) does.

The only "abstraction penalty" of "running on a VM" (by which I think you mean using a JIT compiler), is the warmup time of waiting for the JIT.

by pron 9mo ago

The only "abstraction penalty" of "running on a VM" (by which I think you mean using a JIT compiler), is the warmup time of waiting for the JIT.

andersmurphy 9mo ago

Its a statement of our times that this is getting down voted. JIT is so underrated.

stefs 9mo ago

in my opinion, this assertion suffers from the "sufficiently smart compiler" fallacy somewhat. https://wiki.c2.com/?SufficientlySmartCompiler https://wiki.c2.com/?SufficientlySmartCompiler

remexre 9mo ago
c++ certainly also has and needs a similarly sufficiently smart compiler to be compiled at all…
sswatson 9mo ago
The linked article makes a specific carveout for Java, on the grounds that its SufficientlySmartCompiler is real, not hypothetical.
pron 9mo ago
No, Java's existing compiler is very good, and it generates as good code as you'd want. There is definitely still a cost due to objects not being inlined in arrays yet (this will change soon) that impacts some programs, but in practice Java performs more-or-less the same as C++. In this case, however, it appears that the Java program may have been configured in a suboptimal way. I don't know how much of an impact it has here, but it can be very big.
- galangalalgol 9mo ago
  Even benchmarks that allow for jit warmup consistently show java roughly half the speed of c/c++/rust. Is there something they are doing wrong? I've seen people write some really unusual java to eliminate all runtime allocations, but that was about latency, not throughput.
  - pron 9mo ago
    > Is there something they are doing wrong? Yes. The most common issues are heap misconfiguration (which is more important in Java than any compiler configuration in other languages) and that the benchmarks don't simulate realistic workloads in terms of both memory usage and concurrency. Another big issue is that the effort put into the program is not the same. Low-level languages do allow you to get better performance than Java if you put significant extra work to get it. Java aims to be "the fastest" for a "normal" amount of effort at the expense of losing some control that could translate to better performance in exchange for significantly more work, bot at initial development time, but especially during evolution/maintenance. E.g. I know of a project at one of the world's top 5 software companies where they wanted to migrate a real Java program to C++ or Rust to get better performance (it was probably Rust because there's some people out there who really want to to try Rust). Unsurprisingly, they got significantly worse performance (probably because low-level languages are not good at memory management when concurrency is at play, or at concurrency in general). But they wanted the experiment to be a success, so they put in a tonne of effort - I'm talking many months - hand-optimising the code, and in the end they managed to match Java's performance or even exceed it by a bit (but admitted it was ultimately wasted effort). If the performance of your Java program doesn't more-or-less match or even exceed the performance of a C++ (or other low level language) program then the cause is one of: 1. you've spent more effort optimising the other program, 2. you've misconfigured the Java program (probably a bad heap-size setting), or 3. the program relies on object flattening, which means the Java program will suffer from costly cache misses (until Valhalla arrives, which is expected to be very soon).
    - galangalalgol 9mo ago
      Has anyone done a fork of the benchmark game or plb2 to demonstrate the impacts of jit warmup and heap settings?
- deleted 9mo ago
  [deleted]

xigoi 9mo ago

The true penalty of Java is that product types have to be heap-allocated, as there is no mechanism for stack-allocated product types.

pron 9mo ago
You're right that Java lacks inline types (although it's getting them really soon, now), but the main cost of that isn't because of stack allocation (because heap allocations in Java don't cost much more than stack allocations), but because cache misses due to objects not being inlined in arrays.
- pron 9mo ago
  P.S. Even for flattened types, the "abstraction penalty", or, more precisely, its converse, the "concreteness penalty", in Java will be low, as you don't directly pick when an object is flattened. Instead, you declare whether a class cares about identity or not, and if not, the compiler will transparently choose whether and when to flatten the object, depending on how it's used.
Someone 9mo ago
> product types have to be heap-allocated Conceptually, that’s true, but a compiler is free to do things differently. For example, if escape analysis shows that an object allocated in a block never escapes the block, the optimizer can replace the object by local variables, one for each field in the object. And that’s not theoretical. https://www.bettercodebytes.com/allocation-elimination-when-the-java-keyword-new-doesnt-create-an-object/ https://www.bettercodebytes.com/allocation-elimination-when-..., https://medium.com/@souvanik.saha/are-java-objects-always-created-in-heap-memory-7ba1596143a6 https://medium.com/@souvanik.saha/are-java-objects-always-cr... show that it (sometimes) does.