4 ms·

Even if you completely disable GC, Java is still allocating tons of ephemeral objects on the heap. Which in turn is leading to expensive and unpredictable page

Even if you completely disable GC, Java is still allocating tons of ephemeral objects on the heap. Which in turn is leading to expensive and unpredictable page faults. In contrast, C++'s default is to allocate objects on the heap unless you knowingly call new/malloc. Of course, it's possible to write Java in such a way to minimize this type of heap-thrashing. But by that point, you're already doing the equivalent of C++'s manual memory book-keeping anyway. It's not like "just provision a ton of memory and turn off GC" is a free lunch.

fh973 6y ago
That's covered in Java by escape analysis and allocation on the stack.
- bluestreak 6y ago
  We have written a database in zero GC java and one thing I have not seen any evidence of "escape analysis". @State(Scope.Thread) @BenchmarkMode(Mode.AverageTime) @OutputTimeUnit(TimeUnit.NANOSECONDS) public class EscBenchmark { Rnd rnd = new Rnd(); public static void main(String[] args) throws RunnerException { Options opt = new OptionsBuilder() .include(EscBenchmark.class.getSimpleName()) .warmupIterations(5) .measurementIterations(5) .forks(1) .addProfiler(GCProfiler.class) .build(); new Runner(opt).run(); } @Benchmark public int testEscapeAnalysis() { int[] tuple = {0, 2}; // esc analysis? where are you? return tuple[rnd.nextPositiveInt() % 2]; } } And the output of GC profiler: Benchmark Mode Cnt Score Error Units EscBenchmark.testEscapeAnalysis avgt 5 8.234 ± 0.029 ns/op EscBenchmark.testEscapeAnalysis:·gc.alloc.rate avgt 5 2647.216 ± 9.275 MB/sec EscBenchmark.testEscapeAnalysis:·gc.alloc.rate.norm avgt 5 24.000 ± 0.001 B/op EscBenchmark.testEscapeAnalysis:·gc.churn.G1_Eden_Space avgt 5 2643.140 ± 177.137 MB/sec EscBenchmark.testEscapeAnalysis:·gc.churn.G1_Eden_Space.norm avgt 5 23.963 ± 1.613 B/op EscBenchmark.testEscapeAnalysis:·gc.count avgt 5 157.000 counts EscBenchmark.testEscapeAnalysis:·gc.time avgt 5 103.000 ms
  - pdpi 6y ago
    Scalar replacement has several requirements, and escape analysis is only part of the story. One of the restrictions with scalar replacement for arrays is that indexing has to be constant at JIT time, which your example clearly isn't.
    - whizzter 6y ago
      The point would be that escape analysis could have proven that the array was not used outside of the function and allocated it fully on the stack, thus not providing any GC pressure. Scalar replacement isn't the point of interest really.
    - deleted 6y ago
      [deleted]
  - hawk_ 6y ago
    Which version of Java are you using? And what is Rnd? The class included with JDK is Random. We rely on escape analysis to elide such object creation for us and it works reasonably well. If something this trivial doesn't work for you, file bug with them. We have had success with that as well.
    - bluestreak 6y ago
      This ran on Java 11. This isn't so much of an issue for us. We are trying to avoid allocations, even as trivial as those. There are other examples that allocated where they should not, for example this lambda will allocate. () -> System.out.println(1) I lost hope in escape analysis quite frankly. Rnd is something I have written because Java's Random is slow and clunky. https://github.com/questdb/questdb/blob/master/core/src/main/java/io/questdb/std/Rnd.java https://github.com/questdb/questdb/blob/master/core/src/main...
      - pjmlp 6y ago
        Escape analysis is very JVM implementation specific, but inline classes should finally sort it out.
- pdpi 6y ago
  The JVM does scalar replacement, not stack allocation. The former is much more limited in functionality, and much more finicky. You also have about zero control over it, compared to, say, C# structs.
paulfurtado 6y ago
If you add -XX:+AlwaysPreTouch to the JVM arguments, the JVM will pre-touch the entirety of the heap at startup time to avoid unpredictable page faults through the life of the application. I'd imagine other HFT companies may also pay for the Azul JVM which goes even further and comes with a kernel module that the JVM coordinates with for memory allocation. The kernel module pre-reserves x% of system memory at the time it is loaded.
- nobleach 6y ago
  Is the Azul JVM open source? I thought it was just the support which costed money. (I'm just not familiar enough with it)
  - whizzter 6y ago
    It is commercial, and sadly their GC algorithms are patented. I'm really anxious for those patents to start running out because the functionality is quite clever. Basically one main reason you need to stop the world in common GC's right now (and thus cause pauses) is because if you want to compact memory a GC thread has no idea if other threads reads/writes to an object while moving leading to worst case corrupt/lost writes. What Azul does is to make the pages in question invalid for reading/writing so any access to them would page-fault and then proceeds to move the objects without worrying about other threads. Since the hardware provides the page-fault "for-free" this protection doesn't cost anything in terms of runtime performance in the common case (compared to the new ZGC made by Oracle that has read-write barriers that steals some mutator performance), worst case if another thread tries to read/write memory that is being moved a specialized page-fault handler detects if this was an page that was in motion and can then do a slower read-write but ONLY in the seldom cases where this occurs (compared to all the time of f.ex. ZGC)
    - abledon 6y ago
      why does the hardware provide page-fault for free again?
      - 6y ago

fh973 6y ago

That's covered in Java by escape analysis and allocation on the stack.

bluestreak 6y ago
We have written a database in zero GC java and one thing I have not seen any evidence of "escape analysis". @State(Scope.Thread) @BenchmarkMode(Mode.AverageTime) @OutputTimeUnit(TimeUnit.NANOSECONDS) public class EscBenchmark { Rnd rnd = new Rnd(); public static void main(String[] args) throws RunnerException { Options opt = new OptionsBuilder() .include(EscBenchmark.class.getSimpleName()) .warmupIterations(5) .measurementIterations(5) .forks(1) .addProfiler(GCProfiler.class) .build(); new Runner(opt).run(); } @Benchmark public int testEscapeAnalysis() { int[] tuple = {0, 2}; // esc analysis? where are you? return tuple[rnd.nextPositiveInt() % 2]; } } And the output of GC profiler: Benchmark Mode Cnt Score Error Units EscBenchmark.testEscapeAnalysis avgt 5 8.234 ± 0.029 ns/op EscBenchmark.testEscapeAnalysis:·gc.alloc.rate avgt 5 2647.216 ± 9.275 MB/sec EscBenchmark.testEscapeAnalysis:·gc.alloc.rate.norm avgt 5 24.000 ± 0.001 B/op EscBenchmark.testEscapeAnalysis:·gc.churn.G1_Eden_Space avgt 5 2643.140 ± 177.137 MB/sec EscBenchmark.testEscapeAnalysis:·gc.churn.G1_Eden_Space.norm avgt 5 23.963 ± 1.613 B/op EscBenchmark.testEscapeAnalysis:·gc.count avgt 5 157.000 counts EscBenchmark.testEscapeAnalysis:·gc.time avgt 5 103.000 ms
- pdpi 6y ago
  Scalar replacement has several requirements, and escape analysis is only part of the story. One of the restrictions with scalar replacement for arrays is that indexing has to be constant at JIT time, which your example clearly isn't.
  - whizzter 6y ago
    The point would be that escape analysis could have proven that the array was not used outside of the function and allocated it fully on the stack, thus not providing any GC pressure. Scalar replacement isn't the point of interest really.
  - deleted 6y ago
    [deleted]
- hawk_ 6y ago
  Which version of Java are you using? And what is Rnd? The class included with JDK is Random. We rely on escape analysis to elide such object creation for us and it works reasonably well. If something this trivial doesn't work for you, file bug with them. We have had success with that as well.
  - bluestreak 6y ago
    This ran on Java 11. This isn't so much of an issue for us. We are trying to avoid allocations, even as trivial as those. There are other examples that allocated where they should not, for example this lambda will allocate. () -> System.out.println(1) I lost hope in escape analysis quite frankly. Rnd is something I have written because Java's Random is slow and clunky. https://github.com/questdb/questdb/blob/master/core/src/main/java/io/questdb/std/Rnd.java https://github.com/questdb/questdb/blob/master/core/src/main...
    - pjmlp 6y ago
      Escape analysis is very JVM implementation specific, but inline classes should finally sort it out.
pdpi 6y ago
The JVM does scalar replacement, not stack allocation. The former is much more limited in functionality, and much more finicky. You also have about zero control over it, compared to, say, C# structs.

paulfurtado 6y ago

If you add -XX:+AlwaysPreTouch to the JVM arguments, the JVM will pre-touch the entirety of the heap at startup time to avoid unpredictable page faults through the life of the application. I'd imagine other HFT companies may also pay for the Azul JVM which goes even further and comes with a kernel module that the JVM coordinates with for memory allocation. The kernel module pre-reserves x% of system memory at the time it is loaded.

nobleach 6y ago
Is the Azul JVM open source? I thought it was just the support which costed money. (I'm just not familiar enough with it)
- whizzter 6y ago
  It is commercial, and sadly their GC algorithms are patented. I'm really anxious for those patents to start running out because the functionality is quite clever. Basically one main reason you need to stop the world in common GC's right now (and thus cause pauses) is because if you want to compact memory a GC thread has no idea if other threads reads/writes to an object while moving leading to worst case corrupt/lost writes. What Azul does is to make the pages in question invalid for reading/writing so any access to them would page-fault and then proceeds to move the objects without worrying about other threads. Since the hardware provides the page-fault "for-free" this protection doesn't cost anything in terms of runtime performance in the common case (compared to the new ZGC made by Oracle that has read-write barriers that steals some mutator performance), worst case if another thread tries to read/write memory that is being moved a specialized page-fault handler detects if this was an page that was in motion and can then do a slower read-write but ONLY in the seldom cases where this occurs (compared to all the time of f.ex. ZGC)
  - abledon 6y ago
    why does the hardware provide page-fault for free again?
    - 6y ago