5 ms·

There’s a distinct difference between Java and Rust/C++ which is that Java has a stop-the-world-GC and rust/c++ don’t. And in large scale systems, Java’s GC is

There’s a distinct difference between Java and Rust/C++ which is that Java has a stop-the-world-GC and rust/c++ don’t. And in large scale systems, Java’s GC is usually the limiting factor.

nlitened 5y ago
If I understand correctly, modern Java has multiple GC engines available (that can be chosen with a command-line switch), some of which are either fully concurrent or with sub-millisecond stop times.
pjmlp 5y ago
Yeah, like 1ms when handling heaps of 3TB sizes, when most people are still complaining about 16 GB on laptops, I can live with that.
- Diggsey 5y ago
  Firstly, 1ms for a 3TB heap size is very unrealistic even for modern JVMs. Secondly, the figures that are often quoted for how long the program must be stopped are for the "happy path" where enough memory can be freed that the program can continue without issue. In practice, what often happens with large heap sizes is that the GC starts lagging behind because there's not enough memory head-room. In these cases, the GC stops the world for much longer, because the program hangs on a memory allocation that cannot be satisfied until the GC completes. This primarily affects programs that have high rates of allocation. You can avoid this in Java with effort, but you end up with very non-idiomatic code.
  - pjmlp 5y ago
    Enjoy, https://malloc.se/blog/zgc-jdk16 https://malloc.se/blog/zgc-jdk16 I guess Oracle, IBM, Azul, PTC, Microsoft can provide the necessary data about their pauseless GC implementantions to any relevant customer.
  - native_samples 5y ago
    Modern JVMs have a GC that has O(1) pause times, often of less than 1msec regardless of heap size. In ZGC and Shenandoah, GC is fully concurrent and pauses are basically imperceptible. Even walking the stacks to locate pointers into the heap is done whilst the application is still running.
nobleach 5y ago
Newer versions of Java have more interesting GC implementations that reduce that "stop the world, swab the decks" performance. The fine engineers working on the JVM keep finding ways to incrementally garbage collect. It's getting to a point where it's imperceptible in some applications. Granted, if you're writing high-speed/high-frequency trading apps for Wall Street, Java still isn't there. Rust could be the logical next step in that arena. (Although, most C++ devs I know in that space don't hate C++)
- lordnacho 5y ago
  > if you're writing high-speed/high-frequency trading apps for Wall Street, Java still isn't there A fair few trading system job ads seem to be for Java. I also wrote code that faced a Java server at various banks. Seems to be fast enough for certain market related applications.
  - tuxman 5y ago
    From my understanding of previous talks on this, a fair amount of HFT's and trading firms use Java. The trick is that the trading day is a well-defined window with clear starts and stops. So if you load up your server with huge amounts of memory, you can get away with never calling GC during the trading day. The GC can be paused until after hours or just kill the program and start it up before market open the next day.
    - nobleach 5y ago
      One solution I had been dying to implement is the Erlang style of letting a Kubernetes POD OOM, and just respawning it. If the app performs well right up until it runs out of memory, I see no reason to have a GC turned on at all. (obviously there ARE reasons, I just wanted to try this approach)
  - magila 5y ago
    The use of Java in trading systems is widely misunderstood. Java is typically used to implement the high level business logic where microsecond latency is not required. The output of the Java service is feed to another system which handles the actual trade execution. That part is typically done with C++, or FPGAs driven by C++.

There’s a distinct difference between Java and Rust/C++ which is that Java has a stop-the-world-GC and rust/c++ don’t. And in large scale systems, Java’s GC is usually the limiting factor.

nlitened 5y ago

If I understand correctly, modern Java has multiple GC engines available (that can be chosen with a command-line switch), some of which are either fully concurrent or with sub-millisecond stop times.

pjmlp 5y ago

Yeah, like 1ms when handling heaps of 3TB sizes, when most people are still complaining about 16 GB on laptops, I can live with that.

Diggsey 5y ago
Firstly, 1ms for a 3TB heap size is very unrealistic even for modern JVMs. Secondly, the figures that are often quoted for how long the program must be stopped are for the "happy path" where enough memory can be freed that the program can continue without issue. In practice, what often happens with large heap sizes is that the GC starts lagging behind because there's not enough memory head-room. In these cases, the GC stops the world for much longer, because the program hangs on a memory allocation that cannot be satisfied until the GC completes. This primarily affects programs that have high rates of allocation. You can avoid this in Java with effort, but you end up with very non-idiomatic code.
- pjmlp 5y ago
  Enjoy, https://malloc.se/blog/zgc-jdk16 https://malloc.se/blog/zgc-jdk16 I guess Oracle, IBM, Azul, PTC, Microsoft can provide the necessary data about their pauseless GC implementantions to any relevant customer.
- native_samples 5y ago
  Modern JVMs have a GC that has O(1) pause times, often of less than 1msec regardless of heap size. In ZGC and Shenandoah, GC is fully concurrent and pauses are basically imperceptible. Even walking the stacks to locate pointers into the heap is done whilst the application is still running.

nobleach 5y ago

Newer versions of Java have more interesting GC implementations that reduce that "stop the world, swab the decks" performance. The fine engineers working on the JVM keep finding ways to incrementally garbage collect. It's getting to a point where it's imperceptible in some applications. Granted, if you're writing high-speed/high-frequency trading apps for Wall Street, Java still isn't there. Rust could be the logical next step in that arena. (Although, most C++ devs I know in that space don't hate C++)

lordnacho 5y ago
> if you're writing high-speed/high-frequency trading apps for Wall Street, Java still isn't there A fair few trading system job ads seem to be for Java. I also wrote code that faced a Java server at various banks. Seems to be fast enough for certain market related applications.
- tuxman 5y ago
  From my understanding of previous talks on this, a fair amount of HFT's and trading firms use Java. The trick is that the trading day is a well-defined window with clear starts and stops. So if you load up your server with huge amounts of memory, you can get away with never calling GC during the trading day. The GC can be paused until after hours or just kill the program and start it up before market open the next day.
  - nobleach 5y ago
    One solution I had been dying to implement is the Erlang style of letting a Kubernetes POD OOM, and just respawning it. If the app performs well right up until it runs out of memory, I see no reason to have a GC turned on at all. (obviously there ARE reasons, I just wanted to try this approach)
- magila 5y ago
  The use of Java in trading systems is widely misunderstood. Java is typically used to implement the high level business logic where microsecond latency is not required. The output of the Java service is feed to another system which handles the actual trade execution. That part is typically done with C++, or FPGAs driven by C++.