4 ms·

I worked for a company where we had a 100-150 microseconds order latency. We had to tune jvm to extremes. But it is very doable to optimize Java gc for low gc p

I worked for a company where we had a 100-150 microseconds order latency. We had to tune jvm to extremes. But it is very doable to optimize Java gc for low gc pause times.

snaky 8y ago
If you used to tune JVM only, that might be interesting. Most often the Java code is what being tuned - to the point where there is no point to use Java - with (almost) no gc, off-the-heap memory allocations and all that jazz.
- dnomad 8y ago
  This doesn't make much sense. Using object pools and and off-heap data structures doesn't warp the code that much, if at all. You still end up with pretty much all clean, simple Java code. I've seen (10+ years old) low-latency Java apps where people avoid using any OO (ie everything is static methods)... and even then the code was still often easier to maintain and understand than equivalent C++ applications. Nobody does this any more anyways.
- branchless 8y ago
  There are more java developers you can educate to do this than good c++ dev. Plus the tooling is better in java.
- phakding 8y ago
  It's not either/or. Everything needs to be tuned to achieve the final latency requirements. This includes tuning java code to eliminate creating "garbage", immutable string objects mostly, unnecessary object creation, creating short lived objects compared to long lived etc. The other part right sizing jvm generational partitions, using right gc algo etc. Next part would be to find out the code path with lowest latency requirement and making it hot/jitting. Even next part would be to tune OS for things like thread pinning etc etc. You can go further and tune the network layer. Every little thing counts. You can go even further and employ what's called mechanical sympathy (https://mechanical-sympathy.blogspot.com/ https://mechanical-sympathy.blogspot.com/) and write your code in a way that makes it easier for the processor to process faster. I've never worked with application that use off-heap allocations or no-gc requirements.
eikenberry 8y ago
With those latency requirements why did you use Java?
- dnomad 8y ago
  Plenty of people are writing low-latency trading applications in Java where the latency budget is under 100 microseconds. This has been the case for several years now. It's by no means new or even especially difficult these days. The resulting code is far more maintainable and robust than C++ solutions.
  - devonkim 8y ago
    I’ve heard a pretty effective, lazy strategy for this scenario is to oversize the heap to avoid GCs through the trading day and to simply restart the application before the next day. Unsure if it’s viable for all trading platforms, but it makes sense for some I’m sure.
    - hermitdev 8y ago
      Depends on your order volume. Sure an oversized heap may work for low order volume, but if you're dealing with high volume, even hundreds of gigabytes of RAM isn't even for a trading day. And yeah, we restart everyday (we only trade US equities, so plenty of downtime for a restart).
  - hermitdev 8y ago
    I've had the opposite experience. For green-field applications, it's far easier to write the application in C++ than Java & meet the latency requirements. Java requires far too much tuning, where as C++, from the get go, you can generally just glance at the code and have a good idea of the latency. We're currently fighting a Java app that in general has decent latency (10s of usecs), but has outliers of greater than a second when GC kicks in. We don't have that issue with the C++ components of our trading system.
    - phakding 8y ago

I worked for a company where we had a 100-150 microseconds order latency. We had to tune jvm to extremes. But it is very doable to optimize Java gc for low gc pause times.

snaky 8y ago

If you used to tune JVM only, that might be interesting. Most often the Java code is what being tuned - to the point where there is no point to use Java - with (almost) no gc, off-the-heap memory allocations and all that jazz.

dnomad 8y ago
This doesn't make much sense. Using object pools and and off-heap data structures doesn't warp the code that much, if at all. You still end up with pretty much all clean, simple Java code. I've seen (10+ years old) low-latency Java apps where people avoid using any OO (ie everything is static methods)... and even then the code was still often easier to maintain and understand than equivalent C++ applications. Nobody does this any more anyways.
branchless 8y ago
There are more java developers you can educate to do this than good c++ dev. Plus the tooling is better in java.
phakding 8y ago
It's not either/or. Everything needs to be tuned to achieve the final latency requirements. This includes tuning java code to eliminate creating "garbage", immutable string objects mostly, unnecessary object creation, creating short lived objects compared to long lived etc. The other part right sizing jvm generational partitions, using right gc algo etc. Next part would be to find out the code path with lowest latency requirement and making it hot/jitting. Even next part would be to tune OS for things like thread pinning etc etc. You can go further and tune the network layer. Every little thing counts. You can go even further and employ what's called mechanical sympathy (https://mechanical-sympathy.blogspot.com/ https://mechanical-sympathy.blogspot.com/) and write your code in a way that makes it easier for the processor to process faster. I've never worked with application that use off-heap allocations or no-gc requirements.

eikenberry 8y ago

With those latency requirements why did you use Java?

dnomad 8y ago
Plenty of people are writing low-latency trading applications in Java where the latency budget is under 100 microseconds. This has been the case for several years now. It's by no means new or even especially difficult these days. The resulting code is far more maintainable and robust than C++ solutions.
- devonkim 8y ago
  I’ve heard a pretty effective, lazy strategy for this scenario is to oversize the heap to avoid GCs through the trading day and to simply restart the application before the next day. Unsure if it’s viable for all trading platforms, but it makes sense for some I’m sure.
  - hermitdev 8y ago
    Depends on your order volume. Sure an oversized heap may work for low order volume, but if you're dealing with high volume, even hundreds of gigabytes of RAM isn't even for a trading day. And yeah, we restart everyday (we only trade US equities, so plenty of downtime for a restart).
- hermitdev 8y ago
  I've had the opposite experience. For green-field applications, it's far easier to write the application in C++ than Java & meet the latency requirements. Java requires far too much tuning, where as C++, from the get go, you can generally just glance at the code and have a good idea of the latency. We're currently fighting a Java app that in general has decent latency (10s of usecs), but has outliers of greater than a second when GC kicks in. We don't have that issue with the C++ components of our trading system.
  - phakding 8y ago