There's usually a tradeoff between many small pauses and few big ones (but even with CMS there will be a point where the JVM needs to do a full GC). The usual

There's usually a tradeoff between many small pauses and few big ones (but even with CMS there will be a point where the JVM needs to do a full GC). The usual approach for low latency systems are to spend the GC pauses incrementally over time. But for us, even though we require consistent low latency, big pauses are perfectly fine because we can run several replicas of the system behind a load balancer. Not all systems can be designed to do this, and it usually implies a idempotent architecture where you can run the same transaction multiple times without any problems. But when it's possible, you can start guaranteeing low latency with a very high probability. Worst case: simply run the transaction on all instances and use the result from the first one that answers.