Had the same fix to a wildly different situation. A long running Spark job that accepts 100s of jars and runs 1000s of stages over its lifetime was having inter

Had the same fix to a wildly different situation. A long running Spark job that accepts 100s of jars and runs 1000s of stages over its lifetime was having intermittent massive GC pauses. Too intermittent as it happens, practically periodic. Turns out Spark runs System.GC() every 30 minutes by default. DisableExplicitGC fixed everything right up.