4 ms·

The Java GC uses a mark and sweep algorithm that has to stop the world to collect the garbage. As the number of items on the heap goes up, there are a higher nu

by kt9 14y ago

The Java GC uses a mark and sweep algorithm that has to stop the world to collect the garbage. As the number of items on the heap goes up, there are a higher number of valid pointers (less garbage) and the GC takes a long time to find even a little bit of garbage to clean up. So each GC run takes longer and there are more of them as the heap fills up. Given that the world is stopped during each GC run things take longer and hence the insertion rate is slower.

qwerta 14y ago
Exactly. If there is too many pointers GC becomes very slow.
spartango 14y ago
Java's garbage collector has advanced from pure stop-the-world mark-and-sweep collection over the years. In Java 7, a new generational garbage collector became the default, working concurrently with application threads to minimize GC pauses and collection time on multicore systems. If you're curious, check out these resources about how it works: * http://www.drdobbs.com/jvm/g1-javas-garbage-first-garbage-collector/219401061 http://www.drdobbs.com/jvm/g1-javas-garbage-first-garbage-co... * http://www.oracle.com/technetwork/java/javase/tech/g1-intro-jsp-135488.html http://www.oracle.com/technetwork/java/javase/tech/g1-intro-... Note also that the JRE ships with multiple garbage collectors that you can select manually, and that some of the collectors tune themselves to match the application.
andrewvc 14y ago
What? In java there are multiple GCs available: ConcMarkSwep, G1GC, ParallelGC, and I believe a couple more. All you have to do is set the right CLI flag. From the concurrent mark and sweep docs: "The concurrent mark sweep collector, also known as the concurrent collector or CMS, is targeted at applications that are sensitive to garbage collection pauses. It performs most garbage collection activity concurrently, i.e., while the application threads are running, to keep garbage collection-induced pauses short. The key performance enhancements made to the CMS collector in JDK 6 are outlined below. See the documents referenced below for more detailed information on these changes, the CMS collector, and garbage collection in HotSpot." It does have to stop the world sometimes but those times should be quite rare!
- mbell 14y ago
  If the CMS collector can't 'keep up', meaning it doesn't think it can avoid running out of heap space based on its duty cycle and the current memory state, it will do a full on complete, stop the world GC. Based on the article I'm guessing this is what is kicking in.
  - andrewvc 14y ago
    They never mentioned using CMS, and CMS isn't the default GC, so I assume they weren't using it. CMS actually isn't the highest throughput GC, it's there for low-pause systems where there are extra cores available to perform GC in the background.
    - mbell 14y ago
      My point was that none of the GCs completely avoid full on stop everything, sweep and compact all the things GC cycles. They will all do this if they are in a position where they think the heap will be exhausted (specifically when a concurrent mode failure occurs). There really isn't anything else a GC can do in this situation other than throw an OOM exception. Thus any time your pushing the heap to its limits, regardless of what GC you use, you will see complete GC sweeps of the old generation. If you want stricter failure requirements use GCTimeLimit and GCHeapFreeLimit. By default the JVM with throw an OOM error if it spends 98% of execution time in the GC and frees only 2% of the memory, GCTimeLimit and GCHeapFreeLimit switches allow this these values to be changed. Also only the 'stop the world' portions of a collection apply to the execution time limit, concurrent phases do not. So if you managed to keep the collector in concurrent mode (by cranking up the duty cycle for example) then only the 2 mini-pauses will count even if the concurrent portion were now pegging a CPU core at 100%.

by kt9 14y ago

qwerta 14y ago

Exactly. If there is too many pointers GC becomes very slow.

spartango 14y ago

Java's garbage collector has advanced from pure stop-the-world mark-and-sweep collection over the years. In Java 7, a new generational garbage collector became the default, working concurrently with application threads to minimize GC pauses and collection time on multicore systems. If you're curious, check out these resources about how it works: * http://www.drdobbs.com/jvm/g1-javas-garbage-first-garbage-collector/219401061 http://www.drdobbs.com/jvm/g1-javas-garbage-first-garbage-co... * http://www.oracle.com/technetwork/java/javase/tech/g1-intro-jsp-135488.html http://www.oracle.com/technetwork/java/javase/tech/g1-intro-... Note also that the JRE ships with multiple garbage collectors that you can select manually, and that some of the collectors tune themselves to match the application.

andrewvc 14y ago

What? In java there are multiple GCs available: ConcMarkSwep, G1GC, ParallelGC, and I believe a couple more. All you have to do is set the right CLI flag. From the concurrent mark and sweep docs: "The concurrent mark sweep collector, also known as the concurrent collector or CMS, is targeted at applications that are sensitive to garbage collection pauses. It performs most garbage collection activity concurrently, i.e., while the application threads are running, to keep garbage collection-induced pauses short. The key performance enhancements made to the CMS collector in JDK 6 are outlined below. See the documents referenced below for more detailed information on these changes, the CMS collector, and garbage collection in HotSpot." It does have to stop the world sometimes but those times should be quite rare!

mbell 14y ago
If the CMS collector can't 'keep up', meaning it doesn't think it can avoid running out of heap space based on its duty cycle and the current memory state, it will do a full on complete, stop the world GC. Based on the article I'm guessing this is what is kicking in.
- andrewvc 14y ago
  They never mentioned using CMS, and CMS isn't the default GC, so I assume they weren't using it. CMS actually isn't the highest throughput GC, it's there for low-pause systems where there are extra cores available to perform GC in the background.
  - mbell 14y ago
    My point was that none of the GCs completely avoid full on stop everything, sweep and compact all the things GC cycles. They will all do this if they are in a position where they think the heap will be exhausted (specifically when a concurrent mode failure occurs). There really isn't anything else a GC can do in this situation other than throw an OOM exception. Thus any time your pushing the heap to its limits, regardless of what GC you use, you will see complete GC sweeps of the old generation. If you want stricter failure requirements use GCTimeLimit and GCHeapFreeLimit. By default the JVM with throw an OOM error if it spends 98% of execution time in the GC and frees only 2% of the memory, GCTimeLimit and GCHeapFreeLimit switches allow this these values to be changed. Also only the 'stop the world' portions of a collection apply to the execution time limit, concurrent phases do not. So if you managed to keep the collector in concurrent mode (by cranking up the duty cycle for example) then only the 2 mini-pauses will count even if the concurrent portion were now pegging a CPU core at 100%.