3 ms·

A microbenchmark like this is not very conclusive. The G1 collector is designed to handle the heap fragmentation problem and thus reduce the maximum pause time

by sehugg 15y ago

A microbenchmark like this is not very conclusive. The G1 collector is designed to handle the heap fragmentation problem and thus reduce the maximum pause time due to stop-the-world GC. It's not designed for maximum throughput. Each GC algorithm has its worst-case behavior; saying "avoid at all costs" because of a single scenario is not very helpful. Also, what were the JVM flags for each test? All I can see in these graphs is the bump at the beginning before the heap has sized to a stable level.

NerdsCentral 15y ago
I hope that the article does not imply that it is conclusive but rather that it is cause for concern. The flags were only those required to set the garbage collector - everything else is default. If you want to try other settings, the code is there to use :)
- puredanger 15y ago
  I don't know why I would ever judge the performance of non-tuned GC for any collector especially when the app is one designed to make life hard for the collector? There is no useful conclusion you can draw from this test.
  - deleted 15y ago
    [deleted]
    - scott_s 15y ago
      I think his overall point is that all GCs are susceptible to specially designed adversaries. And it seems like you designed this program to be an adversary. So it's interesting, but it doesn't mean the GC is bad, unless you can provide evidence that the memory behavior of this program is typical of real applications. (You argue that, but an argument is not evidence.) Or, in the other direction, you could experiment with how other GCs behave with this program, and explain what, if any of the differences are. If the differences are huge, and you can explain them, then perhaps that's something to report.
      - NerdsCentral 15y ago
        My point is that I never said the GC was bad. I said I showed up issue with it. As I explained in detail in my post - I was actually researching something else. BTW I did experiment with 2 other garbage collectors - the default and the incremental and gave results for all three, which I think is quite thorough given that I was actually researching something else. I would be just great if someone who is coming at what I am saying with description of the weakness would give a suggestion rather than just shoot down the finding which I never said was anything other than just a finding.
        scott_s 15y ago
        This is how research works. You report your results, and people critique it. Part of that critique is methodology. That you were researching something else, or that we're not giving suggestions to improve it is irrelevant. And you're right, you did compare it to two other collectors - I initially thought it was the same collector with different settings. So what are the assumptions these collectors make? Presumably the different collectors are supposed to be used under different scenarios. Does your benchmark fit into the scenarios the G1 collector should be used? Why or why not?
        NerdsCentral 15y ago
        Actually, it is relevant that I was researching something else. I have published papers on research; in that process one crosses all the ts etc. In this case I am saying - here it is - if you are interested you can run with it. That is very different and so I think it is quite reasonable to expect constructive rather than negative criticism. As for the G1, I have chosen to continue and look at the issue in more detail. So far, it would appear that I cannot find a situation where it offers a benefit over other collectors. However, I am happy to accept that there might be one out there. I will switch over to the Popper approach and attempt to disprove that the G1 performs badly. If I get conclusive result either way - I will report on it.
        scott_s 15y ago
NerdsCentral 15y ago
Oh - also - whilst I get that the G1 is not designed for maximum throughput - note that it has longer maximum pause as well. I did mention this in the post. So, for this situation, it fails in both regards. Now, whilst this is a benchmark and not a real world system, it is cause for some concern that the G1 collector is slowing down code execution so very much.

by sehugg 15y ago

NerdsCentral 15y ago

I hope that the article does not imply that it is conclusive but rather that it is cause for concern. The flags were only those required to set the garbage collector - everything else is default. If you want to try other settings, the code is there to use :)

puredanger 15y ago
I don't know why I would ever judge the performance of non-tuned GC for any collector especially when the app is one designed to make life hard for the collector? There is no useful conclusion you can draw from this test.
- deleted 15y ago
  [deleted]
  - scott_s 15y ago
    I think his overall point is that all GCs are susceptible to specially designed adversaries. And it seems like you designed this program to be an adversary. So it's interesting, but it doesn't mean the GC is bad, unless you can provide evidence that the memory behavior of this program is typical of real applications. (You argue that, but an argument is not evidence.) Or, in the other direction, you could experiment with how other GCs behave with this program, and explain what, if any of the differences are. If the differences are huge, and you can explain them, then perhaps that's something to report.
    - NerdsCentral 15y ago
      My point is that I never said the GC was bad. I said I showed up issue with it. As I explained in detail in my post - I was actually researching something else. BTW I did experiment with 2 other garbage collectors - the default and the incremental and gave results for all three, which I think is quite thorough given that I was actually researching something else. I would be just great if someone who is coming at what I am saying with description of the weakness would give a suggestion rather than just shoot down the finding which I never said was anything other than just a finding.
      - scott_s 15y ago
        This is how research works. You report your results, and people critique it. Part of that critique is methodology. That you were researching something else, or that we're not giving suggestions to improve it is irrelevant. And you're right, you did compare it to two other collectors - I initially thought it was the same collector with different settings. So what are the assumptions these collectors make? Presumably the different collectors are supposed to be used under different scenarios. Does your benchmark fit into the scenarios the G1 collector should be used? Why or why not?
        NerdsCentral 15y ago
        Actually, it is relevant that I was researching something else. I have published papers on research; in that process one crosses all the ts etc. In this case I am saying - here it is - if you are interested you can run with it. That is very different and so I think it is quite reasonable to expect constructive rather than negative criticism. As for the G1, I have chosen to continue and look at the issue in more detail. So far, it would appear that I cannot find a situation where it offers a benefit over other collectors. However, I am happy to accept that there might be one out there. I will switch over to the Popper approach and attempt to disprove that the G1 performs badly. If I get conclusive result either way - I will report on it.
        scott_s 15y ago

NerdsCentral 15y ago

Oh - also - whilst I get that the G1 is not designed for maximum throughput - note that it has longer maximum pause as well. I did mention this in the post. So, for this situation, it fails in both regards. Now, whilst this is a benchmark and not a real world system, it is cause for some concern that the G1 collector is slowing down code execution so very much.