4 ms·

How do you explain the throughput and latency numbers posted by https://github.com/ixy-languages/ixy-languages https://github.com/ixy-languages/ixy-languages? G

by zeeboo 7y ago

How do you explain the throughput and latency numbers posted by https://github.com/ixy-languages/ixy-languages https://github.com/ixy-languages/ixy-languages? Go is beating Java in both latency and throughput, and the author attempted many different Java GC tunings (https://github.com/ixy-languages/ixy-languages/blob/master/Java-garbage-collectors.md https://github.com/ixy-languages/ixy-languages/blob/master/J...) Maybe tuning for latency is the appropriate trade-off given Go's high level of control over allocation and data layout?

pjmlp 7y ago
Because you are mixing up language with implementation. https://www.ptc.com/en/products/developer-tools/perc https://www.ptc.com/en/products/developer-tools/perc https://www.aicas.com/cms/en/JamaicaVM https://www.aicas.com/cms/en/JamaicaVM
- fwip 7y ago
  Go's a pretty simple language. It should be straightforward (not easy, because compilers aren't easy) for somebody to write another implementation, tuned for another use case - I can't imagine it wouldn't be easier than doing it for Java.
- zeeboo 7y ago
  You're jumping to conclusions. I understand the distinction and that there are many options for Java. Please be charitable. To rephrase the question more precisely: can you explain why OpenJDK, which has a multitude of garbage collector implementations tuned for both throughput and latency, performs worse on both throughput and latency than Go on this benchmark which has a garbage collector that, according to the original statement, is "tuned for latency above all else"? Perhaps OpenJDK is problematic in other ways (the author of the benchmarks suspects it's JIT induced, even though that's attempted to be controlled for), or perhaps this test depends less on the GC for some reason. Or maybe typical Go programs don't require as many allocations making allocation time much less important making tuning for latency the correct engineering decision. Would HotSpot do significantly better as is claimed? That's the sort of interesting technical discussion I'd like to have, instead of pedantry.
  - pjmlp 7y ago
    Sorry about that. I guess being a long time part of Java eco-system, kind of gets tiring of having outsiders (in general, not referring to you) always mixing up Java with what comes with their PC, as if C would be defined by GCC. As for the actual question, naturally having value types helps reduce GC pressure. Which on Java's case could be helped by trying out Graal or other JVMs that do better job at escape analysis than Hotspot. Alternatively, although it kind of is cheating, using the language extensions from either Azul or IBM for value types. In any case, when inline classes (aka value types) arrive, Java can easily do the same as Go here. JIT and de-optimizations play a role certainly, and are to blame for some performance impact, which can be further improved if a JVM like J9 gets used, given that it allows for PGO across runs. Finally, while Hotspot has good defaults, tuning all the knobs is a science, even with help of J/Rockit and VisualVM, which opens the door for performance consulting. The JVMs I mentioned, are targeted at soft real time deployment scenarios, as such they have APIs for low level control of memory management, while also supporting AOT with PGO compilation, thus allowing for low level fine tuning out of reach for the regular Java developers (pure Java SE implementations).
    - zeeboo 7y ago
      Thanks. Since most of the explanations that you gave involve things like controlling value types and data layout, it sounds to me like you might agree with the statement that in languages with better control over allocation and data layout, tuning a garbage collector for latency over throughput can be a good idea because your time spent allocating is less important. Is that fair?
      - pcwalton 7y ago
        In this case, "tuning for latency over throughput" means "not having a generational GC". Whether this choice makes sense does not depend on how much a program allocates. Rather, it depends on whether the generational hypothesis holds. The generational hypothesis is one of the most powerful and consistent observations in the entire CS field. It certainly holds for .NET, which has a very similar memory model to that of Go, and therefore .NET has a generational garbage collector.
pcwalton 7y ago
> How do you explain the throughput and latency numbers posted by https://github.com/ixy-languages/ixy-languages https://github.com/ixy-languages/ixy-languages? Because those graphs measure overall throughput and latency in an I/O setting, not GC throughput and latency specifically. There are many other confounding factors. In particular, the application in question has been tuned to perform as few allocations as possible, so GC throughput will naturally not show up as much as in other apps! Thanks to its generational GC and TLABs, allocation in HotSpot is like 5 instructions. > Maybe tuning for latency is the appropriate trade-off given Go's high level of control over allocation and data layout? Go doesn't give you control over allocation in a meaningful way.
- zeeboo 7y ago
  I believe we misunderstood each other. In particular "the application in question has been tuned to perform as few allocations as possible" seems contradictory to "Go doesn't give you control over allocation in a meaningful way." Allocation being fast isn't very important if you don't spend a significant portion of your time allocating. Isn't it possible that some languages (maybe even Go) by the nature of their semantics spend significantly less time allocating than other languages, and so tuning a garbage collector for latency is the appropriate decision? Why do you seem to believe that the Go application has been tuned more to avoid allocations than the Java application? If it hadn't been, then you would have to agree that tuning the garbage collector for latency is better because you can invest some effort into your application to have it perform better on both throughput and latency. Or is the argument that because "[throughput] is typically what applications want", this application is not a good example of most applications (according to what measure of "most")?
  - pcwalton 7y ago
    You can sometimes use unnatural programming patterns in both Go and Java to reduce allocations in practice. Nontrivial programs will always allocate, though, because various language constructs allocate in ways that are not obvious, and escape analysis is complex and hard to reason about. And the allocation semantics are not part of the language and are subject to change: this is what I mean by it not being meaningfully controllable. The most salient difference between Go's GC and HotSpot's GC is that the latter is generational. There is no convincing reason I've seen for Go not to have a generational GC, which would dramatically improve throughput by enabling bump allocation in the TLAB. The tiny amount of latency that this could add is by no means worth the cost of making allocations an order of magnitude slower. Allocation in HotSpot is five instructions.

by zeeboo 7y ago

pjmlp 7y ago

Because you are mixing up language with implementation. https://www.ptc.com/en/products/developer-tools/perc https://www.ptc.com/en/products/developer-tools/perc https://www.aicas.com/cms/en/JamaicaVM https://www.aicas.com/cms/en/JamaicaVM

fwip 7y ago
Go's a pretty simple language. It should be straightforward (not easy, because compilers aren't easy) for somebody to write another implementation, tuned for another use case - I can't imagine it wouldn't be easier than doing it for Java.
zeeboo 7y ago
You're jumping to conclusions. I understand the distinction and that there are many options for Java. Please be charitable. To rephrase the question more precisely: can you explain why OpenJDK, which has a multitude of garbage collector implementations tuned for both throughput and latency, performs worse on both throughput and latency than Go on this benchmark which has a garbage collector that, according to the original statement, is "tuned for latency above all else"? Perhaps OpenJDK is problematic in other ways (the author of the benchmarks suspects it's JIT induced, even though that's attempted to be controlled for), or perhaps this test depends less on the GC for some reason. Or maybe typical Go programs don't require as many allocations making allocation time much less important making tuning for latency the correct engineering decision. Would HotSpot do significantly better as is claimed? That's the sort of interesting technical discussion I'd like to have, instead of pedantry.
- pjmlp 7y ago
  Sorry about that. I guess being a long time part of Java eco-system, kind of gets tiring of having outsiders (in general, not referring to you) always mixing up Java with what comes with their PC, as if C would be defined by GCC. As for the actual question, naturally having value types helps reduce GC pressure. Which on Java's case could be helped by trying out Graal or other JVMs that do better job at escape analysis than Hotspot. Alternatively, although it kind of is cheating, using the language extensions from either Azul or IBM for value types. In any case, when inline classes (aka value types) arrive, Java can easily do the same as Go here. JIT and de-optimizations play a role certainly, and are to blame for some performance impact, which can be further improved if a JVM like J9 gets used, given that it allows for PGO across runs. Finally, while Hotspot has good defaults, tuning all the knobs is a science, even with help of J/Rockit and VisualVM, which opens the door for performance consulting. The JVMs I mentioned, are targeted at soft real time deployment scenarios, as such they have APIs for low level control of memory management, while also supporting AOT with PGO compilation, thus allowing for low level fine tuning out of reach for the regular Java developers (pure Java SE implementations).
  - zeeboo 7y ago
    Thanks. Since most of the explanations that you gave involve things like controlling value types and data layout, it sounds to me like you might agree with the statement that in languages with better control over allocation and data layout, tuning a garbage collector for latency over throughput can be a good idea because your time spent allocating is less important. Is that fair?
    - pcwalton 7y ago
      In this case, "tuning for latency over throughput" means "not having a generational GC". Whether this choice makes sense does not depend on how much a program allocates. Rather, it depends on whether the generational hypothesis holds. The generational hypothesis is one of the most powerful and consistent observations in the entire CS field. It certainly holds for .NET, which has a very similar memory model to that of Go, and therefore .NET has a generational garbage collector.

pcwalton 7y ago

> How do you explain the throughput and latency numbers posted by https://github.com/ixy-languages/ixy-languages https://github.com/ixy-languages/ixy-languages? Because those graphs measure overall throughput and latency in an I/O setting, not GC throughput and latency specifically. There are many other confounding factors. In particular, the application in question has been tuned to perform as few allocations as possible, so GC throughput will naturally not show up as much as in other apps! Thanks to its generational GC and TLABs, allocation in HotSpot is like 5 instructions. > Maybe tuning for latency is the appropriate trade-off given Go's high level of control over allocation and data layout? Go doesn't give you control over allocation in a meaningful way.

zeeboo 7y ago
I believe we misunderstood each other. In particular "the application in question has been tuned to perform as few allocations as possible" seems contradictory to "Go doesn't give you control over allocation in a meaningful way." Allocation being fast isn't very important if you don't spend a significant portion of your time allocating. Isn't it possible that some languages (maybe even Go) by the nature of their semantics spend significantly less time allocating than other languages, and so tuning a garbage collector for latency is the appropriate decision? Why do you seem to believe that the Go application has been tuned more to avoid allocations than the Java application? If it hadn't been, then you would have to agree that tuning the garbage collector for latency is better because you can invest some effort into your application to have it perform better on both throughput and latency. Or is the argument that because "[throughput] is typically what applications want", this application is not a good example of most applications (according to what measure of "most")?
- pcwalton 7y ago
  You can sometimes use unnatural programming patterns in both Go and Java to reduce allocations in practice. Nontrivial programs will always allocate, though, because various language constructs allocate in ways that are not obvious, and escape analysis is complex and hard to reason about. And the allocation semantics are not part of the language and are subject to change: this is what I mean by it not being meaningfully controllable. The most salient difference between Go's GC and HotSpot's GC is that the latter is generational. There is no convincing reason I've seen for Go not to have a generational GC, which would dramatically improve throughput by enabling bump allocation in the TLAB. The tiny amount of latency that this could add is by no means worth the cost of making allocations an order of magnitude slower. Allocation in HotSpot is five instructions.