4 ms·

I believe we misunderstood each other. In particular "the application in question has been tuned to perform as few allocations as possible" seems contradictory

by zeeboo 7y ago

I believe we misunderstood each other. In particular "the application in question has been tuned to perform as few allocations as possible" seems contradictory to "Go doesn't give you control over allocation in a meaningful way." Allocation being fast isn't very important if you don't spend a significant portion of your time allocating. Isn't it possible that some languages (maybe even Go) by the nature of their semantics spend significantly less time allocating than other languages, and so tuning a garbage collector for latency is the appropriate decision? Why do you seem to believe that the Go application has been tuned more to avoid allocations than the Java application? If it hadn't been, then you would have to agree that tuning the garbage collector for latency is better because you can invest some effort into your application to have it perform better on both throughput and latency. Or is the argument that because "[throughput] is typically what applications want", this application is not a good example of most applications (according to what measure of "most")?

pcwalton 7y ago
You can sometimes use unnatural programming patterns in both Go and Java to reduce allocations in practice. Nontrivial programs will always allocate, though, because various language constructs allocate in ways that are not obvious, and escape analysis is complex and hard to reason about. And the allocation semantics are not part of the language and are subject to change: this is what I mean by it not being meaningfully controllable. The most salient difference between Go's GC and HotSpot's GC is that the latter is generational. There is no convincing reason I've seen for Go not to have a generational GC, which would dramatically improve throughput by enabling bump allocation in the TLAB. The tiny amount of latency that this could add is by no means worth the cost of making allocations an order of magnitude slower. Allocation in HotSpot is five instructions.
- zeeboo 7y ago
  And yet here’s an empirical benchmark providing evidence against both of those claims. What empirical evidence do you have supporting yours? Can you provide them? Who defines when a construct is “unnatural”? Can you show examples of “unnatural” patterns in the provided benchmark programs supporting your hypothesis that you have to write unnaturally to control allocations? If not, why are you making those claims? Why is throughput of allocations important for “most” applications? How much time is typically spent on allocations in those programs? What is a typical application? What about typical applications in Go specifically since that’s the only language that Go’s garbage collector matters for? Also, why did you bring up the point about generational collectors? It seems like a red herring. This discussion has been about tuning for latency, and how in this benchmark, Go’s implementation did better on both throughput and latency than any OpenJDK collector under any tuning, and that tuning for latency is possibly a sound engineering decision. I’ve also noticed that you’ve made no explicit attempt to answer my more difficult questions, and if you continue to do so I will no longer assume you are acting in good faith.
  - pcwalton 7y ago
    Again, that benchmark isn't a benchmark of garbage collection; it's a whole-program benchmark. There's plenty of empirical evidence—a wealth of papers dating back to the 80s—that generational GC provides better throughput than non-generational GC on most applications. I'd be extremely surprised if a properly-implemented generational GC with bump allocation in the nursery wouldn't improve the performance of Go's GC by trading off a small amount of latency for increased throughput. The reason why you won't see a benchmark like that for Go is that nobody has implemented such a collector for Go.
    - zeeboo 7y ago
      No cited references, no answers about what “most” is, no answers about what “most” is in the specific context of Go, no answers about any “unnatural” patterns existing, no answers about the claim that it was “tuned” to reduce allocations, answering a question that wasn’t asked dodging the question that was (generational having better throughput vs when does throughput matter). There’s no way I can believe you are acting in good faith. It’s obvious to me now that you’ve just been trolling for years every time you discuss the topic of garbage collectors. I’m no longer going to engage.
      - seebs 7y ago
        I am not sure I'd go that far, but I want to point a thing out: Back in the 90s, it was a running joke that if you were going backpacking, you should take along a 3' length of fiber optic cable. "If you get lost, just bury the fiber optic cable and ask the backhoe operator for a ride back to town." In at least three programming-related communities I'm in, I have made a quip about "I think I'm gonna replace the 3' length of fiber optic cable in my backpacking gear with a short post about Go's garbage collector", and people have filled in "so if I get lost, I can just take the post out, and then ask pcwalton for a ride back to town". I can't say whether or not the argument is made in good faith, but man, there sure is a lot of it.
  - jnordwick 7y ago
    If you're trying to write high performance Java, that driver was a pretty poor attempt. While not idiomatic in the normal sense, there are a lot of low-latency, high throughput Java apps that never allocate after startup (and maybe some object pool growth before hitting a steady state). That benchmark was even running out of memory when using the non-collecting GC which shows they were allocating quite a bit, and their allocations probably escaped and needed to be moved off the TLAB and become expensive. I'm not sure they really knew how to write performance Java. I've been a part of systems that process million of events a second and never allocated. These will beat most C++ systems I've seen, and Go doesn't even start a chance. Java and HotSpot can do some amazing things (esp around inlining), but you do have to do them a little differently (and carefully), but at least you can. My experience with Go is that I never had the same level of control, and I don't think it is possible.
- camgunz 7y ago
  > You can sometimes use unnatural programming patterns in both Go and Java to reduce allocations in practice. This whole "you can't control your allocations in Go" thing is a strawman. You don't have absolute control because of escape analysis, that's true. But there's a pretty wide gap between "oh I'll just move this allocation point out of a loop" and "oh I wrote a bunch of unnatural code". It's pretty idiomatic to manage allocations in Go, just look at all the posts about using pprof. Besides, the spectrum of "clarity" (or whatever) to performance is present no matter what language you're using. Ex: ripgrep takes on more complexity so it can search a big buffer of text instead of just a line [1]. I wouldn't call that "unnatural" at all, just systems programming. [1]: https://blog.burntsushi.net/ripgrep/#mechanics https://blog.burntsushi.net/ripgrep/#mechanics
  - pcwalton 7y ago
    > But there's a pretty wide gap between "oh I'll just move this allocation point out of a loop" and "oh I wrote a bunch of unnatural code". Here's just one example: The Go compiler judges that parameters to indirect calls such as interface method calls as escaping. So in Go if you don't want to allocate you have no choice other than to avoid interfaces. But Go heavily encourages the use of interfaces, especially in the standard library. In C, C++, and Rust, on the other hand, you can use indirect calls without allocating, because the language guarantees the escaping behavior. This is a significant difference.
    - smarterclayton 7y ago
      My experience in the Golang side of this is limited to profiling and optimizing Kubernetes, which doesn't have extensive use of interfaces in general, but over the last 5 years I would say 90% of the wins we have have been: 1. optimizing allocations away in serialization 2. optimizing allocations away in api or biz logic in critical paths around that 3. algorithmic improvements on certain naive code paths 4. blocking ... distant gap 5. everything else Serialization is its own can of worms in Go, and many places we tried to optimize came down to trying to avoid a) stupid (don't use pointers to naive objects) and b) obvious (use value types) and then hit a wall where there weren't many cheap wins. I probably can count on one hand the number of hot paths which were interface related, so while I've occasionally been annoyed at the forced allocation moving into an interface, it's rarely actual something I've gotten a win from. That's just one particular experience, but the AMAZING integration with pprof from very early days has saved me far more time in improving perf than other Go annoyances has cost (relative to my experience in Java 2005-2011).
- deepsun 7y ago
  > There is no convincing reason I've seen for Go not to have a generational GC Well, they explored generational GC (after admitting that their uber-ultimate GC that they marketed as future-proof until 2025 could be better): https://blog.golang.org/ismmkeynote https://blog.golang.org/ismmkeynote > It isn't that the generational hypothesis isn't true for Go, it's just that the young objects live and die young on the stack. The result is that generational collection is much less effective than you might find in other managed runtime languages.

by zeeboo 7y ago

pcwalton 7y ago

You can sometimes use unnatural programming patterns in both Go and Java to reduce allocations in practice. Nontrivial programs will always allocate, though, because various language constructs allocate in ways that are not obvious, and escape analysis is complex and hard to reason about. And the allocation semantics are not part of the language and are subject to change: this is what I mean by it not being meaningfully controllable. The most salient difference between Go's GC and HotSpot's GC is that the latter is generational. There is no convincing reason I've seen for Go not to have a generational GC, which would dramatically improve throughput by enabling bump allocation in the TLAB. The tiny amount of latency that this could add is by no means worth the cost of making allocations an order of magnitude slower. Allocation in HotSpot is five instructions.

zeeboo 7y ago
And yet here’s an empirical benchmark providing evidence against both of those claims. What empirical evidence do you have supporting yours? Can you provide them? Who defines when a construct is “unnatural”? Can you show examples of “unnatural” patterns in the provided benchmark programs supporting your hypothesis that you have to write unnaturally to control allocations? If not, why are you making those claims? Why is throughput of allocations important for “most” applications? How much time is typically spent on allocations in those programs? What is a typical application? What about typical applications in Go specifically since that’s the only language that Go’s garbage collector matters for? Also, why did you bring up the point about generational collectors? It seems like a red herring. This discussion has been about tuning for latency, and how in this benchmark, Go’s implementation did better on both throughput and latency than any OpenJDK collector under any tuning, and that tuning for latency is possibly a sound engineering decision. I’ve also noticed that you’ve made no explicit attempt to answer my more difficult questions, and if you continue to do so I will no longer assume you are acting in good faith.
- pcwalton 7y ago
  Again, that benchmark isn't a benchmark of garbage collection; it's a whole-program benchmark. There's plenty of empirical evidence—a wealth of papers dating back to the 80s—that generational GC provides better throughput than non-generational GC on most applications. I'd be extremely surprised if a properly-implemented generational GC with bump allocation in the nursery wouldn't improve the performance of Go's GC by trading off a small amount of latency for increased throughput. The reason why you won't see a benchmark like that for Go is that nobody has implemented such a collector for Go.
  - zeeboo 7y ago
    No cited references, no answers about what “most” is, no answers about what “most” is in the specific context of Go, no answers about any “unnatural” patterns existing, no answers about the claim that it was “tuned” to reduce allocations, answering a question that wasn’t asked dodging the question that was (generational having better throughput vs when does throughput matter). There’s no way I can believe you are acting in good faith. It’s obvious to me now that you’ve just been trolling for years every time you discuss the topic of garbage collectors. I’m no longer going to engage.
    - seebs 7y ago
      I am not sure I'd go that far, but I want to point a thing out: Back in the 90s, it was a running joke that if you were going backpacking, you should take along a 3' length of fiber optic cable. "If you get lost, just bury the fiber optic cable and ask the backhoe operator for a ride back to town." In at least three programming-related communities I'm in, I have made a quip about "I think I'm gonna replace the 3' length of fiber optic cable in my backpacking gear with a short post about Go's garbage collector", and people have filled in "so if I get lost, I can just take the post out, and then ask pcwalton for a ride back to town". I can't say whether or not the argument is made in good faith, but man, there sure is a lot of it.
- jnordwick 7y ago
  If you're trying to write high performance Java, that driver was a pretty poor attempt. While not idiomatic in the normal sense, there are a lot of low-latency, high throughput Java apps that never allocate after startup (and maybe some object pool growth before hitting a steady state). That benchmark was even running out of memory when using the non-collecting GC which shows they were allocating quite a bit, and their allocations probably escaped and needed to be moved off the TLAB and become expensive. I'm not sure they really knew how to write performance Java. I've been a part of systems that process million of events a second and never allocated. These will beat most C++ systems I've seen, and Go doesn't even start a chance. Java and HotSpot can do some amazing things (esp around inlining), but you do have to do them a little differently (and carefully), but at least you can. My experience with Go is that I never had the same level of control, and I don't think it is possible.
camgunz 7y ago
> You can sometimes use unnatural programming patterns in both Go and Java to reduce allocations in practice. This whole "you can't control your allocations in Go" thing is a strawman. You don't have absolute control because of escape analysis, that's true. But there's a pretty wide gap between "oh I'll just move this allocation point out of a loop" and "oh I wrote a bunch of unnatural code". It's pretty idiomatic to manage allocations in Go, just look at all the posts about using pprof. Besides, the spectrum of "clarity" (or whatever) to performance is present no matter what language you're using. Ex: ripgrep takes on more complexity so it can search a big buffer of text instead of just a line [1]. I wouldn't call that "unnatural" at all, just systems programming. [1]: https://blog.burntsushi.net/ripgrep/#mechanics https://blog.burntsushi.net/ripgrep/#mechanics
- pcwalton 7y ago
  > But there's a pretty wide gap between "oh I'll just move this allocation point out of a loop" and "oh I wrote a bunch of unnatural code". Here's just one example: The Go compiler judges that parameters to indirect calls such as interface method calls as escaping. So in Go if you don't want to allocate you have no choice other than to avoid interfaces. But Go heavily encourages the use of interfaces, especially in the standard library. In C, C++, and Rust, on the other hand, you can use indirect calls without allocating, because the language guarantees the escaping behavior. This is a significant difference.
  - smarterclayton 7y ago
    My experience in the Golang side of this is limited to profiling and optimizing Kubernetes, which doesn't have extensive use of interfaces in general, but over the last 5 years I would say 90% of the wins we have have been: 1. optimizing allocations away in serialization 2. optimizing allocations away in api or biz logic in critical paths around that 3. algorithmic improvements on certain naive code paths 4. blocking ... distant gap 5. everything else Serialization is its own can of worms in Go, and many places we tried to optimize came down to trying to avoid a) stupid (don't use pointers to naive objects) and b) obvious (use value types) and then hit a wall where there weren't many cheap wins. I probably can count on one hand the number of hot paths which were interface related, so while I've occasionally been annoyed at the forced allocation moving into an interface, it's rarely actual something I've gotten a win from. That's just one particular experience, but the AMAZING integration with pprof from very early days has saved me far more time in improving perf than other Go annoyances has cost (relative to my experience in Java 2005-2011).
deepsun 7y ago
> There is no convincing reason I've seen for Go not to have a generational GC Well, they explored generational GC (after admitting that their uber-ultimate GC that they marketed as future-proof until 2025 could be better): https://blog.golang.org/ismmkeynote https://blog.golang.org/ismmkeynote > It isn't that the generational hypothesis isn't true for Go, it's just that the young objects live and die young on the stack. The result is that generational collection is much less effective than you might find in other managed runtime languages.