3 ms·
I really hope that they will release this as a library. We're having the exact same challenges running Go in production. The biggest challenge with Go in produ
by atombender 5y ago
I really hope that they will release this as a library. We're having the exact same challenges running Go in production.
The biggest challenge with Go in production is that, as the article points out, Go doesn't have a maximum memory setting that can be used to tune the GC. If you run Go with a memory limit on Kubernetes, it's common for it to simply run out of memory rather than using the limit for backpressure.
This is kind of surprising, since Go is so entrenched in the Kubernetes world (and both Docker and Kubernetes are written in Go). It's possible that Google itself doesn't develop that much stuff in Go, and that the Go team doesn't have a huge incentive to innovate in this area.
There have been a few attempts at improving heap management, including an aborted attempt at respecting ulimit [1], a promising implementation of a SetMaxHeap() function [2], and a propsoal for dealing with backpressure [3], but these projects have mostly failed to get proper traction. It's a complex problem that needs a cohesive solution.
Fortunately, there is now a proposal [4], which has been accepted, to add a soft limit to Go, which has a more thought-through design [5], though I'm not sure if it's being actively worked on yet.
I'm also not sure if that proposal, when implemented, will make the Uber approach redundant, or if these are in fact complementary. If Uber could open-source their library, it might be a good solution until Go itself has better GC management.
[1] https://github.com/golang/go/issues/5049 https://github.com/golang/go/issues/5049
[2] https://github.com/golang/go/issues/16843 https://github.com/golang/go/issues/16843
[3] https://github.com/golang/go/issues/29696 https://github.com/golang/go/issues/29696
[4] https://github.com/golang/go/issues/48409 https://github.com/golang/go/issues/48409
[5] https://github.com/golang/proposal/blob/master/design/48409-soft-memory-limit.md https://github.com/golang/proposal/blob/master/design/48409-...
- EugeneOZ 5y agoI have 0 experience in Go, and for me it looks like "Figure 11" in their article contains a full example of the needed code. Maybe I'm wrong.
- atombender 5y agoNo, their code adjusts the GOGC value. The code fragment, as I understand, just shows how to essentially get a notification every time the GC runs.