11 ms·
Getting to Go: Garbage collection and runtime issues
- collinf 8y agoWhat an amazing read. These folks really pulled off some fantastic engineering without a JIT. It will be interesting to see if the team decides to add additional knobs to configure the Go GC for different workloads like Java has now.
- squiguy7 8y ago> We also do not intend to increase the GC API surface. We've had almost a decade now and we have two knobs and that feels about right. There is not an application that is important enough for us to add a new flag. It sounds like they don't intend to add any more tuning parameters for the time being. Considering they have made it this far without many ways to adjust the garbage collector, I'll bet this remains true.
- kjeetgill 8y agoGo has one BIG advantage compared to Java with respect to both JIT and GC. Go has structs. This introduces a little more complexity that Java chose (reasonably) to avoid by making everything a primitive or a reference. There isn't any "direct" objects or explicit pointers like Go has. This may muddy up the language vs Java a little bit, but it also gives a lot more tools to work your code around the runtime's limitations. And thats a perfectly fine design point by me. If you're going to have platform limitations (no JIT, simple GC) then I need reasonable tools when I hit them. It's much easier to avoid frequent allocations or implement arenas in Go. This takes the pressure of the runtime to be one size fits all. I know, I know, Java is getting Value Types some day! We'll see where that lands.
- staticassertion 8y agoIt's then interesting to compare to C#, which also can do stack allocation, but still chooses a generational GC.
- kjeetgill 8y agoI knew it had it but I have no C# experience. Does idiomatic code use this or only more niche optimizations?
- pjmlp 8y agoAll performance critical data structures are value types. Meaning int, long, char, bool, ..., points, rectangles, pairs, colours, enumerations. Then generics are reiffed. So each instance gets a copy if value types are used. Additionally, you can explicitly allocate value types and arrays on the stack and there are APIs for manual memory management.
- masklinn 8y agoC# defaults to reference types, structs are not usually recommended as the default way to build your own types, only as an optimisation.
- nemo1618 8y agoI love Go's value types. If you're careful/obsessive, you can write programs that don't allocate anything on the heap. That sort of power is really nice to have in a pinch.
- mike_hearn 8y agoI agree it's a good and well written talk. I enjoyed reading it. WRT knobs. Apparently they aren't going to do so, but I suspect that's a mistake. It's apparent from reading this talk that GC design for Go has often been constrained by this "no knobs" design philosophy in ways that seem problematic. For instance they designed a GC to have the shortest pauses possible, but then kept hitting throughput problems on a classic batch job where latency doesn't matter at all (their compiler). I wrote about this problem some time ago: https://blog.plan99.net/modern-garbage-collection-911ef4f8bd8e https://blog.plan99.net/modern-garbage-collection-911ef4f8bd... The essay points out that GC tuning exists because of fundamental tradeoffs between low pause times and throughput. Java lets you tune the GC precisely to avoid the nasty situation Go got itself into, where the demands of very very different kinds of workload (super latency sensitive Google servers vs compilers) are difficult to reconcile with each other. You can see this tension come through several times, like when they admit that they would have done a normal "enterprise" style GC as they rather condescendingly called it, but they felt Go's market position was under threat, they only had a year to make improvements but were constrained by the requirement to not slow down the Go compiler itself: So we were limited. We had basically a year of compiler performance improvements that we could eat up by making the GC concurrent. But that was it. We couldn't slow down Go programs. That would have been untenable in 2014 ... Go also desperately needed short term success in 2015. So they ended up with this design that they knew was kind of weird and not that great, but it let them promote low latency above all else to their users, and they sort of buried the tradeoffs they were making along the way. Which, reading between the lines, might have been related to internal staffing issues at Google - why else would a well funded project like Go have "desperately needed short term success in 2015"? It's not like they sell the Go compiler. More quotes that support this: As you can see if you have [request oriented collector] on and not a lot of sharing, things actually scale quite nicely. If you don't have ROC on it wasn't nearly as good. So they had a GC that scaled better. But it got killed by, you guessed it, the needs of batch jobs: But that wasn't good enough, we also had to make sure that ROC didn't slow down other pieces of the system. At that point there was a lot of concern about our compiler and we could not slow down our compilers. Unfortunately the compilers were exactly the programs that ROC did not do well at. We were seeing 30, 40, 50% and more slowdowns and that was unacceptable. Go is proud of how fast its compiler is so we couldn't slow the compiler down, certainly not this much. They could have solved this problem by having two GCs for Go programs tuned for throughput and latency, or doing what the Java guys did and trying to make an adaptive GC like G1 where you can give it pause time targets: if the target is very high then it tunes for throughput otherwise it tunes for latency. But instead they seem to have sacrificed it all on the altar of "no knobs", which is a pity because they ended up introducing two knobs anyway: SetGCPercent and SetMaxHeap. Well hey, guess what, if you use the G1 GC in Java you are only supposed to tweak two knobs too - max heap size (same as Go) and pause time target. If anything G1's knobs seem better because pause time target is a lot easier for developers to reason about than "heap overhead multiplier". In the end it's unclear their sacrifices were really worth it, although I don't know enough about Go to know what dire straits in 2015 were causing them to need such short term success.
- kjeetgill 8y agoNow this is the kind of post I come to HN for. Deeply technical explanations of real expert work! I'm particularly keen on some of the details around goroutines since Project Loom hopes to bring something similar to the JVM and it's nice to see what the design space looks like. Another is that go sometimes gets dinged for not being modern enough in a bunch of different directions, e.g. no rust-like immutability/borrowing, generics, or generational GC, but there is a lot of thought put into what they are working on and where they want to focus the language.
- pjmlp 8y agoVery interesting.
- steveklabnik 8y agoThere's so many good quotes in this article. Two of my favorites: "It isn't that the generational hypothesis isn't true for Go, it's just that the young objects live and die young on the stack." "... we are led to the conclusion that memory, or at least chip capacity, will follow Moore's law longer than CPUs" Kudos to the Go team!
- pjmlp 8y ago"No more C, no long tail of bugs due to C coders not understanding GC but having a cool idea about how to copy strings."
- w8rbt 8y agoI've seen more home made string implementations in C than I can remember. Most all of them were horrible. Say what you like about C++, but it certainly fixed that mess. And I love Go... I think it's the future.
- deleted 8y ago[deleted]
- pjmlp 8y ago> Say what you like about C++, but it certainly fixed that mess. Already in the MS-DOS days, even if string class back then was compiler framework specific, and part of ritual of passage as C++ developer.
- munificent 8y agoI've seen more home-made string implementations in C++ than C. :-/
- pjmlp 8y agoI bet all of them derive either from the days before std::string, which only started to be commonly supported after 2000, as compilers were catching up with C++98. The typical rite of passage that every C++ developer needs to write their own string and vector classes as part of learning C before C++, instead of properly C++ as we can do nowadays. Finally those shops that don't allow to use proper C++ on their code, rather C with Classes.
- geodel 8y agoSo Go set Heap to 2X and CPU 25% as GC overhead. In Java side may be due to generational/copying GC Heap is recommended 3-4X and CPU ~20%. This is good read about Java GC tuning: https://engineering.linkedin.com/garbage-collection/garbage-collection-optimization-high-throughput-and-low-latency-java-applications https://engineering.linkedin.com/garbage-collection/garbage-...
- the8472 8y agoThe article you're linking to is outdated. It talks about java 7 and CMS, which is deprecated as of OpenJDK9. There are other collectors in OpenJDK and then there are other JVMs too. They all come with a lot of different overhead, complexity, latency and throughput tradeoffs.
- kodablah 8y agoSo they had to work with a few algorithms and tweaks. I wonder if there would be value in exposing the GC (and alloc) iface so others can implement without recompiling the language. I would love to toy with a pluggable GC. Last I looked at .Net's when it was open sourced, it was a gargantuan cpp file. I know LLVM has it somewhat pluggable. We can see a lot of Go GC's work at https://github.com/golang/go/blob/master/src/runtime/mgc.go https://github.com/golang/go/blob/master/src/runtime/mgc.go but at quick glance, I couldn't see how I could plug in an alternative impl. Surely they had to build an experimentation abstraction to test their ideas, right? Or are GC's just so specialized and use so much internal knowledge that extracting a common iface is unreasonable? Also, > The math is absolutely fascinating, ping me for the design docs Ping! Publish them please.
- Scaevolus 8y agoThe design docs are linked right below :) Go 1.5 concurrent garbage collector pacing: https://golang.org/s/go15gcpacing https://golang.org/s/go15gcpacing Proposal: Separate soft and hard heap size goal: https://github.com/golang/proposal/blob/master/design/14951-soft-heap-limit.md https://github.com/golang/proposal/blob/master/design/14951-...
- mike_hearn 8y agoThe latest versions of OpenJDK have a pluggable GC interface, at the source level at least - you can't literally drop a DLL into a directory and load it dynamically, but the source code is modularised quite well and there are several different GC engines available to be chosen at startup.
- kodablah 8y agoGood to know, I knew that I could choose different ones at runtime via java args, but unaware there was a runtime interface. I see cpp code at [0] and some adjacent impls. is there a single C-level h file or is it cpp only? Any hello-world trivial impl? 0 - http://hg.openjdk.java.net/jdk/jdk11/file/a0de9a3a6766/src/hotspot/share/gc/shared http://hg.openjdk.java.net/jdk/jdk11/file/a0de9a3a6766/src/h...
- deleted 8y ago[deleted]
- riobard 8y agoDoes anyone have comparable GC latency numbers comparing Go vs Java as of 2018?
- nvarsj 8y agoIt’s not even a contest on the latency front. Golang gc targets sub ms, hotspot easily hits 100s of ms. A well tuned hotspot can reach 10s of ms pauses on average, with a long tail of 100 ms+. This is on typical apps.
- pjmlp 8y agoUnless one gets to use Azul or real time JDK. It is easy to forget there isn't a single answer with Java.
- deleted 8y ago[deleted]
- nvarsj 8y agoYeah true, but C4 is a commercial product, and I've yet to meet someone that used it. But on paper, Azul's C4 GC sounds really great. I think it might require a custom kernel as well, and some other OS restrictions - which would also somewhat limit its usage as a general purpose collector.
- pron 8y ago1. This is not due to GC design but to language semantics (value types, which aren't yet available in Java). 2. In September '18, ZGC will be available in OpenJDK on Linux/x86 (available today in early access), which also targets sub ms pauses (and guarantees under 10ms): https://wiki.openjdk.java.net/display/zgc/Main https://wiki.openjdk.java.net/display/zgc/Main Note that at some point, worst-case latency becomes far less meaningful than throughput, because unless running on a realtime OS, the OS introduces bigger pauses, and it makes no sense for the GC to try and beat them.
- 8y ago
- deleted 8y ago[deleted]
- igtztorrero 8y agoBreak
- js2 8y agoNot currently loadable for me due to the Google Cloud Loadbalancer outage[0]: Google Cache (irony...): http://webcache.googleusercontent.com/search?q=cache:BtSPqHcAa0kJ:https://blog.golang.org/ismmkeynote&num=1&hl=en&gl=us&prmd=ivn&strip=1&vwsrc=0 http://webcache.googleusercontent.com/search?q=cache:BtSPqHc... [0] https://news.ycombinator.com/item?id=17552532 https://news.ycombinator.com/item?id=17552532
- pcwalton 8y agoI still don't see any comparison of Go's current collector with a generational GC with bump allocation in the nursery. The article states that copying would have been too much work to implement in 2014. Fair enough. But that's an engineering decision specific to Go's circumstances. It does not necessarily support forgoing generational garbage collection in general. The primary advantage of generational GC is the huge throughput advantage that bump allocation in the nursery gives you. If you're not comparing against such a GC, then you're not properly comparing non-generational GC to generational GC. The write barriers you need reduce throughput, but they're ordinarily counterbalanced by the throughput gains you get from bump allocation. Of course you will take a throughput loss if your generational GC is non-copying; that's why people don't usually deploy non-copying generational GCs.
- the8472 8y agoWouldn't a non-copying generational collector with bump allocation be a recipe for fragmentation?
- pcwalton 8y agoYes. That is why they have to be copying collectors.
- readittwice 8y agoIsn't JSC kind of an exception to this? JSC has a generational non-copying GC. It uses bump-pointer allocation for empty blocks (block=memory with objects of same size) and then switches to allocation via free-list if the block is full. For implementing the generational GC they simply use sticky mark-bits.
- pcwalton 8y agoI don't consider what JSC does bump allocation—it's just a highly optimized traditional allocator. Most malloc implementations have bump allocation somewhere. SpiderMonkey used to use an allocator much like that of JSC before it switched to copying generational GC, to significant performance improvements.
- skybrian 8y agoI wonder if they'd consider changing the bit that indicates whether a word is a pointer or not based on the value rather than the type? Interfaces in Go seem pretty inefficient for unions of small types. If you have a slice of interfaces, it's all pointers, even if many of the values fit within a word and could be stored inside the interface.
- weberc2 8y agoIIRC they originally were stored inside of the interface, but then the GC would have to check whether they were a value or a pointer and this switching was too costly.
- qazwe 8y agoWith all improvements to the GC, in 2018 Go is ready to develop real-time AAA games? In the past there is discussions that Go is not very well for game development. (GC latency)
- pcwalton 8y agoI would think the biggest barriers (no pun intended) to using Go in CPU-bound apps like games would be the lack of mature compiler optimizations, compared to GCC and LLVM. Additionally, the performance of cgo can cause problems when linking to libraries like Vulkan that can't effectively be lowered into syscalls. I also think that throughput of memory allocation, not just latency, matters a lot more than people give it credit for.
- ngrilly 8y ago> Additionally, the performance of cgo can cause problems when linking to libraries like Vulkan that can't effectively be lowered into syscalls. Agreed. But you also ask for very fast memory allocation, which implies a bump allocator, which implies a copying GC, which implies pinning objects shared with C through cgo (Go's foreign function interface), which would probably make cgo even slower. > I also think that throughput of memory allocation, not just latency, matters a lot more than people give it credit for. At 60 fps, minimizing latency is critical. You have to optimize this first. When your longest GC pause is well below 1/60 s, then you can optimize throughput of memory allocation. Anyway, most AAA video game engines being written in C/C++, my guess is they don't use a bump allocator, which means they do quite well with allocators like tcmalloc or jemalloc, probably by allocating as much as possible early, and using object pools.
- pcwalton 8y agoI agree that these are all tradeoffs. My view is that languages like Rust and C++ exist for a reason, and demanding games are among the domains that they're most appropriate in. :) Note that Unity is an existence proof that most games can get by just fine with high-level logic in GC.
- corydominguez 8y agoHas anyone found a video of this keynote?
- incadenza 8y agoI find this stuff fascinating but so much of it went over my head. Can anybody point me in the right direction to read more about the relevant topics? Is this standard material in compiler design, or another related field?
- pcwalton 8y agoMy favorite resource is: http://memorymanagement.org/ http://memorymanagement.org/ Its glossary is invaluable. Even though the site is old, GC has been around for decades and the state of the art hasn't changed much since it was written. It's opinionated in a good way, explaining why production GCs work as they do.
- incadenza 8y agoAwesome! Thanks.
- amelius 8y agoSeems like Go may become an interesting target-language for compilers.
- rurban 8y agoWhen they'll finally get a good generational copying collector, yes. Now it's just too slow for that.
- abacate 8y agoThe article is indeed interesting when it goes to the GC/allocation part and it's a nice read, but it really gets C++ wrong when talking about values/references. C++ is not "reference oriented language" the same way it is not an object-oriented language. For instance, what restricts one from doing: int blah(myobj a); Instead of: int blah(myobj &a); For passing your a myobj value? The usual argument is efficiency - since "myobj" may be a big and have an expensive copy constructor - and you may happen to see this pattern being commonly used in C++ programs, but restrictions? There are none. C++ even makes it possible to assert coherency with copy/assign constructors, and modern C++ supports the more efficient moving as well. So, the point is: nothing in the language restricts you from passing things by value. I'd go further an say that blindly passing things by reference is bad practice and detrimental for simple objects - since you now have to consider lifetime/ownership of the reference you passed along, and this will probably complicate an otherwise simple piece of code. The argument about fields in memory also makes not much sense. How is that different from C++? Keeping references/pointers in structures is possible but not exactly what one needs to do. Otherwise the article is great I just think it misses the point when comparing to "C and C++", since they are not the same language - even though syntax is similar and there is some level of compatibility.