8 ms·
An interesting shift in the wind for modern garbage collection comes from microservices. The work on JVM garbage collectors over the last ten years has increas
by twic 10y ago
An interesting shift in the wind for modern garbage collection comes from microservices.
The work on JVM garbage collectors over the last ten years has increasingly been about supporting bigger and bigger heaps efficiently, as physical machines have got bigger, and programs' (or at least application servers') heaps have grown to fill them. Even Shenandoah, a current prototype collector, is designed around large heaps.
But with microservices, we use our cavernous machines to house many small heaps instead. That leaves us with rather different garbage collection challenges.
So, it's possible that the Go developers will get to ignore decades of progress, as they love to do, and still be competitive, because that progress was towards a different future to the one we arrived at.
- agumonkey 10y agoI'd love to know about other GC schemes. Between no GC and whole VM GC. By type GC, localized sub tree GC. edit: thank you all for your answers, much appreciatedly
- osi 10y agoLook at what Erlang's BEAM VM does. It has per-"process" heaps iirc.
- micmus 10y agoErlang has a per-process GC - each Erlang process has it's own heap and it's own garbage collector. There's no VM-wide GC. This usually means the GC pauses apply only to a small part of your system, while the other parts are running completely normally. There's a good article that describes it in depth: https://www.erlang-solutions.com/blog/erlang-19-0-garbage-collector.html https://www.erlang-solutions.com/blog/erlang-19-0-garbage-co...
- noss 10y agoAlso if you have a lot of data you use an ets table which is more or less an in-process but off-gc-heap key-value database.
- staticassertion 10y agoThanks, I've been wondering how Erlang does GC.
- AstralStorm 10y agoOf course, the cost is extra heap fragmentation.
- dfox 10y agoIn Erlang case, that is not so much of a problem as most allocated objects have essentially same size. On the other hand this causes another problem that GC's and allocators in Erlang VMs tend to have enormous overhead because most most GC'd objects are very small with sizes comparable to per-object GC overhead.
- agumonkey 10y agoI joked once that in lisp, a vast majority of memory allocation are basically malloc(2). I wondered if it made fragmentation irrelevant.
- twic 10y agoI used to idly browse papers from programming language conferences, and there was often interesting GC stuff. PLDI is probably your bet bet - here's 2016: http://dl.acm.org/citation.cfm?id=2908080&picked=prox http://dl.acm.org/citation.cfm?id=2908080&picked=prox When there were two GC papers: Idle time garbage collection scheduling - "We implemented idle time garbage collection scheduling in V8, an open-source, production JavaScript virtual machine running within Chrome. We present performance results on various benchmarks running popular webpages and show that idle time garbage collection scheduling can significantly improve latency and memory consumption." Assessing the limits of program-specific garbage collection performance - "We demonstrate that there is considerable promise to reduce garbage collection costs by developing program-specific collection policies." Or going back to 2012: And then there were none: a stall-free real-time garbage collector for reconfigurable hardware - "We present the first implementation of a complete garbage collector in hardware (as opposed to previous "hardware-assist" techniques), using an FPGA and its on-chip memory. Using a completely concurrent snapshot algorithm, it provides single-cycle access to the heap, and never stalls the mutator for even a single cycle, achieving a deterministic mutator utilization (MMU) of 100%." That's by David Bacon, one of the runtime implementation greats, who has been after low-pause GC for years. A golden oldie from him is: A Pure Reference Counting Garbage Collector - "When processor or memory resources are limited, the Recycler usually runs at about 90% of the speed of the mark-and-sweep collector. However, with an extra processor to run collection and with a moderate amount of memory headroom, the Recycler is able to operate without ever blocking the mutators and achieves a maximum measured mutator delay of only 2.6 milliseconds for our benchmarks. End-to-end execution time is usually within 5%." As for by-type GC, i remember reading a paper years (decades?) ago, perhaps from Microsoft, where they'd tried putting every class in its own region, the idea being that when you wanted to collect some particular region, you could exploit your knowledge of types and their fields to only scan regions which could possibly have pointers to it (eg to collect BlogPost, you have to scan User). Clever, but didn't work well in practice, because so much garbage is lists or strings that can be pointed to from a lot of places.
- scott_s 10y agoISMM, http://dl.acm.org/event.cfm?id=RE149 http://dl.acm.org/event.cfm?id=RE149, is also likely to have papers on garbage collection. The table of contents from 2016: http://dl.acm.org/citation.cfm?id=2926697 http://dl.acm.org/citation.cfm?id=2926697
- mike_hearn 10y agoCheck out Singularity. It was a research effort from Microsoft Research to build a whole OS in a modified .NET Singularity was a single address space OS, thus there were no "processes" in a traditional sense. They still had a notion of isolated spaces though partly to allow different heaps to be GCd independently, and to use different GC algorithms (with different barriers) exactly due to the lack of a one-size-fits-all approach. Messaging between these process-like-things was done by moving objects onto a "transfer heap" and back again, which boiled down to just a simple pointer write. There were no syscalls. A very nice architecture, from a theoretical perspective. However it's not clear to me that for standard desktop-sized machines you really need segmented GC spaces these days. With an incremental collector you can probably get pause times in the milliseconds range for a 32GB heap and I guess 32GB is as much RAM as you need on a Pro-level machine. For servers of course you go bigger. One interesting GC algorithm I came across did thread-local collections. Objects were tracked with write barriers to discover when they became 'globally reachable' and moved to a global heap. Threads could stop independently whilst others continued running to collect their thread local heaps. That opens up the possibility in desktop/mobile apps of having a main thread that's doing an animation of some kind in a low-garbage way, whilst the background thread does work as quickly as possible, with the background thread collecting independently whilst the animation thread still runs.
- mpweiher 10y agoYou might be interested in: "A Unified Theory of Garbage Collection" https://pdfs.semanticscholar.org/91dc/a25f8cb407fc68218f7d5adb912e7db35e81.pdf https://pdfs.semanticscholar.org/91dc/a25f8cb407fc68218f7d5a... Basic idea is that tracing and reference counting are duals of each other. Reference counting finds dead objects, tracing finds live objects. Optimized variants of each of these tend to approach the other side.
- deleted 10y ago[deleted]
- me2i81 10y agoTake a look at The Garbage Collection Handbook by Jones et. al. ISBN-13 is 978-1420082791.
- osi 10y agoWhere is the data kept? The very large heaps are partially driven by distributed in-memory storage systems.
- twic 10y agoHmm, well if the processing is microservices, presumably the data is in a NoSQL database. More seriously, though, isn't most large in-memory data storage done off the garbage collected heap? I have to confess i'm actually not sure what anyone is doing with >20 GB of normal objects.
- dkersten 10y agoI'm sure people do use huge heaps, but most web applications I know of (microservice or not) store the bulk of their data in a (separate) database. So, unless your application is a database, which might want a huge heap so it can cache a lot in-memory, I don't think huge heaps are really necessary. IMHO, at least.
- dom0 10y agoSince it's not really possible to do reliable giant user space caches in the POSIX model of things you'd usually delegate to the page cache.
- sheeshkebab 10y agoSearch (elastic/solr), Cassandra, presto, and most other java backend storage and data processing engines need 20+Gb (at least) to be mildly useful. It's not uncommong to go with 64gb+ heaps there too. Microservices/API won't help with lowering these.
- osi 10y agoyes, exactly this. and by having scalable GC algos so you don't need to do "off heap" tricks in java, things are much easier.
- geodel 10y agoA lot of 'decades of progress' in software is similar to progress in College tuitions, Health Insurance premiums and Cable TV bills. Yes advanced solutions have been developed and now price has to extracted by any means irrespective of user needs or agreement.
- baus 10y agoI don't agree with the snarky tone of this comment. I'm not a Go programmer, but I've seen many concepts touted as "progress" over the past 20 years which have been dead ends. I think it is reasonable for designers to curate advances to address the problems they are trying to solve.
- refulgentis 10y agoThe comment doesn't come off as snarky, but instead a milder version of the content of the article. Namely: I don't agree with Go marketing their GC as if it magically provides no trade offs. Since reading the original design doc for it, I've assumed that Google was simply lazy/didn't care about Android, because the Android GC didn't provide this magic.
- Merovius 10y ago> I don't agree with Go marketing their GC as if it magically provides no trade offs. I don't believe this to be true. They always point out painstakingly, that the advances made on pause times come with less throughput, for example. What the go people are doing (and with which I tend to agree and I believe it's hard to argue against the success they're having), is to say that they believe it's possible to build a GC which is good enough for most modern use cases and doesn't come with a lot of tuneables. (However, even that comes with tradeoffs; it limits the use cases that go can be used for. For large enterprises, being able to minutely tune a GC might be an important thing)
- refulgentis 10y agoApologies for the brusque contrarianism, but this simply isn't true. I spent hours pouring over all the material they published when starting on this journey, because I was astounded that a team with a reputation for purposely avoiding groundbreaking work in favor of productivity today, had done something truly groundbreaking. It wasn't remotely clear that the advancement here was "We don't provide settings, because we're guesstimating your workload matches the standard Go use case!", and all the histrionics further masked this truth. At this point I'm just repeating the article, though. :)
- jblow 10y agoAll the Hacker News kids are crazy about microservices right now. In three years, people will have realized it wasn't such a good idea, and moved on to the next "this will solve everything" fad. Once you have seen enough of these fads go by, it's pretty obvious. I'd be careful about extrapolating future directions of computer science from something that seems cool this year.
- bsder 10y ago> All the Hacker News kids are crazy about microservices right now. > In three years, people will have realized it wasn't such a good idea, and moved on to the next "this will solve everything" fad. Microservices are simply the youngsters rediscovering that encapsulation is a damn good idea. As I explain to people, microservices do NOT make your life easier. If you can exist in a monolith, that's way easier to implement. However, once things start getting complicated, microservices make certain things possible.
- neeleshs 10y agoAnd if its a business application, and the company survives long enough, it will invariably get complicated.
- jeffdavis 10y agoMicroservices go beyond encapsulation in important ways, and regress in other ways. Build/deploy/release cycles become independent with microservices, which can be great. Also you can write microservices in different languages. But the APIs can become less rich. Try passing callbacks between microservices written in different languages. And it can be more of a pain to change APIs. And if there's any state to worry about, you need to manage that and maybe use a stateful protocol (which are harder to get right).
- acveilleux 10y agoThe independence of microservice is not always achieved. You often end up with a mess of interrelated services depending on fairly narrow version ranges to retain compatibility. The more homogeneous your production environment is, the less that's a problem. The worst case is when you have a dozen (or hundreds...) of production clusters that are all independent and not necessarily in-synch as far as deployed version goes.
- jeffdavis 10y agoPartially true, and Joe Armstrong (creator of erlang) makes similar arguments. But that doesn't work well for everything. Erlang is more clear about their assumptions and goals. They don't claim to have an amazing new GC, they just say they have a simple generational GC, and that's all they need because they assuming many small heaps.
- qaq 10y agoit's also not an STW GC so the pauses affect only small number of processes.
- stcredzero 10y agoSo, it's possible that the Go developers will get to ignore decades of progress, as they love to do, and still be competitive, because that progress was towards a different future to the one we arrived at. Are other fields as bad as programming? Do engineering people get to do the equivalent of having a paper published or getting to present at a conference because they implemented something in language A which was already done three years ago in language B? Do other fields have people grousing about how no one uses X, even though it is clearly "better," where "better" is basically an unsubstantiated opinion? Some embarrassingly large number of "best practices" in our field are pretty much based on a feeling it "seems right" combined with a "well, it worked a couple times before" level of empiricism. Vital decisions about tooling, such as choice of programming language, are made by some combination of: following the herd, following one's own prejudices, or being enamored of a nifty idea. One thing a company at the scale of Google could probably afford to do, would be to collect real data about the effects of programming practices.
- flogic 10y agoGo isn't a research project. It's more a manifesto on programming philosophy born of "old man rage" in language form. This works out mostly because the old men in question happen to include Rob Pike and Ken Thompson. However, that results in a very specific feel and set of tradeoffs in Go.
- baus 10y agoYou might not agree with the design decisions, but to say one of the most successful contemporary systems programming languages was born out "old man rage" is insulting
- stcredzero 10y agoto say one of the most successful contemporary systems programming languages was born out "old man rage" is insulting Should I be insulted by your finding it insulting? My thought on reading that phrase was, "Right on!"
- eva1984 10y agoWho said microservices don't need large heaps? The prefix 'micro' means the share of responsibility and boundary of functionality, not any hints at memory size. Of cause JVM needs to focus on large heaps. Now it still stucks somewhere at 32GB for performance reason. And for heavy-lifting application like HBase/ElasticSearch, that is currently OK, but still makes it quite inconvenient to utilize larger memory(you might need deploy multiple instances on the same box, etc). You can't invent a new buzzword and tell other people they are solving the wrong problem.
- user5994461 10y ago> Now it still stucks somewhere at 32GB for performance reason Short version: No it's not. Long version: Heap is under 30 GB or over 40 GB. Never between. There is a JVM optimization that allows it to use 4 byte pointers up to 30GB of heap.
- eva1984 10y agoWhat is the reasoning for this? I think JVM still has problem handling real large heaps, the GC stop will be unbearable..Does thing change recently?
- user5994461 10y agoThere is no problem to handle large heaps. There is simply a gap: A 30 GB heap (with 4 bytes pointers) may be similar in capacity to a 40 GB heap (with 8 bytes pointers). Guides and documentation (elastisearch) are simplifying this as "java can't do more than 30 GB heap", which is factually incorrect. You can do a 60GB if you want.
- bad_user 10y ago> There is no problem to handle large heaps. That's not true, dealing with large heaps is problematic in any garbage collected language because due to fragmentation you end up in stop-the-world full GC phases. Which for a big heap (i.e. bigger than 4 GB!), you can end up with latencies measured in seconds or even minutes. The article mentions the CMS GC for Java, which has this problem. The G1 GC, introduced in Java 7 I think, is much better, however from what I've read it too isn't immune to full GCs. Of course for Java you also have the Azul "pauseless GC", which is supposed to never do this, but it is a commercial solution. And the article also mentioned another GC currently being contributed by Red Hat, which is good to hear.
- AnimalMuppet 10y ago> So, it's possible that the Go developers will get to ignore decades of progress, as they love to do, and still be competitive, because that progress was towards a different future to the one we arrived at. Progress in the wrong direction is not progress. Ignoring "progress" that was in the wrong direction is progress.