6 ms·
A Well Known but Forgotten Trick: Object Pooling
- tbirdz 11y agoObject pools are certainly a very useful technique, but can they really be considered a "forgotten trick"? This kind of pooling is pretty basic CS knowledge, and thus is a very popular technique in many applications, libraries, etc. I would have assumed most people working with a need for scale in a GC language like Java would be familiar with this technique (dare I call it a "design pattern") Perhaps I am wrong and the technique is less familiar than I had thought?.
- ajuc 11y agoIt's everywhere in java ee. Still I guess people writing fronted may have never heard of it (except thread pools?).
- ericflo 11y agoPeople writing frontend learn how to pool objects very quickly in both iOS and Android for list/collection view cells.
- rturben 11y agoI know that in things like games and other highly concurrent applications that object pooling is a must, otherwise you inevitably run in to the problem of attempting to update or render some game object that's died and been deleted. Maybe it's less common in other types of application development?
- RyanZAG 11y ago> update or render some game object that's died and been deleted Woah, woah. No, object pooling is not for that case and definitely doesn't help with that. If you've re-used the object for something else and you've still got a reference to it - and you're updating it - you've got some seriously nasty bugs and randomly broken code. Object pooling is not a 'must'. You need object pooling when you have a real time app and your total create/delete bandwidth exceeds the garbage collectors ability to process it in the 'early generation', forcing eventual 'stop the world' GC.
- pcwalton 11y agoIn fact, object pooling makes this problem worse. You suddenly have to remember to remove objects from the pool at the right time, resulting in memory leaks if you forget or use-after-free (though not the exploitable kind) if you do it too early. It's better than unsafe manual memory management from a security perspective, but it's not any better from an ergonomic perspective.
- vitalyd 11y agoYes, ergonomics suck. It's particularly frustrating since a lot of things that end up pooled are confined to current thread execution (e.g. request) and could easily be stack alloc'd if that facility was there. Along the same lines, sharing messages across threads where one would simply like to memcpy into a ringbuffer become a pain in the rear.
- RyanZAG 11y agoIn Java8, escape analysis usually catches most of the objects that can be moved to the stack, and they don't touch the heap anyway. https://en.wikipedia.org/wiki/Escape_analysis https://en.wikipedia.org/wiki/Escape_analysis http://www.burnison.ca/articles/escape-analysis-stack-allocated-objects http://www.burnison.ca/articles/escape-analysis-stack-alloca... It would be nice to be able to control it though. (value objects?)
- vitalyd 11y agoEscape analysis hasn't lived up to its promises/potential, IMO. Current Hotspot uses control flow insensitive escape detection, meaning an object that escapes on any path, even if that path is never taken, is considered escaping. This can be improved to be CF sensitive, and Graal does that. Secondly, beyond trivial examples, sufficient inlining needs to occur for compiler to have full horizon, and that can fail for a variety of reasons, even with run to run variance of exact same code shape. As you say, there's no control and you're fully at the mercy of the compiler's mood that day. This is well into Sufficiently Smart Compiler land ... The bottom line is that anyone for whom those allocations can be problematic take manual measures to ensure they don't happen and do not rely on EA. Value types should help for a few reasons, however they appear to be somewhat limited (e.g. immutable, cannot be passed by ref).
- amelius 11y agoI can see a security nightmare lurking around the corner.
- pcwalton 11y agoI don't know why this was downvoted. This vulnerability in Jetty was the result of sharing a buffer between requests: http://blog.gdssecurity.com/labs/2015/2/25/jetleak-vulnerability-remote-leakage-of-shared-buffers-in-je.html http://blog.gdssecurity.com/labs/2015/2/25/jetleak-vulnerabi...
- jaltekruse 11y agoThis is a reasonable concern, and as the other response mentions, it has happened, even to well-respected projects. I will however say that there are some ways to combat this if you are in a situation where security is an absolute must. While it removes some of the performance benefits, a simple defense is to include a call to clear the state of the object before letting it out of the pool, or upon returning it to the pool.
- twic 11y agoObject pooling was very common in Java last century. Today, it survives only for the most heavyweight objects: threads, database connections, web browsers, that sort of thing. What killed it off was generational collection. Compacting generational collectors make allocation very cheap, and they make maintenance of short-lived objects very cheap. As a result, it's reliably cheaper to allocate an object and throw it away than it is to attempt to reuse it. So it's surprising and interesting to read that the author of this article clearly thinks that there are still benefits to object pooling. Since Alex Petrov is a very sharp guy, i have to take him at his word. But it's a shame he didn't include benchmarks to compare it to idiomatic use of a modern GC.
- j_baker 11y agoWell, there are still cases where object pooling might make sense. Some games on mobile devices still use object pooling. I suppose I could also see the need to use object pooling for a very busy server or a very large hadoop job.
- vitalyd 11y agoIt's very much alive in the java low latency space where even young GCs are a problem. Allocations of temps/short lived objects also wash your cpu caches, and generally there are other reasons to avoid any GC in this space.
- ketralnis 11y ago> java low latency space where even young GCs are a problem ... there are other reasons to avoid any GC in this space I'm no Java hater, but that sounds like the wrong place to use Java in the first place
- vitalyd 11y agoIt has its pros/cons but you'd be surprised (perhaps) at the throughput and latency you can get if you're mindful of things to avoid.
- wangii 11y agoquestion: which is cheaper, stack object allocation/creation or object pooling, in languages like C/C++/Rust?
- pcwalton 11y agoGenerally the stack, because the cache behavior is better and you can allocate an entire activation record at one time. The compiler is also often able to optimize the allocations better, for example by promoting stack objects to registers.
- vitalyd 11y agoFor the most part stack allocs would be faster unless ctor does something ridiculously expensive. Also, one typically doesn't pool objects that are confined to the current thread's execution - it's mostly for heap references that outlive requests/transactions/etc.
- jacquesm 11y agoStack for sure. But if you want any kind of persistence then stack is right out and an object pool would be faster.
- tbirdz 11y agoThe stack is cheaper, but it is a limited size. If you need more objects than would fit on the stack, pooling might be a better choice.
- joelwilliamson 11y agoThe stack doesn't have to be limited. On a 64-bit system, you can give each thread a couple of terabytes of stack space with no problem.
- aidenn0 11y agoFor amd64 on linux you only have 128TB; with green threads I regularly use a lot more than 64 threads.
- zwerdlds 11y agoIt's been a while, but this was a common technique for agent based modeling in MASON back in the day. Having tens of thousands of agents constantly being garbage invalidated was the use case.
- lucb1e 11y agoIf this is a forgotten trick, then why did we learn about this in school? It was in Java classes though, never heard of it being needed elsewhere, but the concept is generally applicable.
- nemesisrobot 11y agoIIRC, LMAX Disruptor[0] uses the same concept with its ring buffer to avoid GC-related slowdown. [0]: https://github.com/LMAX-Exchange/disruptor https://github.com/LMAX-Exchange/disruptor
- vitalyd 11y agoWhile it does avoid GC, it's mostly a faster way to share data between threads than the builtin concurrent collections.
- nemesisrobot 11y agoYep, that's what Disruptor is for. Here's a blog post by one of the devs giving more details about the ring buffer where she specifically mentions that objects are pooled to avoid the GC http://mechanitis.blogspot.com/2011/06/dissecting-disruptor-whats-so-special.html http://mechanitis.blogspot.com/2011/06/dissecting-disruptor-...
- macrael 11y agoAnother example of this: table views in Cocoa. Enough cells to fill the screen are allocated and then just updated instead of re-created as cells scroll off screen.
- davelnewton 11y agoIf it's well-known it's not forgotten. (Not really sure it's forgotten, either; it's used regularly for heavy objects, or when speed is absolutely critical.)
- newobj 11y agoPooling and in-place reuse are two different things, to be sure. This was a fun thing I remember writing: https://github.com/newobj/ffwd/blob/master/Src/Core/Std/Allocator.h https://github.com/newobj/ffwd/blob/master/Src/Core/Std/Allo... When returned to the pool, each object used the first 4 bytes of itself to point to the next free one in a list, so there was no additional overhead.
- LCDninja 11y agoObject pooling is essential for performance in Unity 3D if you're dealing with large amounts of game objects.
- shanemhansen 11y agoObject pooling is alive and well. Take a look at go's sync.Pool.