4 ms·
Erlang has a parameter called initial_heap_size. Each new actor-process in Erlang gets its own isolated heap, for which it does its own garbage-collection on it
by derefr 7y ago
Erlang has a parameter called initial_heap_size. Each new actor-process in Erlang gets its own isolated heap, for which it does its own garbage-collection on its own execution thread. This initial_heap_size parameter determines how large each newly-spawned actor’s heap will be.
Why would you tune it? Because, if you set it high enough, then for all your short-lived actors, memory allocation will become a no-op (= bump allocation), and the actor will never experience enough memory-pressure to trigger a garbage-collection pass, before the actor exits and the entire process heap can be deallocated as a block. The actor will just “leak” memory onto its heap, and then exit, never having had to spend time accounting for it.
This is also done in many video games, where there is a per-frame temporaries heap that has its free pointer reset at the start of each frame. Rather than individually garbage-collecting these values, they can all just be invalidated at once at the end of the frame.
The usual name for such “heaps you pre-allocate to a capacity you’ve tuned to ensure you will never run out of, and then deallocate as a whole later on” is a memory arena. See https://en.wikipedia.org/wiki/Region-based_memory_management https://en.wikipedia.org/wiki/Region-based_memory_management for more examples of memory arenas.
- saagarjha 7y agoNote that most general-purpose allocators also keep around internal arenas from which they hand out memory.
- catblast 7y agoNot sure how this is related. A general purpose allocator with a plain malloc interface can’t use this to do anything useful wrt lifetime because there is no correlation to lifetime provided by the interface. Internal arenas can be useful to address contention and fragmentation.
- saagarjha 7y agoI'm pointing out that an arena is more about "a region of memory that you can split up to use later" than "a region of memory that must be allocated and deallocated all at once".
- asdfasgasdgasdg 7y agoI've never heard that use of the term "arena". Are you thinking of slabs? Arenas are typically allocated and deallocated at once. That's their main feature.
- saagarjha 7y agohttps://sourceware.org/glibc/wiki/MallocInternals#Arenas_and_Heaps https://sourceware.org/glibc/wiki/MallocInternals#Arenas_and...
- asdfasgasdgasdg 7y agoSeems like a bit of an ideosyncratic use of the word. In tcmalloc these per thread zones are just called the thread cache (hence the name "thread caching malloc").
- saagarjha 7y agoGlibc’s malloc has its own thread cache, which contains small chunks that can be accessed without locking at all: https://sourceware.org/glibc/wiki/MallocInternals#Thread_Local_Cache_.28tcache.29 https://sourceware.org/glibc/wiki/MallocInternals#Thread_Loc.... (Interestingly, this cache used to be so optimized for speed at some point that it made a popular target for attackers, because it lacked many safety checks.)
- needusername 7y ago> memory allocation will become a no-op (= bump allocation) No, that's a cache miss.
- Tuna-Fish 7y agoNo, as memory is allocated linearly the cpu prefetchers will most likely keep the heap in cache.
- twic 7y agoSpeaking of which: https://www.opsian.com/blog/jvms-allocateprefetch-options/ https://www.opsian.com/blog/jvms-allocateprefetch-options/
- needusername 7y agoNo, in practice this is not true. You will also need to write out to memory all the newly allocated objects which you don't need anymore. https://podcasts.apple.com/us/podcast/hidden-gc-bandwidth-costs/id1286422919?i=1000420777005 https://podcasts.apple.com/us/podcast/hidden-gc-bandwidth-co...
- imtringued 7y agoHow is that related to memory allocation costs? We are talking about the cost of obtaining a chunk of memory. The cost of actually creating an object is allowed to be much higher because constructors are allowed to execute any arbitrary code. Just think about how expensive it would be to allocate a 3d vector consisting of 3 floats with malloc() 20000 times and then later deallocate it. Nobody is worrying about the cost of writing 3 floats to RAM. Everyone is worrying about the cost of malloc traversing a free list and it causing memory fragmentation in the process. Meanwhile the arena allocator would be at least as efficient as using an array of 3d vectors.
- needusername 7y ago> How is that related to memory allocation costs? It's a cost that has to be paid when using bump pointer allocation. > We are talking about the cost of obtaining a chunk of memory. The cost of actually creating an object is allowed to be much higher because constructors are allowed to execute any arbitrary code. Accessing main memory is about two orders of magnitude slower than accessing L1. For that time you can run a lot of arbitrary code that accesses data in L1 and registers. > Just think about how expensive it would be to allocate a 3d vector consisting of 3 floats with malloc() 20000 times and then later deallocate it. Nobody is worrying about the cost of writing 3 floats to RAM. Everyone is worrying about the cost of malloc traversing a free list and it causing memory fragmentation in the process. Meanwhile the arena allocator would be at least as efficient as using an array of 3d vectors. malloc doesn't mandate free lists, other implementations exist. It's not about relative costs. OP claimed bump pointer allocation to be a "no-op" when it's clearly not.
- monocasa 7y agoSo I kind of disagree with the idea that arenas are all about deallocation at once. There's other contexts where you have separate arenas but don't plan on deallocating in blocks, mainly around when you have memory with different underlying semantics. "This block of memory is faster, but not coherent and needs to be manually flushed for DMA", "this block of memory is fastest but just not DMA capable at all", "there's only 2k of this memory, but it's basically cache", "this memory is large, fairly slow, but can do whatever", "this block of memory is non volatile, but memory mapped", etc. I'd say that arenas are kind of a superset of both what you and I are talking about.
- hinkley 7y agoI can’t remember the last time I read C code, but I do recall a particular time when I was reading a library that had been written with a great deal of attention to reliability. The first thing it did was allocate enough memory for the shutdown operations. That way on a malloc() failure, it could still do a a completely orderly shutdown. Or never start in the first place. From that standpoint, you could also categorize arenas on a priority basis. This one is for recovery operations, this one for normal operation, and whatever is left for low priority tasks.
- bch 7y ago> The first thing it did was allocate enough memory for the shutdown operations. That is clever and beautiful. Have to look for chances to do similar to see if I can establish a new habit myself.
- Someone 7y agoThat strategy is more important on systems that don’t do demand paged virtual memory. In Think Class Library on classic Mac OS, it was called the “Rainy day fund”. One can also do that in stages: - allocate a large block at startup - when running out of memory, reallocate it at a smaller size and warn the user - when running out of memory again, free the block and attempt an orderly shutdown.
- MaxBarraclough 7y ago
- dahart 7y agoThe games and GPU apps I’ve worked on use memory pools for small allocations, where there will be individual pools for all, say, 1-16 byte allocations, 16-64 byte allocations, 64-256 byte allocations, etc. (Sizes just for illustration, not necessarily realistic). The pool sizes always get tuned over time to match the approximate high water mark of the application. I think pools and arenas mean pretty much the same thing. https://en.wikipedia.org/wiki/Memory_pool https://en.wikipedia.org/wiki/Memory_pool I’ve mostly heard this discussed in terms of pools, but I wonder if there’s a subtle difference, or if there’s a historical reason arena is popular in some circles and pool in others...? I haven’t personally see a per-frame heap while working in console games, even though games I’ve worked on probably had one, or something like it. Techniques that I did see and are super-common are fixed maximum-size allocations: just pre-allocate all the memory you’ll ever need for some feature and never let it go; stack allocations sometimes with alloca(); and helper functions/classes that put something on the stack for the lifetime of a particular scope.
- monocasa 7y agoI've seen Jason Gregory talk about per frame arenas in Game Engine Architecture as a fundamental piece of how the Naughty Dog engines tend to work. Totally agreed that they aren't required for shipping great console games (and they're really hard to use effectively in C++ since you're pretty much guaranteed to have hanging references if you don't have ascetic levels of discipline). This is mainly just meant as a "here's an example of how they can be used and are by at least one shop".
- foota 7y agoSeems like this could be handled with a wrapper type with runtime checks during debug? Like make any pointer to the per frame allocation be a TempPointer or something and then assert they're all gone with a static count variable of them? Then you just have to be cautious whenever you pass a reference to one or convert to a raw pointer. I don't think this would be too awful for performance in debug builds.
- monocasa 7y ago