3 ms·

From the tcmalloc paper[0]: > Speed is important for a malloc implementation because if malloc is not fast enough, application writers are inclined to write th

by dward 10y ago

From the tcmalloc paper[0]: > Speed is important for a malloc implementation because if malloc is not fast enough, application writers are inclined to write their own custom free lists on top of malloc. This can lead to extra complexity, and more memory usage unless the application writer is very careful to appropriately size the free lists and scavenge idle objects out of the free list. An example of such an optimization is the arena allocator[1][2] employed by protobuf. Custom memory management schemes are not uncommon in performance critical code. 0: http://goog-perftools.sourceforge.net/doc/tcmalloc.html http://goog-perftools.sourceforge.net/doc/tcmalloc.html 1: https://en.m.wikipedia.org/wiki/Region-based_memory_management https://en.m.wikipedia.org/wiki/Region-based_memory_manageme... 2: https://developers.google.com/protocol-buffers/docs/reference/arenas https://developers.google.com/protocol-buffers/docs/referenc...

haberman 10y ago
> An example of such an optimization is the arena allocator[1][2] employed by protobuf. I work on the protobuf team at Google, so I'm aware of this. Two things about that: 1. The underlying blocks for the arena allocator still come from the system allocator. 2. Because the arena allocator inhibits the capabilities of standard malloc-debugging tools like ASAN and Valgrind, the protobuf arena allocator includes special ASAN-aware code to mitigate this: https://github.com/google/protobuf/blob/d64a2d9941c36a7bc2a7959ea10ab8363192ac14/src/google/protobuf/arena.cc#L137 https://github.com/google/protobuf/blob/d64a2d9941c36a7bc2a7... However, that code is ASAN-specific. It won't help other tools like Valgrind. So yes, different allocators are sometimes warranted for specific patterns like arenas. But if all you want is plain malloc()/free(), you should call malloc()/free(). If you're writing a library, letting the user specify their own allocation callback is also great, since it lets the user do whatever custom bookkeeping/pooling/etc. they want to do. But by default just call malloc()/free() (IMHO).

by dward 10y ago

haberman 10y ago

> An example of such an optimization is the arena allocator[1][2] employed by protobuf. I work on the protobuf team at Google, so I'm aware of this. Two things about that: 1. The underlying blocks for the arena allocator still come from the system allocator. 2. Because the arena allocator inhibits the capabilities of standard malloc-debugging tools like ASAN and Valgrind, the protobuf arena allocator includes special ASAN-aware code to mitigate this: https://github.com/google/protobuf/blob/d64a2d9941c36a7bc2a7959ea10ab8363192ac14/src/google/protobuf/arena.cc#L137 https://github.com/google/protobuf/blob/d64a2d9941c36a7bc2a7... However, that code is ASAN-specific. It won't help other tools like Valgrind. So yes, different allocators are sometimes warranted for specific patterns like arenas. But if all you want is plain malloc()/free(), you should call malloc()/free(). If you're writing a library, letting the user specify their own allocation callback is also great, since it lets the user do whatever custom bookkeeping/pooling/etc. they want to do. But by default just call malloc()/free() (IMHO).