3 ms·

> C: doesn't do weird stuff like bypassing malloc(), avoids undefined behavior. Can you elaborate?

jmspring 10y ago
It's a bit of a naive comment, because it wholly depends on the libraries you include and what those do. Use any of the debugging libraries that shim in their own alloc routines, you bypass malloc (or at least do a bunch of things that impact what gets malloc'd).
- haberman 10y ago
  Sure, but those libraries exist for the explicit purpose of giving malloc() special behavior. I'm talking about general-purpose libraries that implement their own malloc(), thereby preventing the use of these malloc()-hooking libraries.
dward 10y ago
From the tcmalloc paper[0]: > Speed is important for a malloc implementation because if malloc is not fast enough, application writers are inclined to write their own custom free lists on top of malloc. This can lead to extra complexity, and more memory usage unless the application writer is very careful to appropriately size the free lists and scavenge idle objects out of the free list. An example of such an optimization is the arena allocator[1][2] employed by protobuf. Custom memory management schemes are not uncommon in performance critical code. 0: http://goog-perftools.sourceforge.net/doc/tcmalloc.html http://goog-perftools.sourceforge.net/doc/tcmalloc.html 1: https://en.m.wikipedia.org/wiki/Region-based_memory_management https://en.m.wikipedia.org/wiki/Region-based_memory_manageme... 2: https://developers.google.com/protocol-buffers/docs/reference/arenas https://developers.google.com/protocol-buffers/docs/referenc...
- haberman 10y ago
  > An example of such an optimization is the arena allocator[1][2] employed by protobuf. I work on the protobuf team at Google, so I'm aware of this. Two things about that: 1. The underlying blocks for the arena allocator still come from the system allocator. 2. Because the arena allocator inhibits the capabilities of standard malloc-debugging tools like ASAN and Valgrind, the protobuf arena allocator includes special ASAN-aware code to mitigate this: https://github.com/google/protobuf/blob/d64a2d9941c36a7bc2a7959ea10ab8363192ac14/src/google/protobuf/arena.cc#L137 https://github.com/google/protobuf/blob/d64a2d9941c36a7bc2a7... However, that code is ASAN-specific. It won't help other tools like Valgrind. So yes, different allocators are sometimes warranted for specific patterns like arenas. But if all you want is plain malloc()/free(), you should call malloc()/free(). If you're writing a library, letting the user specify their own allocation callback is also great, since it lets the user do whatever custom bookkeeping/pooling/etc. they want to do. But by default just call malloc()/free() (IMHO).
bjourne 10y ago
http://www.tedunangst.com/flak/post/analysis-of-openssl-freelist-reuse http://www.tedunangst.com/flak/post/analysis-of-openssl-free... http://www.tedunangst.com/flak/post/heartbleed-vs-mallocconf http://www.tedunangst.com/flak/post/heartbleed-vs-mallocconf
therein 10y ago
He might be referring to "sbrk" [0] [0] http://linux.die.net/man/2/sbrk http://linux.die.net/man/2/sbrk

It's a bit of a naive comment, because it wholly depends on the libraries you include and what those do. Use any of the debugging libraries that shim in their own alloc routines, you bypass malloc (or at least do a bunch of things that impact what gets malloc'd).

haberman 10y ago
Sure, but those libraries exist for the explicit purpose of giving malloc() special behavior. I'm talking about general-purpose libraries that implement their own malloc(), thereby preventing the use of these malloc()-hooking libraries.

dward 10y ago

From the tcmalloc paper[0]: > Speed is important for a malloc implementation because if malloc is not fast enough, application writers are inclined to write their own custom free lists on top of malloc. This can lead to extra complexity, and more memory usage unless the application writer is very careful to appropriately size the free lists and scavenge idle objects out of the free list. An example of such an optimization is the arena allocator[1][2] employed by protobuf. Custom memory management schemes are not uncommon in performance critical code. 0: http://goog-perftools.sourceforge.net/doc/tcmalloc.html http://goog-perftools.sourceforge.net/doc/tcmalloc.html 1: https://en.m.wikipedia.org/wiki/Region-based_memory_management https://en.m.wikipedia.org/wiki/Region-based_memory_manageme... 2: https://developers.google.com/protocol-buffers/docs/reference/arenas https://developers.google.com/protocol-buffers/docs/referenc...

haberman 10y ago
> An example of such an optimization is the arena allocator[1][2] employed by protobuf. I work on the protobuf team at Google, so I'm aware of this. Two things about that: 1. The underlying blocks for the arena allocator still come from the system allocator. 2. Because the arena allocator inhibits the capabilities of standard malloc-debugging tools like ASAN and Valgrind, the protobuf arena allocator includes special ASAN-aware code to mitigate this: https://github.com/google/protobuf/blob/d64a2d9941c36a7bc2a7959ea10ab8363192ac14/src/google/protobuf/arena.cc#L137 https://github.com/google/protobuf/blob/d64a2d9941c36a7bc2a7... However, that code is ASAN-specific. It won't help other tools like Valgrind. So yes, different allocators are sometimes warranted for specific patterns like arenas. But if all you want is plain malloc()/free(), you should call malloc()/free(). If you're writing a library, letting the user specify their own allocation callback is also great, since it lets the user do whatever custom bookkeeping/pooling/etc. they want to do. But by default just call malloc()/free() (IMHO).

bjourne 10y ago

http://www.tedunangst.com/flak/post/analysis-of-openssl-freelist-reuse http://www.tedunangst.com/flak/post/analysis-of-openssl-free... http://www.tedunangst.com/flak/post/heartbleed-vs-mallocconf http://www.tedunangst.com/flak/post/heartbleed-vs-mallocconf

therein 10y ago

He might be referring to "sbrk" [0] [0] http://linux.die.net/man/2/sbrk http://linux.die.net/man/2/sbrk