4 ms·

Jemalloc 1.0.0 released: the malloc used in FreeBSD, NetBSD, and Mozilla Firefox

22 points by self 16y ago

NathanKP 16y ago
Rather than linking to a paragraph on someone's blog, this ought to link directly to: http://www.canonware.com/jemalloc/ http://www.canonware.com/jemalloc/ It looks like in addition to jemalloc serving FreeBSD, NetBSD, and Mozilla, it is also used for Facebook servers: Since 2009 I have adapted jemalloc to handle the extreme loads Facebook servers commonly operate under. Facebook uses jemalloc in many components that are integral to serving its website.
- self 16y ago
  The blog entry was written by Jason Evans, the author of jemalloc.
  - NathanKP 16y ago
    Okay, I understand that, but judging from the briefness of the post and its lack of detail it looks like Jason Evans must have simply intended to direct his readers to the official site. Therefore it would seem to me that this post should have linked straight to the official site as well.
    - self 16y ago
      The official site doesn't have an announcement, though. It's not exactly "news" to link there.
shin_lao 16y ago
Before considering changing the allocator to improve performances, review the way you consume memory.
- yason 16y ago
  While you can try to not do anything stupid with regard to your allocations, there's often very little you can do. If you try to implement your own allocator you're often just wasting time. You either use the best allocator library you can find or end up having a half-baked allocator of your own on top of a poor default allocator library. A good allocator is essential for performance. Writing a good allocator is not trivial so your home-made allocator is likely to be slower, consume more memory than an optimized generic allocator, and exhibit all kinds of weird random hiccups when a critical malloc() or free() happens exactly when the library can't avoid housekeeping anymore and goes into a costly cleanup mode. An optimized allocator has loads of knowhow in guarding against these corner cases, both algorithmically and by having empirical special cases to help amortize the cost of maintenance. I've seen many programs that couldn't possibly have predicted their allocations any better but were still stuck until an alternate allocator was bolted in and the juices began to flow smoothly again. I expect a modern allocator to adapt to various use cases, from managing large variable-size chunks of memory to intelligently pooling a thousands and thousands of small objects of one size. And much more. Automatically.
  - shin_lao 16y ago
    Most software I audited overuse dynamic memory. Before considering changing the allocator, consider allocating less (zero copy, use of the stack, in place allocation, etc.).
  - ajross 16y ago
    I think the point was that a "home made" allocator is going to be able to exploit structure in the data that won't be visible to a general purpose malloc -- c.f. the pool allocators used in Apache, which know a priori that they'll be freeing everything at the end of the request. Likewise objects with the same lifetimes can live in the same block, etc...
jey 16y ago
Promoting it as "the malloc used in Firefox" isn't a good PR strategy. ;-)

22 points by self 16y ago

Rather than linking to a paragraph on someone's blog, this ought to link directly to: http://www.canonware.com/jemalloc/ http://www.canonware.com/jemalloc/ It looks like in addition to jemalloc serving FreeBSD, NetBSD, and Mozilla, it is also used for Facebook servers: Since 2009 I have adapted jemalloc to handle the extreme loads Facebook servers commonly operate under. Facebook uses jemalloc in many components that are integral to serving its website.

self 16y ago
The blog entry was written by Jason Evans, the author of jemalloc.
- NathanKP 16y ago
  Okay, I understand that, but judging from the briefness of the post and its lack of detail it looks like Jason Evans must have simply intended to direct his readers to the official site. Therefore it would seem to me that this post should have linked straight to the official site as well.
  - self 16y ago
    The official site doesn't have an announcement, though. It's not exactly "news" to link there.

shin_lao 16y ago

Before considering changing the allocator to improve performances, review the way you consume memory.

yason 16y ago
While you can try to not do anything stupid with regard to your allocations, there's often very little you can do. If you try to implement your own allocator you're often just wasting time. You either use the best allocator library you can find or end up having a half-baked allocator of your own on top of a poor default allocator library. A good allocator is essential for performance. Writing a good allocator is not trivial so your home-made allocator is likely to be slower, consume more memory than an optimized generic allocator, and exhibit all kinds of weird random hiccups when a critical malloc() or free() happens exactly when the library can't avoid housekeeping anymore and goes into a costly cleanup mode. An optimized allocator has loads of knowhow in guarding against these corner cases, both algorithmically and by having empirical special cases to help amortize the cost of maintenance. I've seen many programs that couldn't possibly have predicted their allocations any better but were still stuck until an alternate allocator was bolted in and the juices began to flow smoothly again. I expect a modern allocator to adapt to various use cases, from managing large variable-size chunks of memory to intelligently pooling a thousands and thousands of small objects of one size. And much more. Automatically.
- shin_lao 16y ago
  Most software I audited overuse dynamic memory. Before considering changing the allocator, consider allocating less (zero copy, use of the stack, in place allocation, etc.).
- ajross 16y ago
  I think the point was that a "home made" allocator is going to be able to exploit structure in the data that won't be visible to a general purpose malloc -- c.f. the pool allocators used in Apache, which know a priori that they'll be freeing everything at the end of the request. Likewise objects with the same lifetimes can live in the same block, etc...

jey 16y ago

Promoting it as "the malloc used in Firefox" isn't a good PR strategy. ;-)