5 ms·

For those wondering why on earth this isn’t simply a contribution to Ruby, the FAQ has a really nice explanation - tldr is that this is a variant optimised for

For those wondering why on earth this isn’t simply a contribution to Ruby, the FAQ has a really nice explanation - tldr is that this is a variant optimised for the very specific context of Linux, x86/64, server environments, with pre compiled and packaged binaries, with special memory allocators (jemalloc). These changes are specific to this context and are unlikely to be accepted into the more generalised Ruby core. Same team as Phusion Passenger, so excellent background.

joelbluminator 5y ago
I'm not really buying the FAQ explanation, idk this feels too good to be true. Would have loved to see some core team discussion on this if there was one.
- FooBarWidget 5y ago
  Author of Fullstaq Ruby here. There have been discussions. Discussions on jemalloc integration preceded Fullstaq Ruby for years. The bug tracker issue on the malloc_trim patch went stale. All this can be found on the issue tracker. I also spoke to Matz during EuRuKo 2019. And this is the impression I got.
  - joelbluminator 5y ago
    Are you referring to this one? https://bugs.ruby-lang.org/issues/15667 https://bugs.ruby-lang.org/issues/15667 P.S did Discourse adopt this approach ? Also found a discussion on jemalloc https://bugs.ruby-lang.org/issues/14718 https://bugs.ruby-lang.org/issues/14718
    - FooBarWidget 5y ago
      Yes
      - deleted 5y ago
        [deleted]
    - throwdbaaway 5y ago
      My observations after reading https://bugs.ruby-lang.org/issues/14718 https://bugs.ruby-lang.org/issues/14718: - Shyouhei Urabe did a marvelous job of resisting pressure from several well-known developers representing multiple high profile ruby companies/projects. - Everyone eventually came to their senses in https://bugs.ruby-lang.org/issues/14718#note-89 https://bugs.ruby-lang.org/issues/14718#note-89, and concluded that the best way forward is to use glibc with MALLOC_ARENA_MAX=2. My personal feeling is that while the "Jemalloc FTW" section in http://oldblog.antirez.com/post/everything-about-redis-24.html http://oldblog.antirez.com/post/everything-about-redis-24.ht... might have been correct back in the jemalloc 3.x days, a lot has changed on both jemalloc and glibc since then. Furthermore, there is also musl in the equation these days.
      - joelbluminator 5y ago
        I tend to be skeptical as well. Why isn't Shopify / Github / Heroku adopting this? I'd stay away.
        FooBarWidget 5y ago
        Many of them are adopting Jemalloc, but they do the Jemalloc integration work themselves because they have the engineering talent and capacity. Fullstaq Ruby is for everybody else: those who can't, don't want to, or can't be bothered to do it themselves. It's for the masses.
        strzibny 5y ago
        You don't need special talent to do that. I actually have a mention of this in my book on deploying Rails (https://deploymentfromscratch.com/ https://deploymentfromscratch.com/). Here's a one liner with ruby-install: $ sudo dnf install -y jemalloc-devel $ ruby-install 3.0.0 -- --with-jemalloc Then check you have it: $ ruby -r rbconfig -e "puts RbConfig::CONFIG['LIBS']" ... -ljemalloc
        FooBarWidget 5y ago
- rurban 5y ago
  Everybody can do it by themselves. Using jemalloc is trivial. Adding the malloc_trim() call to the full GC is trivial. The patches are open. Why MRI ruby likes to keep unused memory pages around is only, that they don't like to add that compile-time option and probe for jemalloc. It's too trivial to worry
  - FooBarWidget 5y ago
    Here I explain why it can quickly become not so trivial: https://news.ycombinator.com/item?id=27872696 https://news.ycombinator.com/item?id=27872696
jashmatthews 5y ago
jemalloc isn’t a special allocator. It’s basically the FreeBSD default.
deleted 5y ago
[deleted]

joelbluminator 5y ago

I'm not really buying the FAQ explanation, idk this feels too good to be true. Would have loved to see some core team discussion on this if there was one.

FooBarWidget 5y ago
Author of Fullstaq Ruby here. There have been discussions. Discussions on jemalloc integration preceded Fullstaq Ruby for years. The bug tracker issue on the malloc_trim patch went stale. All this can be found on the issue tracker. I also spoke to Matz during EuRuKo 2019. And this is the impression I got.
- joelbluminator 5y ago
  Are you referring to this one? https://bugs.ruby-lang.org/issues/15667 https://bugs.ruby-lang.org/issues/15667 P.S did Discourse adopt this approach ? Also found a discussion on jemalloc https://bugs.ruby-lang.org/issues/14718 https://bugs.ruby-lang.org/issues/14718
  - FooBarWidget 5y ago
    Yes
    - deleted 5y ago
      [deleted]
  - throwdbaaway 5y ago
    My observations after reading https://bugs.ruby-lang.org/issues/14718 https://bugs.ruby-lang.org/issues/14718: - Shyouhei Urabe did a marvelous job of resisting pressure from several well-known developers representing multiple high profile ruby companies/projects. - Everyone eventually came to their senses in https://bugs.ruby-lang.org/issues/14718#note-89 https://bugs.ruby-lang.org/issues/14718#note-89, and concluded that the best way forward is to use glibc with MALLOC_ARENA_MAX=2. My personal feeling is that while the "Jemalloc FTW" section in http://oldblog.antirez.com/post/everything-about-redis-24.html http://oldblog.antirez.com/post/everything-about-redis-24.ht... might have been correct back in the jemalloc 3.x days, a lot has changed on both jemalloc and glibc since then. Furthermore, there is also musl in the equation these days.
    - joelbluminator 5y ago
      I tend to be skeptical as well. Why isn't Shopify / Github / Heroku adopting this? I'd stay away.
      - FooBarWidget 5y ago
        Many of them are adopting Jemalloc, but they do the Jemalloc integration work themselves because they have the engineering talent and capacity. Fullstaq Ruby is for everybody else: those who can't, don't want to, or can't be bothered to do it themselves. It's for the masses.
        strzibny 5y ago
        You don't need special talent to do that. I actually have a mention of this in my book on deploying Rails (https://deploymentfromscratch.com/ https://deploymentfromscratch.com/). Here's a one liner with ruby-install: $ sudo dnf install -y jemalloc-devel $ ruby-install 3.0.0 -- --with-jemalloc Then check you have it: $ ruby -r rbconfig -e "puts RbConfig::CONFIG['LIBS']" ... -ljemalloc
        FooBarWidget 5y ago
rurban 5y ago
Everybody can do it by themselves. Using jemalloc is trivial. Adding the malloc_trim() call to the full GC is trivial. The patches are open. Why MRI ruby likes to keep unused memory pages around is only, that they don't like to add that compile-time option and probe for jemalloc. It's too trivial to worry
- FooBarWidget 5y ago
  Here I explain why it can quickly become not so trivial: https://news.ycombinator.com/item?id=27872696 https://news.ycombinator.com/item?id=27872696

jashmatthews 5y ago

jemalloc isn’t a special allocator. It’s basically the FreeBSD default.

deleted 5y ago

[deleted]