4 ms·

My observations after reading https://bugs.ruby-lang.org/issues/14718 https://bugs.ruby-lang.org/issues/14718: - Shyouhei Urabe did a marvelous job of resistin

My observations after reading https://bugs.ruby-lang.org/issues/14718 https://bugs.ruby-lang.org/issues/14718: - Shyouhei Urabe did a marvelous job of resisting pressure from several well-known developers representing multiple high profile ruby companies/projects. - Everyone eventually came to their senses in https://bugs.ruby-lang.org/issues/14718#note-89 https://bugs.ruby-lang.org/issues/14718#note-89, and concluded that the best way forward is to use glibc with MALLOC_ARENA_MAX=2. My personal feeling is that while the "Jemalloc FTW" section in http://oldblog.antirez.com/post/everything-about-redis-24.html http://oldblog.antirez.com/post/everything-about-redis-24.ht... might have been correct back in the jemalloc 3.x days, a lot has changed on both jemalloc and glibc since then. Furthermore, there is also musl in the equation these days.

joelbluminator 5y ago
I tend to be skeptical as well. Why isn't Shopify / Github / Heroku adopting this? I'd stay away.
- FooBarWidget 5y ago
  Many of them are adopting Jemalloc, but they do the Jemalloc integration work themselves because they have the engineering talent and capacity. Fullstaq Ruby is for everybody else: those who can't, don't want to, or can't be bothered to do it themselves. It's for the masses.
  - strzibny 5y ago
    You don't need special talent to do that. I actually have a mention of this in my book on deploying Rails (https://deploymentfromscratch.com/ https://deploymentfromscratch.com/). Here's a one liner with ruby-install: $ sudo dnf install -y jemalloc-devel $ ruby-install 3.0.0 -- --with-jemalloc Then check you have it: $ ruby -r rbconfig -e "puts RbConfig::CONFIG['LIBS']" ... -ljemalloc
    - FooBarWidget 5y ago
      It's 2021. Installing from source is no longer a low-friction, non-scary, default thing to do or something that people want to bother with. Users now expect binaries and proper package management. Furthermore, it's not just "dnf install jemalloc-devel". The Jemalloc version matters a lot. For reasons that are not yet clear, significant memory savings are only achieved with Jemalloc 3, not with Jemalloc 5. And your distribution only ships 1 Jemalloc version. So likely you need to compile Jemalloc 3 yourself. But Jemalloc 3 no longer compiles by default on some modern distributions, such as Debian 10. Fullstaq Ruby fixes this by patching Jemalloc: https://github.com/fullstaq-labs/fullstaq-ruby-server-edition/blob/main/resources/jemalloc_cxx_fix.patch https://github.com/fullstaq-labs/fullstaq-ruby-server-editio... Another caveat is that LD_PRELOADing Jemalloc is more effective than compiling with --with-jemalloc. Furthermore, installing Ruby is one thing. Keeping it security-patched without having to constantly monitor things, is a whole other can of worms. Here's where Fullstaq Ruby's second value proposition comes in: we supply DEB/RPMs and do the tiny version patching for you, you don't have to go through the whole compilation process and gem migration process every time there's a new Ruby tiny version. See also this FAQ entry, "Fullstaq Ruby vs LD_PRELOADing Jemalloc yourself": https://github.com/fullstaq-labs/fullstaq-ruby-server-edition#vs-ld_preloading-jemalloc-yourself https://github.com/fullstaq-labs/fullstaq-ruby-server-editio...
  - joelbluminator 5y ago
    I'm sure Ruby would have adopted it if it would have benefitted most users . It's either not benefitting most, is gonna break with future changes (like Guilds/Ractors) or a combination of the two. Sam Saffron: "After spending a bit too much time thinking about this, I would like to recommend against any jemalloc related changes " https://bugs.ruby-lang.org/issues/14718#note-89 https://bugs.ruby-lang.org/issues/14718#note-89
    - FooBarWidget 5y ago
      Do you believe copy-on-write friendly garbage collection — which saves ~33% memory, and is essential to Passenger's "smart spawning" methode, Unicork's preload_app mode, Puma's cluster mode — is a thing that benefits most users? Besides being the author of Fullstaq Ruby, I am also the author of the first copy-on-write-GC-friendliness patch for Ruby. Consider the fact that it took ~5 years for the copy-on-write-friendly-GC feature to find its way back to upstream Ruby. If you lived in 2008, would you be willing to wait 5 years in order to save 33% memory?
      - joelbluminator 5y ago
        You're not really answering to anything I said though. Sorry man I get that this is emotional for you, not trying to bring you down. You need to see it through my eyes; I'm not gonna introduce into my company anything that's high risk, what's in it for me? If this goes fine I maybe saved some memory, if this goes wrong I'm gonna look silly (how do I even justify going this route to anyone? it's not like the internet is clear this is the way to go; it's quite controversial or even not the way to go).
        FooBarWidget 5y ago
        I think you're misunderstanding me. I am not selling you anything. I make no money off this. I am not invested in whether you personally choose to move forward or not. My point is different. Your original statement basically equates quality with authority: you believe a project is qualitatively good if and only if it's officially blessed by an authority you trust. And that belief is fine. Most people either aren't able to, or don't care enough to, verify quality themselves, so we defer to trusted authorities. Nothing wrong with that approach in general. The thing I'm trying to tell you though, is that in case of MRI, there have been multiple past cases when the lack of official authority blessing did not in fact mean a lack of quality. This isn't hypothetical: there's a proven track record. My own track record is proven as well. What is in it for you? Well, reduced memory usage, improved performance, faster and lower-hassle installation and security patching! There is always a risk, yes. But all I'm trying to say is that the equation "lack of authority blessing == lack of quality" is proven not to be true for a Ruby project such as this. Whether you want to take the risk of trying it out, that's entirely up to you.
  - joelbluminator 5y ago
    Any source for Shopify / Github / Discourse adopting Jemalloc?
    - FooBarWidget 5y ago
      Source for Github: https://twitter.com/eileencodes/status/1289310891914088448 https://twitter.com/eileencodes/status/1289310891914088448
- ksec 5y ago
  Dont know about Heroku, but Github, Gitlab, Shopify are all using Jemalloc in production. Practically all big Rails shop are using it to the point I thought it was common knowledge in 2021. There has been 8 - 10 glibc release since then I am not sure if things have improved or not.
  - joelbluminator 5y ago
    Not really common knowledge, where is Shopify talking about jemalloc? I can't find anything. Github there's a tweet confirming it, they never really wrote about what they're doing with jemalloc afaik. I mean wrote as in not a tweet. But yes Eileen confirmed they use it. If people blogged about it more it would have become common knowledge I guess.
throw288199 5y ago
Shyouhei 's response https://twitter.com/shyouhei/status/1416924523979448322 https://twitter.com/shyouhei/status/1416924523979448322
- FooBarWidget 5y ago
  Machine translation isn't very accurate. Can any Japanese-speaking person translate this?
  - emodendroket 5y ago
    Best effort: > Wow, a lot of comments. What Fullstaq Ruby is doing is difficult to add into mainline, so my overall impression is I'm thankful someone's going to the trouble of doing it themselves. > It's not quite right to say that on Linux with jemalloc it uses less memory. In reality, it would be more correct to say that using a super-old version of jemalloc that for some reason used less memory leads to reduced memory usage (for unknown reasons). So it can't go into mainline. > That said, I can understand that some people are just happy to see reduced memory usage, so if those people want to keep using the super-old version of jemalloc with reduced memory usage, I have no objection. Go for it, is my feeling.

joelbluminator 5y ago

I tend to be skeptical as well. Why isn't Shopify / Github / Heroku adopting this? I'd stay away.

FooBarWidget 5y ago
Many of them are adopting Jemalloc, but they do the Jemalloc integration work themselves because they have the engineering talent and capacity. Fullstaq Ruby is for everybody else: those who can't, don't want to, or can't be bothered to do it themselves. It's for the masses.
- strzibny 5y ago
  You don't need special talent to do that. I actually have a mention of this in my book on deploying Rails (https://deploymentfromscratch.com/ https://deploymentfromscratch.com/). Here's a one liner with ruby-install: $ sudo dnf install -y jemalloc-devel $ ruby-install 3.0.0 -- --with-jemalloc Then check you have it: $ ruby -r rbconfig -e "puts RbConfig::CONFIG['LIBS']" ... -ljemalloc
  - FooBarWidget 5y ago
    It's 2021. Installing from source is no longer a low-friction, non-scary, default thing to do or something that people want to bother with. Users now expect binaries and proper package management. Furthermore, it's not just "dnf install jemalloc-devel". The Jemalloc version matters a lot. For reasons that are not yet clear, significant memory savings are only achieved with Jemalloc 3, not with Jemalloc 5. And your distribution only ships 1 Jemalloc version. So likely you need to compile Jemalloc 3 yourself. But Jemalloc 3 no longer compiles by default on some modern distributions, such as Debian 10. Fullstaq Ruby fixes this by patching Jemalloc: https://github.com/fullstaq-labs/fullstaq-ruby-server-edition/blob/main/resources/jemalloc_cxx_fix.patch https://github.com/fullstaq-labs/fullstaq-ruby-server-editio... Another caveat is that LD_PRELOADing Jemalloc is more effective than compiling with --with-jemalloc. Furthermore, installing Ruby is one thing. Keeping it security-patched without having to constantly monitor things, is a whole other can of worms. Here's where Fullstaq Ruby's second value proposition comes in: we supply DEB/RPMs and do the tiny version patching for you, you don't have to go through the whole compilation process and gem migration process every time there's a new Ruby tiny version. See also this FAQ entry, "Fullstaq Ruby vs LD_PRELOADing Jemalloc yourself": https://github.com/fullstaq-labs/fullstaq-ruby-server-edition#vs-ld_preloading-jemalloc-yourself https://github.com/fullstaq-labs/fullstaq-ruby-server-editio...
- joelbluminator 5y ago
  I'm sure Ruby would have adopted it if it would have benefitted most users . It's either not benefitting most, is gonna break with future changes (like Guilds/Ractors) or a combination of the two. Sam Saffron: "After spending a bit too much time thinking about this, I would like to recommend against any jemalloc related changes " https://bugs.ruby-lang.org/issues/14718#note-89 https://bugs.ruby-lang.org/issues/14718#note-89
  - FooBarWidget 5y ago
    Do you believe copy-on-write friendly garbage collection — which saves ~33% memory, and is essential to Passenger's "smart spawning" methode, Unicork's preload_app mode, Puma's cluster mode — is a thing that benefits most users? Besides being the author of Fullstaq Ruby, I am also the author of the first copy-on-write-GC-friendliness patch for Ruby. Consider the fact that it took ~5 years for the copy-on-write-friendly-GC feature to find its way back to upstream Ruby. If you lived in 2008, would you be willing to wait 5 years in order to save 33% memory?
    - joelbluminator 5y ago
      You're not really answering to anything I said though. Sorry man I get that this is emotional for you, not trying to bring you down. You need to see it through my eyes; I'm not gonna introduce into my company anything that's high risk, what's in it for me? If this goes fine I maybe saved some memory, if this goes wrong I'm gonna look silly (how do I even justify going this route to anyone? it's not like the internet is clear this is the way to go; it's quite controversial or even not the way to go).
      - FooBarWidget 5y ago
        I think you're misunderstanding me. I am not selling you anything. I make no money off this. I am not invested in whether you personally choose to move forward or not. My point is different. Your original statement basically equates quality with authority: you believe a project is qualitatively good if and only if it's officially blessed by an authority you trust. And that belief is fine. Most people either aren't able to, or don't care enough to, verify quality themselves, so we defer to trusted authorities. Nothing wrong with that approach in general. The thing I'm trying to tell you though, is that in case of MRI, there have been multiple past cases when the lack of official authority blessing did not in fact mean a lack of quality. This isn't hypothetical: there's a proven track record. My own track record is proven as well. What is in it for you? Well, reduced memory usage, improved performance, faster and lower-hassle installation and security patching! There is always a risk, yes. But all I'm trying to say is that the equation "lack of authority blessing == lack of quality" is proven not to be true for a Ruby project such as this. Whether you want to take the risk of trying it out, that's entirely up to you.
- joelbluminator 5y ago
  Any source for Shopify / Github / Discourse adopting Jemalloc?
  - FooBarWidget 5y ago
    Source for Github: https://twitter.com/eileencodes/status/1289310891914088448 https://twitter.com/eileencodes/status/1289310891914088448
ksec 5y ago
Dont know about Heroku, but Github, Gitlab, Shopify are all using Jemalloc in production. Practically all big Rails shop are using it to the point I thought it was common knowledge in 2021. There has been 8 - 10 glibc release since then I am not sure if things have improved or not.
- joelbluminator 5y ago
  Not really common knowledge, where is Shopify talking about jemalloc? I can't find anything. Github there's a tweet confirming it, they never really wrote about what they're doing with jemalloc afaik. I mean wrote as in not a tweet. But yes Eileen confirmed they use it. If people blogged about it more it would have become common knowledge I guess.

throw288199 5y ago

Shyouhei 's response https://twitter.com/shyouhei/status/1416924523979448322 https://twitter.com/shyouhei/status/1416924523979448322

FooBarWidget 5y ago
Machine translation isn't very accurate. Can any Japanese-speaking person translate this?
- emodendroket 5y ago
  Best effort: > Wow, a lot of comments. What Fullstaq Ruby is doing is difficult to add into mainline, so my overall impression is I'm thankful someone's going to the trouble of doing it themselves. > It's not quite right to say that on Linux with jemalloc it uses less memory. In reality, it would be more correct to say that using a super-old version of jemalloc that for some reason used less memory leads to reduced memory usage (for unknown reasons). So it can't go into mainline. > That said, I can understand that some people are just happy to see reduced memory usage, so if those people want to keep using the super-old version of jemalloc with reduced memory usage, I have no objection. Go for it, is my feeling.