6 ms·

15% faster is great. But at what cost? > Since Ruby 3.3.0-preview2 YJIT generates more code than Ruby 3.2.2 YJIT, this can result in YJIT having a higher memor

15% faster is great. But at what cost? > Since Ruby 3.3.0-preview2 YJIT generates more code than Ruby 3.2.2 YJIT, this can result in YJIT having a higher memory overlead. We put a lot of effort into making metadata more space-efficient, but it still uses more memory than Ruby 3.2.2 YJIT. I'm hoping/assuming the increased memory usage is trivial compared to the cpu-efficiency gains, but it would be nice to see some memory-overhead numbers as part of this analysis.

nomilk 3y ago
This is a particularly valid concern given ruby+rails seems quite memory inefficient to begin with. I've sometimes had smallish apps on 500mb heroku dynos crashing due to memory slowly climbing and eventually slowing things down as the dyno uses swap, and eventually 500mb of swap. IME ruby+rails doesn't seem to free up memory after it uses it, and that causes problems as the hours go by until the pod/dyno crashes or is restarted.
- Ocha 3y ago
  I’ve observed same, and every time I switched to jemalloc and the issue was fixed.
  - nomilk 3y ago
    Was it difficult to switch? What were the downsides / tradeoffs? (I read about jemalloc recently but don't know enough about it to confidently pursue it, but may try it on a small app if it's straight forward).
    - Ocha 3y ago
      Super easy and have not had an issue with it in over 10 years of using it. There is an example here on how to do it with docker image. https://mailsnag.com/blog/optimized-ruby-dockerfile/ https://mailsnag.com/blog/optimized-ruby-dockerfile/.
      - nomilk 3y ago
        Going to try this right now! Will report back. OOC, why isn't this part a ruby default? Isn't it always better to be more memory efficient. (I'm trying to understand what the trade offs are, if any) EDIT: well, exactly 6 minutes later, I'm done. I followed these instructions: https://elements.heroku.com/buildpacks/gaffneyc/heroku-buildpack-jemalloc https://elements.heroku.com/buildpacks/gaffneyc/heroku-build... The app seems to work like usual, I'll just have to wait and see what happens to memory use. I will reply here in 12 hours with a screen shot showing the results (before/after memory use), whatever they may be. Also, for reference, here's the metrics for the past 24 hours (LOTS of memory problems): https://imgur.com/a/M8IHd5z https://imgur.com/a/M8IHd5z
        bmulholland 3y ago
        Yeah, I had the same reaction last week. It's not a Ruby default because some versions of Linux can't use it. ¯\_(ツ)_/¯
        nomilk 3y ago
        For anyone interested, here's the result: https://imgur.com/a/c62gjKQ https://imgur.com/a/c62gjKQ (the red vertical line is the point from which jemalloc was used). It looks like memory usage did indeed go down, and critical errors fell by about 84%.
        nomilk 3y ago
        For completeness, here are the metrics a full 24 hours after the change: https://imgur.com/a/lbdzFvN https://imgur.com/a/lbdzFvN
      - vlovich123 3y ago
        Have you compared it against newer allocators like mimalloc or the rewritten tcmalloc (not the one in gperftools)? Jemalloc is a bit long in the tooth now.
- jacobsenscott 3y ago
  Ruby processes don't return the memory to the system,they reuse memory already allocated. This is for efficiency - allocating and freeing system memory isn't free. Even if it did, your peak memory usage would be the same. It doesn't allocate memory it doesn't need. If your memory usage doesn't plateau you have a memory leak which would be caused by a bug in your code or a dependency. But 500 to 1gb of memory required for a production rails app isn't unusual. Heroku knows this, which explains their bonkers pricing for 2gb of memory. They know where to stick the knife.
  - vlovich123 3y ago
    That kind of thinking is a bit flawed unfortunately. You might hit your peak for 20 minutes a day but you’ve provisioned your system for that temporary worst case for the entire day and other services are paying that penalty. If it’s the only thing you’re running, maybe. But in practice there are other things you want to run on the machine to improve utilization rate (since services are not all hitting their peak simultaneous generally) That’s why good modern allocators like mimalloc and tcmalloc return memory when they notice it’s going unused, so that other services running on the machine can access resources. And this is in c++ land where things are even more perf sensitive.
    - adgjlsfhk1 3y ago
      Theoretically virtual memory and swap solve this problem really well. The OS is free to write the unused pages to disc to let other programs use the real memory.
      - vlovich123 3y ago
        Swap is horribly expensive and most hyperscalars run their servers without swap and set per-process memory limits, automatically killing workloads that go above their threshold..
        adgjlsfhk1 3y ago
        swap is only expensive if you are using the swapped out memory. if you are in a case where a program is just holding on to pages it isn't using, swap is basically free. for most users, turning off swap is just losing performance since the OS can always use all of your RAM to cache disk access.

nomilk 3y ago

This is a particularly valid concern given ruby+rails seems quite memory inefficient to begin with. I've sometimes had smallish apps on 500mb heroku dynos crashing due to memory slowly climbing and eventually slowing things down as the dyno uses swap, and eventually 500mb of swap. IME ruby+rails doesn't seem to free up memory after it uses it, and that causes problems as the hours go by until the pod/dyno crashes or is restarted.

Ocha 3y ago
I’ve observed same, and every time I switched to jemalloc and the issue was fixed.
- nomilk 3y ago
  Was it difficult to switch? What were the downsides / tradeoffs? (I read about jemalloc recently but don't know enough about it to confidently pursue it, but may try it on a small app if it's straight forward).
  - Ocha 3y ago
    Super easy and have not had an issue with it in over 10 years of using it. There is an example here on how to do it with docker image. https://mailsnag.com/blog/optimized-ruby-dockerfile/ https://mailsnag.com/blog/optimized-ruby-dockerfile/.
    - nomilk 3y ago
      Going to try this right now! Will report back. OOC, why isn't this part a ruby default? Isn't it always better to be more memory efficient. (I'm trying to understand what the trade offs are, if any) EDIT: well, exactly 6 minutes later, I'm done. I followed these instructions: https://elements.heroku.com/buildpacks/gaffneyc/heroku-buildpack-jemalloc https://elements.heroku.com/buildpacks/gaffneyc/heroku-build... The app seems to work like usual, I'll just have to wait and see what happens to memory use. I will reply here in 12 hours with a screen shot showing the results (before/after memory use), whatever they may be. Also, for reference, here's the metrics for the past 24 hours (LOTS of memory problems): https://imgur.com/a/M8IHd5z https://imgur.com/a/M8IHd5z
      - bmulholland 3y ago
        Yeah, I had the same reaction last week. It's not a Ruby default because some versions of Linux can't use it. ¯\_(ツ)_/¯
      - nomilk 3y ago
        For anyone interested, here's the result: https://imgur.com/a/c62gjKQ https://imgur.com/a/c62gjKQ (the red vertical line is the point from which jemalloc was used). It looks like memory usage did indeed go down, and critical errors fell by about 84%.
        nomilk 3y ago
        For completeness, here are the metrics a full 24 hours after the change: https://imgur.com/a/lbdzFvN https://imgur.com/a/lbdzFvN
    - vlovich123 3y ago
      Have you compared it against newer allocators like mimalloc or the rewritten tcmalloc (not the one in gperftools)? Jemalloc is a bit long in the tooth now.
jacobsenscott 3y ago
Ruby processes don't return the memory to the system,they reuse memory already allocated. This is for efficiency - allocating and freeing system memory isn't free. Even if it did, your peak memory usage would be the same. It doesn't allocate memory it doesn't need. If your memory usage doesn't plateau you have a memory leak which would be caused by a bug in your code or a dependency. But 500 to 1gb of memory required for a production rails app isn't unusual. Heroku knows this, which explains their bonkers pricing for 2gb of memory. They know where to stick the knife.
- vlovich123 3y ago
  That kind of thinking is a bit flawed unfortunately. You might hit your peak for 20 minutes a day but you’ve provisioned your system for that temporary worst case for the entire day and other services are paying that penalty. If it’s the only thing you’re running, maybe. But in practice there are other things you want to run on the machine to improve utilization rate (since services are not all hitting their peak simultaneous generally) That’s why good modern allocators like mimalloc and tcmalloc return memory when they notice it’s going unused, so that other services running on the machine can access resources. And this is in c++ land where things are even more perf sensitive.
  - adgjlsfhk1 3y ago
    Theoretically virtual memory and swap solve this problem really well. The OS is free to write the unused pages to disc to let other programs use the real memory.
    - vlovich123 3y ago
      Swap is horribly expensive and most hyperscalars run their servers without swap and set per-process memory limits, automatically killing workloads that go above their threshold..
      - adgjlsfhk1 3y ago
        swap is only expensive if you are using the swapped out memory. if you are in a case where a program is just holding on to pages it isn't using, swap is basically free. for most users, turning off swap is just losing performance since the OS can always use all of your RAM to cache disk access.