5 ms·
> If you're running (say) 10 Rails processes on a machine with 10 GB RAM, and particular request paths/background jobs whatever pushes the process size to 2 GB
by FooBarWidget 5y ago
> If you're running (say) 10 Rails processes on a machine with 10 GB RAM, and particular request paths/background jobs whatever pushes the process size to 2 GB RAM size, you can release the memory back to the OS after you're done, but you still have the underlying problem which is the process gets to that size in the first place, and if all the processes hit 2GB at once, now you're 10 GB in debt, either in swap, or OOM errors.
Fullstaq Ruby author here. I have done research on the memory bloat problem[1], and this statement is not true. With the way memory allocation in glibc's ptmalloc2 works, multithreaded processes can reach bloaty proportions even if they don't do much work. I started investigating this whole issue because I had a trivial Ruby HTTP proxy server: it had no reason to use 1.3 GB of memory, it doesn't do anything besides forwarding small requests. That's when I found that ~1 GB of that 1.3 GB of memory wasn't even due to anything that my app did by itself, but it's just how the memory allocator works.
[1] https://www.joyfulbikeshedding.com/blog/2019-03-14-what-causes-ruby-memory-bloat.html https://www.joyfulbikeshedding.com/blog/2019-03-14-what-caus...
- wlll 5y agoDid you check to see what happened when you used jemalloc?
- FooBarWidget 5y agoThe bloat went away and memory usage became much more "normal".
- wlll 5y agoThat's what we see in our app, and I used to see all the time in others (I was a performance and scaling consultant for years). It's trivial to use jemalloc on heroku, so you may as well, and after that it's mostly preventing too much loading of AR objects.
- jrochkind1 5y ago> It's trivial to use jemalloc on heroku Can you share the method you are using to do it? Not challenging, actually answering. Googling I'm not sure which thing I'm finding is the trivial one you use/recommend.
- wlll 5y agoSure thing! This is what we use: https://elements.heroku.com/buildpacks/gaffneyc/heroku-buildpack-jemalloc https://elements.heroku.com/buildpacks/gaffneyc/heroku-build...