11 ms·
Ruby Garbage Collection: Still Not Ready for Production
- noelwelsh 13y agoWhy not use the JVM (e.g. JRuby)? These problems have long been solved (if they ever occurred to start with.)
- entresi 13y agoBecause the threading system in JRuby is not very friendly, it is a trade-off worth considering though
- rubiquity 13y agoI think a lot of Rubyists unfortunately don't write thread safe code :( I've worked on many Rails apps that have some mutable state on the ApplicationController and the models. Also, developing on the JVM requires a change in tooling that a lot of Rubyists aren't used to, as well as the slow JVM start up times which can make TDD with JRuby a pain. JRuby is consistently getting better, though.
- mbillie1 13y ago> mutable state on the ApplicationController and the models Well, mutable state on the models seems pretty fair, unless I'm misunderstanding you. It's rather unforgivable on ApplicationController though. Edit: and the slow startup time doing TDD with JRuby was my primary complaint when I developed in it. Although iirc you can use nailgun or something to keep the JVM active and mitigate that pain.
- rubiquity 13y agoSorry, I meant mutable state on classes, which is just a sneaky way of having global state.
- OutThisLife 13y agoCan you give an example? I'm learning Rails and I'd like to avoid bad practices.
- Twirrim 13y agoIs the JRuby environment sufficiently different that you can't TDD against MRI and then save JRuby for final validation testing and deployment? They shouldn't be that different, unless you're not writing pure ruby code. Alternatively, Nailgun should reduce a lot of that pain. Effectively you have a running server instance of JVM and run some clients on it. That gives you JVM speed advantages without the slow startup. JRuby natively supports it: https://github.com/jruby/jruby/wiki/JRubyWithNailgun https://github.com/jruby/jruby/wiki/JRubyWithNailgun
- r4um 13y agoExactly. Jruby warbler rocks, create easily deployable artifacts jars, wars etc and you have rock solid JVM at your disposal.
- deleted 13y ago[deleted]
- gdeglin 13y agohttp://www.isrubyfastyet.com/ http://www.isrubyfastyet.com/ seems to show that Rails+JRuby performance remains disappointingly bad. The memory usage is kind of nuts too, not sure why it's so high. Edit: This page explains the high memory usage and it seems more reasonable now: https://github.com/jruby/jruby/wiki/Troubleshooting-Memory-Use https://github.com/jruby/jruby/wiki/Troubleshooting-Memory-U...
- RyanZAG 13y agoI think this would be a better benchmark than that one as it runs a more apples-apples comparison: http://www.techempower.com/benchmarks/#section=data-r8&hw=i7&test=json&l=35s http://www.techempower.com/benchmarks/#section=data-r8&hw=i7...
- teacup50 13y agoThese benchmarks are uselessly bad: - The client connections are run without concurrency, serially, on the same machine: https://github.com/brianhempel/isrubyfastyet/blob/master/runner/benchmarks/rails/requests_per_second_benchmark.rb#L16 https://github.com/brianhempel/isrubyfastyet/blob/master/run... - The JIT is only given 3 seconds to warm up: https://github.com/brianhempel/isrubyfastyet/blob/master/runner/benchmarks/rails/requests_per_second_benchmark.rb#L12 https://github.com/brianhempel/isrubyfastyet/blob/master/run... -- HotSpot kicks ass, but not after only 3 seconds of serially issued requests. - By running the tests single-threaded, they're throwing away one of the major wins of the JVM -- low overhead concurrency.
- joevandyk 13y agoSlow startup time for large ruby applications.
- Legion 13y ago> you are using Unicorn, right? You should be! No. http://puma.io/ http://puma.io/
- derwiki 13y agoAre these numbers still accurate? http://ylan.segal-family.com/blog/2013/05/20/unicorn-vs-puma-redux/ http://ylan.segal-family.com/blog/2013/05/20/unicorn-vs-puma... At any rate, Unicorn would still be better than the default Heroku server.
- fomb 13y agoThere is no 'default' Heroku server on Cedar, it runs what your app asks to run. If that's Thin you'll be running on Thin. However, Heroku recommend Unicorn.
- rubiquity 13y agoI can't imagine why a company that charges a ton for RAM would recommend a preforking application server ;) Also, when using vCPUs versus using real CPUs there's a noticeable drop off in thread performance so I doubt Puma performs all that great on Heroku :(
- thinkbohemian 13y agoI recommend Puma and work for Heroku, though threadafety is still a huge issue. It also pre-forks in hybrid mode which makes it super awesome. Also it takes care of slow clients!
- midas007 13y agoIt's also way, way cheaper in terms of ram. We fire up sidekiq and clockwork in threads within the puma process rather than take ~ 100 MiB/process hit of running them separately.
- rubiquity 13y ago> (you are using Unicorn, right? You should be!) No, you should be using Puma.[0] Unicorn is a work of art and it is incredibly simple, but with that comes a lot of waste. Puma won't fix this problem as it lies in MRI, but you will be able to run way less processes so total memory consumption won't be such an issue. Ruby 2.1 comes with an asterisk. It's a lot faster but you should take some time to tune the GC to your application's needs. Aman Gupta[1] has some excellent posts on his blog about how to do this. On Rails apps that I have upgraded from 2.0 to 2.1 we have seen around 25% (and up to as high as 50% in some places) decrease in response times. The GC oddities will all certainly get better in Ruby 2.2 (and maybe even in minor releases of 2.1 but I doubt it). 0 - http://puma.io http://puma.io 1 - http://tmm1.net http://tmm1.net
- nnq 13y agoWhy not Passenger?
- rubiquity 13y agoI think Passenger only offers multi-threading in the enterprise edition.
- FooBarWidget 13y agoYes. But support for Aman Gupta's improved out-of-band garbage collector, is in the open source version. The Enterprise version also happens to support out-of-band garbage collection even for multithreaded apps, which as far as I know is not supported by Puma (yet?)
- midas007 13y agoNew passenger is a great btw, what mongrel 2 wanted to be.
- midas007 13y agoPuma offers both multi-threading and multi-process in clustered mode to take advantage of multiple cores. Here's our config/puma.rb that we used with Rails 4.1beta and 2.1.1: https://gist.github.com/9838767 https://gist.github.com/9838767
- csfrancis 13y agoRuby 2.1.1 introduces a new environment variable, RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR, that helps to mitigate the heap growth caused by the generational garbage collector added in 2.1. By setting this variable to a value lower than the default of 2 (we are using the suggested value of 1.3) you can indirectly force the garbage collector to perform more major GCs, which reduces heap growth. See https://bugs.ruby-lang.org/issues/9607 https://bugs.ruby-lang.org/issues/9607 for more information.
- gnufied 13y agoThis needs to be upvoted more. We went through similar pains when upgrading rather big app to Ruby 2.1.1 and setting RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR value to 1.3 did help reduce memory usage of Unicorn workes by almost 100MB. I am not saying this is actual fix and I would like to hear from OP if he tried this environment variable and if it had any impact. TL;DR New generational garbage collector of Ruby has some issues and ruby-core is working to fix it.
- jamesharker 13y agoWe're using this & found it works well at slowing down heap growth however we still end up with the same maximum memory usage as before, it just takes longer to get there.
- timr 13y agoYes, exactly. You're changing the slope of the growth, but the growth still happens.
- x0x0 13y agostuff like this is why people use the jvm; at this point, it's pretty bulletproof
- Fasebook 13y agoGotta love those reflection api attacks and other remote procedure injections. Also, don't forget that everything is an object.
- uaygsfdbzf 13y agocould you expand on that?
- mavroprovato 13y agoCould you provide an example of those attacks/injections?
- Fasebook 13y agoRead this backwards: http://en.wikipedia.org/wiki/Java_EE_version_history http://en.wikipedia.org/wiki/Java_EE_version_history of course, Jvm is insecure by design, that's the whole point of a VM, really, to run anything no matter the context, so they're always be the same ways to escape the sandbox: http://seclists.org/fulldisclosure/2013/Jul/172 http://seclists.org/fulldisclosure/2013/Jul/172 The only thing keeping it secure is legions of devs and an "application server" propagating trust.
- berdario 13y agoHow can a JRuby (or Clojure, or Jython...) REST api trigger a vulnerability in the Java reflection? I was always under the impression that such kind of vulnerabilities were only a problem for Java applets (where the sandbox is actually important) Has any exploit ever been built for such circumstances? (I admit that I'm ignorant of their existance)
- mavroprovato 13y ago
- purephase 13y agoAnother vote for puma. We're using Unicorn in our production environment right now with the unicorn-worker-killer gem, but our initial tests with ruby 2.1 and puma in dev/QA are going well so we're looking to move to that setup in the near future.
- FooBarWidget 13y agoPhusion Passenger author here. The article mentions that Unicorn's out-of-band garbage collection is problematic because the way it works - running the GC after every request, and requiring turning off the normal GC - is overkill. But there the community is working on a better solution. In particular, Aman Gupta described this very same problem and created a gem which improves out-of-band garbage collector, by only running it when it is actually necessary, and by not requiring one to turn off the normal GC. Phusion Passenger (even the open source version) already integrates with this improved out-of-band garbage collector through a configuration option. This all is described here: http://blog.phusion.nl/2014/01/31/phusion-passenger-now-supports-the-new-ruby-2-1-out-of-band-gc/ http://blog.phusion.nl/2014/01/31/phusion-passenger-now-supp... Just one caveat: it's still a work in progress. There's currently 1 known bug open that needs reviewing. Besides the GC stuff, Phusion Passenger also has a very nice feature called passenger_max_requests. It allows you to automatically restart a process after it has processed the given number of requests, thereby lowering its peak memory usage. As far as I know, Unicorn and Puma don't support this (at least not out of the box; whether there are third party tools for this, I don't know). And yes, this feature is in open source.
- boundlessdreamz 13y agoWhat's the known bug?
- benesch 13y agoThe project appears to be called gctools: https://github.com/tmm1/gctools https://github.com/tmm1/gctools And the known bug is probably "Allow dry-running the OOBGC": https://github.com/tmm1/gctools/pull/5 https://github.com/tmm1/gctools/pull/5 Without knowing anything about the project, it seems like the bug is particular to Passenger's use case.
- sams99 13y agoThe bug, is not a "bug" its a feature missing to allow running with passenger. It works fine with unicorn.
- dclowd9901 13y agoUsing ARC with Objective C has been an amazing experience, and I come from the brainless world of Javascript GC. Any reason why more environments don't adopt this approach? It seems entirely efficient, reasonable and well-designed...
- muaddirac 13y agoI would suspect that it's because a ruby ARC would require thinking more about references (the strong/weak paradigm would, I think, need to come into play) which isn't really a concept in Ruby. I think it would be difficult to ensure code written without arc would still run properly. But I agree with you. I much prefer ARC over any GC I've come into contact with.
- ksec 13y agoWell RubyMotion is using ARC.
- dunham 13y agoPython does reference counting. The downside is that you can leak memory if you have circular references. The programmer needs to take care to not let this happen. (This is essentially the cause of the Javascript memory leaks in IE, although the reference counting only occurred at the boundary between Javascript objects and browser objects.) I believe python handles this by occasionally running a garbage collector that looks for leaked objects. Objective C offers weak pointers (pointers that "don't count" and zero themselves out when all the other pointers to the object disappear), which can aid a programmer in avoiding circular references. Finally, I believe it has been shown that reference counting can be slower that a well-implemented garbage collector, but I suspect this depends on your specific workload.
- berdario 13y agoPython also has weakrefs to avoid creating cycles http://docs.python.org/3.4/library/weakref.html http://docs.python.org/3.4/library/weakref.html
- 13y ago
- rubyn00bie 13y agoOh Unicorn... a little GC tuning goes a long way, and isn't that hard... For most apps the time spent in GC, while annoying, is far and away not the bottleneck. ... especially since Ruby 2.0 as the GC is much, much faster. I'll be honest, I don't think the memory bloat is that problematic if you design the app well (business logic definitely prevents this some time)... but in general you can pass most of that off to a background worker, or pre-cache responses so you aren't bloating your app server instances/threads.
- znowi 13y agoWhenever I read that "you should be using X", credibility of the author rapidly diminishes for me.
- deleted 13y ago[deleted]
- iancarroll 13y agoHa, so Unicorn did have a bug in it. I thought it was me.
- jrochkind1 13y agoSeems like a reason to stick to ruby 2.0 and not upgrade to 2.1, no?
- sams99 13y agoThis article enrages me. I totally agree with codinghorror that a blog without comments in not a blog, this is a prime example. No way to respond to the author without jumping through crazy hoops. As to the issue. 1. NEVER use unicorn oobgc that ships with unicorn or the old one that ships with passenger. Use gctools on 2.1.1 https://github.com/tmm1/gctools https://github.com/tmm1/gctools or this on 2.0 http://samsaffron.com/archive/2013/11/22/demystifying-the-ruby-gc http://samsaffron.com/archive/2013/11/22/demystifying-the-ru.... If you are disabling GC you are doing it wrong and creating rogue processes. 2. Expect memory doubling with Ruby 2.1.1. Not happy with that? You have 2 options. Tune it down by reducing RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR. At 1 your memory consumption will be on par with 2.0. It ships with the 2 default. Option 2, wait for a future release of Ruby, this will be fixed in 2.2 maybe even patched a bit more in 2.1.2 . See: https://bugs.ruby-lang.org/issues/9607 https://bugs.ruby-lang.org/issues/9607 and http://vimeo.com/89491942 http://vimeo.com/89491942 and https://speakerdeck.com/samsaffron/why-ruby-2-dot-1-excites-me https://speakerdeck.com/samsaffron/why-ruby-2-dot-1-excites-... As for the, full-of-bait, title. Ruby's GC is ready for production and being used in production in plenty of places. Just don't go expecting rainbows if you disable it. And expect memory doubling with 2.1.1 if left untuned. You can choose how much you want the memory to increase from 2.0
- timr 13y agoOK, I'll bite. First, sorry we don't have comments yet. I'm the author, so you no longer have to be enraged. Perhaps just annoyed. Second, with Ruby 2.1, out of the box, this code: while true do "a" * (1024 ** 2) end leads to infinite process growth. There should be no need for "memory doubling" to run this code -- you're generating throwaway strings of identical size. Similar code, in other languages, does not lead to out of control memory consumption. Also, the problem isn't caused by disabling the GC. This happens in stock Ruby 2.1. Disabling the GC (and running it once per request) is the fix for the problem.
- vidarh 13y ago> There should be no need for "memory doubling" to run this code -- you're generating throwaway strings of identical size. How do you know they are throwaway? Your code involves two method calls. While it would be unlikely that someone has overridden them given that they are core String and Fixnum methods, it is perfectly possible. E.g: class String; alias :old :* def * right $store ||= [] $store << self old(right) end end "foo" * 5 "bar" * 3 p $store In other words, even with seemingly innocent calls like that, it takes extra work to be able to reuse that memory without a full GC pass. It's certainly not impossible, but it's not there yet. So I agree with you in principle, but this is one of those areas where the malleability of Ruby objects makes it tricky to optimize. (one possible example approach is to set aside a bit to indicate "has at least once been stored somewhere where it can escape" as "poor mans escape analysis" - if your object is only ever stored in local variables that have no been captured by a lambda, or passed as arguments, then it can't escape higher than where it was created, and so you can take shortcuts, otherwise you'd still need a full gc pass)
- jafaku 13y agoSeems like Rubysts spent too much time bashing PHP and very little time improving their language, and now they have been left behind.
- sayrer 13y agoDoesn't Ruby need to break compatibility with its C API to get a good result here? To get a copying, generational GC as in Java, it would need to stop handing out raw pointers.
- ptx 13y agoCouldn't this problem be largely solved the same way C# does it, with its GC.AddMemoryPressure function? If you're using lots of unmanaged memory, you simply inform the GC when you're allocating and freeing it, so that in can be taken into account. http://msdn.microsoft.com/en-us/library/system.gc.addmemorypressure(v=vs.110).aspx http://msdn.microsoft.com/en-us/library/system.gc.addmemoryp...