5 ms·
Benchmarking Ruby 2.6 to 3.2
- taf2 4y agoWould be nice to see performance stats for running unicorn web app... I am pretty sure Shopify published some stats during the 3.1 and possibly 3.2 preview phase...
- gls2ro 4y agoHere is another benchmark from running Discourse: https://twitter.com/rafael_falco/status/1607395317661458432 https://twitter.com/rafael_falco/status/1607395317661458432
- xfalcox 4y agoHey, that's me! Numbers were generated using https://github.com/discourse/discourse/blob/main/script/bench.rb https://github.com/discourse/discourse/blob/main/script/benc... if anyone wants to try it out too. We already have 3.2 working internally and plan to roll it out to everyone soon.
- stillSlow 4y agoOh look Ruby is still slow.
- gettalong 4y agoYes, it is slower but not that slow, e.g https://hexapdf.gettalong.org/documentation/benchmarks/optimization.html#results https://hexapdf.gettalong.org/documentation/benchmarks/optim...
- brigandish 4y agoIf I'm reading those results right, it seems that Ruby was getting slower between 2.6.10 and 3.1.3 with yjit, and can still outperform Ruby 3+ versions without yjit. Does anyone know why this is?
- gettalong 4y agoYes, there was a bit of a drop off in performance but only when YJIT is not used. With YJIT there are performance gain. Also note that YJIT in 3.1 was still experimental and more proof-of-concept. Only with the 3.2 release was it now deemed production ready. And YJIT's creator, Shopify, is already using it in production with Ruby 3.2 for all their store fronts!
- chucke 4y agoSmall nitpick: Maxime Chevalier-Boisvert is the creator of YJIT and the BBV optimization technique that underpins it. Shopify is the company which employs her and funds its further improvement.
- gettalong 4y agoYes, I know but thought it would be simpler to just say Shopify. Sorry for the inaccuracy.
- faraaz98 4y agoI'd like to know this from a more informed person as well. My guess is that yjit is only now production ready, they've been building it for a while and enabled behind an experimental flag
- maxfurman 4y agoYJIT was disabled by default before 3.2
- alberth 4y agoI’m surprised to see how little speed-up have happened over the years. Especially when you compare it to something like PHP which has massive gains. Seems like most of the Ruby gains are related to memory usage.
- jwcooper 4y agoI think these charts are more informative, especially for Rails apps: https://speed.yjit.org/ https://speed.yjit.org/ Ruby 3.2 and production-ready YJIT feel like a big deal as someone who has been in the Rails and Ruby ecosystem for 15 years. Rails in particular has seemed to be very difficult to optimize for performance in Ruby, and YJIT seems to have done it.
- brasic 4y agoTruffleRuby is leading the pack here: https://eregon.me/blog/2022/01/06/benchmarking-cruby-mjit-yjit-jruby-truffleruby.html https://eregon.me/blog/2022/01/06/benchmarking-cruby-mjit-yj...
- mrinterweb 4y agoEvery time I try to get a project going with truffleruby, I get all kinds of dependency problems. I haven't had the patience to resolve all of the library problems yet, so I give up.
- skrtskrt 4y agoI don’t have a ton of love for Python now that I’ve moved on to static typing but it’s hard for me to see myself choosing Ruby over Python for anything. Python has gotten significantly faster, has static typing built in with an ecosystem of static typecheckers, and async is an ergonomic first class language feature - sure async can have rough edges but the 99.9% case is dead simple with no surprises. Obviously packaging for Python is still a pain but I don’t see Ruby being better
- foxandmouse 4y agoI've only ever known Javascript/ Typescript. I've been thinking of picking up ruby to learn "practical" functional programming and maybe see what the infatuation with rails is, but I'm starting to think it might be more "valuable" to learn python.
- artagnon 4y agoIt's interesting that Ruby now pulls Rust as a dependency, since YJIT is written in Rust [https://github.com/ruby/ruby/tree/master/yjit https://github.com/ruby/ruby/tree/master/yjit].
- jwcooper 4y agoIt seems well designed, as it's an optional dependency as of now. > By default, YJIT does not get compiled and cargo/rustc is not required. If YJIT is built in dev mode, then cargo is used to fetch development dependencies, but when building in release, cargo is not required, only rustc. At the moment YJIT requires Rust 1.60.0 or newer. https://github.com/ruby/ruby/pull/5826 https://github.com/ruby/ruby/pull/5826
- firecall 4y agoI thought Ruby 3 was supposed to be 3x faster than version 2?
- Mikeb85 4y agoIt is for many workloads. Just not Rails.
- joshspankit 4y agoThat would have been almost impossible. Ruby was never designed as a “fast” language, but 3x is a huge number for anything this mature.
- galangalalgol 4y agoLooking at the benchmark game, it seems 3x current performance would put it in the company of racket, which is still noticably slower than node js. While right now it is much closer to python. All benchmarks are misleading but it shows 10x slower than java, lisp, and pascal. Is there a reason that isn't actually representative of idiomatic use?
- WJW 4y agoThe benchmarks in the benchmark game are typically CPU bound, which heavily favors compiled languages for obvious reasons. With regards to what "idiomatic use" is for Ruby, we could argue that most ruby apps are web backends and most of those spend the majority of their waiting for DB and/or API calls and don't spend all that much doing work in actual Ruby.
- galangalalgol 4y agoThe same could be said for python with calls into numpy and pytorch consuming much of the time, but no one calls it designed for speed.
- spleen 4y ago3x increase when 3.0 is compared to 2.0[1]. I think it was largely achieved. [1] https://blog.heroku.com/ruby-3-by-3 https://blog.heroku.com/ruby-3-by-3
- lakomen 4y agoHow many concurrent requests is 3.2. able to handle on an average consumer PC 5 years old, 4 cores? Just plain hello world. Go(lang) about 5-6k.
- joshmn 4y agoHow many of us need 5-6k concurrent requests?
- lakomen 4y agoIdk why the fuck you Ruby people are acting so butthurt
- claudiug 4y agograb ruby, run a test and show us.
- WJW 4y agoI did 65k concurrent requests back in 2018 on a single core in Ruby 2.4/2.5 or so (see https://wjwh.eu/posts/2018-10-29-double-hijack.html https://wjwh.eu/posts/2018-10-29-double-hijack.html) as long as each request doesn't do much. Having 6k open connections is not that much. Did you actually mean request/seq? EDIT: I was curious on how many req/s modern Ruby would actually do for simple hello world, so I coded up the smallest Hello world example from https://github.com/rack/rack https://github.com/rack/rack and ran it with the (default) Puma webserver, limited to 4 threads (and thus cores) with `-t 4`. According to `ab`, it averages around 8500 requests per second over a sample size of 100k requests. I didn't find out how to run rack apps with YJIT yet, but it seems reasonable to expect that to speed it up a bit further.
- samsquire 4y agoI am interested in programming language theory and development but I am a beginner in both. Do JIT compilers suffer from boxing pointer chasing primitives since everything is an object and not a memory location value type? Can hot loops over arrays be efficient if every item in the array is a pointer?
- kaba0 4y agoI am by no means an expert on the topic, but I don’t think that “array of pointers”-chasing is the primary performance bottle neck of most managed language. It happens all too often with C programs as well (or even worse, linked lists) and if the pointed to objects are close to each other in memory/are in cache (as they might be in case of a generational GC) than it is not a big performance hit. Also, most apps don’t spend most of their time in a hot loop iterating over arrays, ordinary programs seldom have huge arrays. Dynamic properties, non-uniform object shapes may be more responsible for this generic slowness of very dynamic languages, and from what I gathered assuming a shape for these objects constitutes a kind of advanced optimization (done by JS engines, truffleruby, not sure about these engines), even if its done depending on language idioms it may not give that big of a boost. Ruby is a very dynamic language with e.g. hitting not implememted methods and dynamically handling, etc.
- samsquire 4y agoThank you for your explanation. My understanding is that determining jump target at compile time is a big source of performance gain at runtime. If you know the function to jump to at compile time and not wait to runtime to decide where to jump. I've looked into arrays of structs and structs or arrays for performance. But I am curious how to create a managed language that is fast. I think we look Java and C# for guidance.
- kaba0 4y ago> If you know the function to jump to at compile time and not wait to runtime to decide where to jump Well, I don’t know about more dynamic languages, but single dispatch can be very well optimized away, Java for example can determine how many possible implementation of a given method is loaded, and simply JIT compile or even inline the function if there is only a single implementation. Later, it can deoptimize that call, e.g. when a new class is loaded at runtime that also implements that method. This is one area where JIT is definitely in a better position than AOT compilers.