4 ms·
There are four benchmarks in there that compare CRuby and TruffleRuby and two of them are slower. TruffleRuby is 2x faster on bench.b (brainfuck implementation)
by kipply 6y ago
There are four benchmarks in there that compare CRuby and TruffleRuby and two of them are slower. TruffleRuby is 2x faster on bench.b (brainfuck implementation), 3x faster on matmul, 3x slower on base64 and 7x slower on json. All four of these are pretty specific benchmarks. Optcarrot has code that is much more representative of what a Ruby program might look like.
- tom_mellior 6y agoIn addition, it's interesting to see that TruffleRuby does better than CRuby on the longer running benchmarks, and worse on the shorter ones. The page claims that JIT warmup "is applied when necessary", but there are no further details. If I had to bet, I would guess that in the case of TruffleRuby maybe not enough warmup is being done in these experiments.
- kipply 6y agoThat sounds likely, but if it's not warmed up in some amount of time that a decent programmer believes to be enough, that probably means that it's warming up too slowly! With that being said, it could also mean that the faster ones could be even faster. I totally expect base64 to perform like this though. It spends like one line in Ruby and then it's shipped off into the C extension, so really it's testing the differences between Sulong (LLVM bitcode interpreter for Graal) and a proper C compiler.
- eregon 6y agoSeems there isn't any warmup accounted for: https://github.com/kostya/benchmarks/issues/246 https://github.com/kostya/benchmarks/issues/246
- headius 6y agoThe optcarrot benchmark is pretty far from being typical Ruby code. It simulates several chips inside the Nintendo entertainment system and essentially interprets the instructions those chips would execute. In actuality, real world benchmarks like web applications perform best on jruby,