3 ms·

I just ran a comparison of a benchmarking script we use for our own product and saw wildly different results. There's no way I'd consider migrating to truffle t

I just ran a comparison of a benchmarking script we use for our own product and saw wildly different results. There's no way I'd consider migrating to truffle today: # With MRI ruby-2.6.6 Measuring [loading ruby dependencies] 3.560960 2.507366 6.075755 ( 6.154898) Measuring [loading configuration yaml] 0.823779 0.085605 0.909384 ( 0.931334) Measuring [creating objects from YAML and performing filtering] 0.499576 0.042023 0.541599 ( 0.552427) Measuring [object validation] 2.362139 0.226133 2.588272 ( 2.674958) Total time: 7.246786 2.861230 10.115445 ( 10.314025) # With truffleruby-21.3.0 Measuring [loading ruby dependencies] 91.818473 5.598149 97.427109 ( 36.676471) Measuring [loading configuration yaml] 28.032630 0.811577 28.844207 ( 8.616437) Measuring [creating objects from YAML and performing filtering] 14.952781 0.479387 15.432168 ( 4.634803) Measuring [object validation] 86.747057 2.788266 89.535323 ( 28.324909) Total time: 221.595872 9.680172 231.286531 ( 78.264226)

eregon 5y ago
Startup is likely to be worse, because it typically runs a lot of different code for not long and the JIT might not have enough time to optimize much of that. OTOH the JIT needs to learn what the program is doing, i.e., profile it in a sense, and that has a cost on interpreter speed. Still these numbers are worse than expected, especially [object validation], so if there is a way to reproduce it'd be great if you can open an issue about it. Persisting the JITed code is what we think can solve the slower startup entirely: https://www.graalvm.org/graalvm-as-a-platform/language-implementation-framework/AOTOverview/ https://www.graalvm.org/graalvm-as-a-platform/language-imple... Also other things mentioned in https://eregon.me/blog/2022/01/06/benchmarking-cruby-mjit-yjit-jruby-truffleruby.html#warmup https://eregon.me/blog/2022/01/06/benchmarking-cruby-mjit-yj...

eregon 5y ago

Startup is likely to be worse, because it typically runs a lot of different code for not long and the JIT might not have enough time to optimize much of that. OTOH the JIT needs to learn what the program is doing, i.e., profile it in a sense, and that has a cost on interpreter speed. Still these numbers are worse than expected, especially [object validation], so if there is a way to reproduce it'd be great if you can open an issue about it. Persisting the JITed code is what we think can solve the slower startup entirely: https://www.graalvm.org/graalvm-as-a-platform/language-implementation-framework/AOTOverview/ https://www.graalvm.org/graalvm-as-a-platform/language-imple... Also other things mentioned in https://eregon.me/blog/2022/01/06/benchmarking-cruby-mjit-yjit-jruby-truffleruby.html#warmup https://eregon.me/blog/2022/01/06/benchmarking-cruby-mjit-yj...