4 ms·

Not sure how comparing different versions of rails shows if its 'fast' yet. I'd think setting up some request target and seeing how far RoR is from that target

by RyanZAG 13y ago

Not sure how comparing different versions of rails shows if its 'fast' yet. I'd think setting up some request target and seeing how far RoR is from that target with each version would be better. Or comparing RoR to other platforms is also a good bet, eg http://www.techempower.com/benchmarks/ http://www.techempower.com/benchmarks/ I just don't really see how this answers the question other than 'ehh I think it looks a bit faster than it was awhile ago?' EDIT: Also interestingly, JRuby seems to be slightly better on the small instance EC2 test at techempower than regular Ruby (small instance should be similar to a laptop?). Yet in these graphs it's much slower. Anybody know the reason?

jarrett 13y ago
Amazon says their "m1.small" instance has 1.7 GiB of memory. That's a lot less than most current laptops. Maybe JRuby uses more memory in total, but can handle more concurrent requests per MB of memory used? That could explain the discrepancy if it's true, though it's just speculation on my part. Another theory is that the benchmarks are too application-specific. Rails is a complex framework with which you can build many different kinds of applications, each with its own performance characteristics. Conceivably, application A could perform better on JRuby, while application B could perform better on MRI. Depending on which application you use for your benchmark, either interpreter could appear faster. This calls into question the utility of benchmarking an entire framework. It might be better to do micro-benchmarks, with each one testing a specific scenario. Scenarios could include, for example: A request that loads 1,000 database records into memory; a request for a static image; a request that queries a many-to-many relationship; a request that saves an uploaded file to disk.
- lukeschlather 13y ago
  The thing is, those are all I/O bound operations and in the case of the databases most of the code you're benchmarking is native extensions, so I would expect the difference to be negligible. Testing whole Rails apps really does seem like the best way to get good data. The interesting speedups are likely going to be in places like string matching / string copying and other places that don't involve network access.
bhauer 13y ago
>EDIT: Also interestingly, JRuby seems to be slightly better on the small instance EC2 test at techempower than regular Ruby (small instance should be similar to a laptop?). Yet in these graphs it's much slower. Anybody know the reason? Thanks to some community experts, we're now using TorqBox (TorqueBox 3, built on Undertow [1]) for our JRuby tests. The JRuby community seems satisfied [2]. :) [1] http://torquebox.org/news/2013/12/04/torquebox-next-generation/ http://torquebox.org/news/2013/12/04/torquebox-next-generati... [2] https://twitter.com/headius/status/413021218728443904 https://twitter.com/headius/status/413021218728443904
- RyanZAG 13y ago
  Ah, that explains it. Pretty massive difference really. I guess that makes Torquebox+JRuby the best way to run ruby code? I wonder why the RoR community generally ignores it? Maybe it's just a timing thing and as more companies learn about Torquebox they will swap?
  - sluukkonen 13y ago
    I'd wager that it's the TB/Undertow code running most of the time on that benchmark. On real-world benchmarks the difference is going to be a lot smaller. Not saying that you shouldn't run your JRuby apps on Torquebox - it's a great choice.
    - bhauer 13y ago
      Indeed. In our tests, the plain Rack implementation is extremely impressive, but that is probably because there is so little Ruby code being run [1]. By contrast, the Rails implementation on JRuby is just a bit better than plain Ruby, but that is an improvement versus our previous rounds that were not using TorqBox. Previously, we had been using JRuby deployed on Resin, and the results put that JRuby+Rails combination slightly slower than plain Ruby+Rails. For what it's worth, Resin is fast as far as Java Servlet containers go. Since TorqBox/TorqueBox 3 is very new, I don't suspect it's being used much yet. [1] https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/rack/config.ru https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...

by RyanZAG 13y ago

jarrett 13y ago

Amazon says their "m1.small" instance has 1.7 GiB of memory. That's a lot less than most current laptops. Maybe JRuby uses more memory in total, but can handle more concurrent requests per MB of memory used? That could explain the discrepancy if it's true, though it's just speculation on my part. Another theory is that the benchmarks are too application-specific. Rails is a complex framework with which you can build many different kinds of applications, each with its own performance characteristics. Conceivably, application A could perform better on JRuby, while application B could perform better on MRI. Depending on which application you use for your benchmark, either interpreter could appear faster. This calls into question the utility of benchmarking an entire framework. It might be better to do micro-benchmarks, with each one testing a specific scenario. Scenarios could include, for example: A request that loads 1,000 database records into memory; a request for a static image; a request that queries a many-to-many relationship; a request that saves an uploaded file to disk.

lukeschlather 13y ago
The thing is, those are all I/O bound operations and in the case of the databases most of the code you're benchmarking is native extensions, so I would expect the difference to be negligible. Testing whole Rails apps really does seem like the best way to get good data. The interesting speedups are likely going to be in places like string matching / string copying and other places that don't involve network access.

bhauer 13y ago

>EDIT: Also interestingly, JRuby seems to be slightly better on the small instance EC2 test at techempower than regular Ruby (small instance should be similar to a laptop?). Yet in these graphs it's much slower. Anybody know the reason? Thanks to some community experts, we're now using TorqBox (TorqueBox 3, built on Undertow [1]) for our JRuby tests. The JRuby community seems satisfied [2]. :) [1] http://torquebox.org/news/2013/12/04/torquebox-next-generation/ http://torquebox.org/news/2013/12/04/torquebox-next-generati... [2] https://twitter.com/headius/status/413021218728443904 https://twitter.com/headius/status/413021218728443904

RyanZAG 13y ago
Ah, that explains it. Pretty massive difference really. I guess that makes Torquebox+JRuby the best way to run ruby code? I wonder why the RoR community generally ignores it? Maybe it's just a timing thing and as more companies learn about Torquebox they will swap?
- sluukkonen 13y ago
  I'd wager that it's the TB/Undertow code running most of the time on that benchmark. On real-world benchmarks the difference is going to be a lot smaller. Not saying that you shouldn't run your JRuby apps on Torquebox - it's a great choice.
  - bhauer 13y ago
    Indeed. In our tests, the plain Rack implementation is extremely impressive, but that is probably because there is so little Ruby code being run [1]. By contrast, the Rails implementation on JRuby is just a bit better than plain Ruby, but that is an improvement versus our previous rounds that were not using TorqBox. Previously, we had been using JRuby deployed on Resin, and the results put that JRuby+Rails combination slightly slower than plain Ruby+Rails. For what it's worth, Resin is fast as far as Java Servlet containers go. Since TorqBox/TorqueBox 3 is very new, I don't suspect it's being used much yet. [1] https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/rack/config.ru https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...