11 ms·
Ruby Benchmark
- jackowayed 16y agoThis is awesome. It would be really cool if they added comments so that people in the know could explain things (and possibly dispute the outcomes)
- sandrot 16y agoThanks. Comments would be cool, I'll give it a shot.
- Groxx 16y agoA handy list, I'll have to read through it more completely some time. There are some surprising speed differences. For instance, array.drop is hella-faster than array.clear, and array.shift(n) is faster yet (what?). Nitpick on their array tests, however: measure "Array#delete_if" do a = ARRAY.dup a.delete_if {|o| o} end Ah, no. If o == false, that won't delete it. Also, you're calling object == true; how about a.delete_if {true}? "Array#delete_if{true}" is up to 30% faster over 10000 repetitions ------------------------------------------------------------------ Array#delete_if{true} 1.42472314834595 secs Fastest Array#delete_if{|o| o} 2.03995013237 secs 30% Slower Also: >gsubbing with string literals vs regex. For some reason regexp is faster. I suspect it converts string literals to a regexp anyway. Really? You're surprised regex literals are faster than regex strings? The gem however is interesting and nice and simple, I may end up using it for my future testing. Haven't looked through the code, however: anyone know if it's making any mistakes?
- rbxbx 16y agoYou should use http://github.com/sandro/bench_press http://github.com/sandro/bench_press and share your findings :)
- Groxx 16y agochanged the example and sent a pull request. Thanks for pointing that out :)
- chc 16y agoIf I'm reading right, he's not surprised regez literals are faster than regex strings. He's surprised that regex search-and-replace is faster than a literal string search-and-replace.
- Groxx 16y agoAaaah, that's a good point. That's quite a bit more rational.
- wzdd 16y agoI'm not a Ruby programmer, so I'm interested to hear your view on this: as a non-Rubyist reading Ruby code, the "assign variable to a new array" method is much more readable to me than than the .shift(n) method and is only 2% slower. Is shift(length-of-array) an idiomatic way to clear an array in Ruby? From my perspective the site looks like a great way to encourage confusing code for little gain.
- rbxbx 16y agoIn some cases the gain is more dramatic, and in other cases, that 2% gain may be worth it. If you're simply learning a language I don't believe looking at benchmarks is a good way to do so. Typically the most performant options are hacky at best (in most all languages), imo.
- Groxx 16y agoNo, it's just a benchmark testing many ways to clear an array; there's nothing idiomatic about shift(n) that I'm aware of. Benchmarking code should always be taken with a grain of salt, it can get pretty needlessly esoteric at times in the pursuit of speed. I would be interested in if anyone knows how the garbage collector handles a.drop vs a = []. Maybe a.drop collects earlier, and the gc is causing the slowdown?
- cheald 16y agoGC should be disabled for proper benchmarks anyhow, honestly. Or at least measured. REE has handy switches to measure that sort of thing.
- sandrot 16y agoI wouldn't be surprised if the code was making mistakes - it hasn't had much peer review. I think the percentage calculation may be wrong? We can always fix it if it's broken and resubmit the benchmarks.
- davidw 16y agoIt does convert string literals to regexps, which is kind of disappointing. I once did some benchmarking with Tcl, and there, glob patterns were significantly faster than regexps, let along string literals.
- zackadams 16y agoNice work Sandro. Andy Stone has a benchmark file he's been maintaining that has some similarity to your project. Check it out if your looking for inspiration on additional tests: http://github.com/stonean/benchmark/blob/master/benchmarks.rb http://github.com/stonean/benchmark/blob/master/benchmarks.r...
- sandrot 16y agoGreat resource. Looks like I've got a lot of porting to do =). Thanks.
- xsltuser2010 16y agoThe site is awfully slow at the moment. Is it written in Ruby ? That would make the benchmark even more convincing ;)
- jacquesm 16y agoFunny, just finished installing rails 3 and the latest ruby. What a headache that was, really. In spite of following a tutorial I ran in to a whole series of showstoppers and only just now finally got it to work. I've never played with ruby/rails before so I figured I'd better go and see what all the noise is all about but I had definitely not counted on so many installation issues. I should probably clean this machine and do it all over again to be able to write up the exact steps to get it to work (from not having ruby before) but I'm too scared of having to go through another day of fiddling.
- rbxbx 16y agoI'm not sure the relevance to the OG post, but I'll grant you that Rails 3 is less than stable, thusly it's tagged RC2, ie: not a release proper.
- jacquesm 16y agoYou're right, it's not relevant, I just thought it was coincidence to see a rails post at the top of HN while I'm taking a break from my troubles. It looks like there is light at the end of the tunnel though.
- rbxbx 16y agoIf you're entirely new to the ecosystem, I would not be running Ruby 1.9.x or Rails 3. It may seem counter-productive to learn 'legacy' software, but these version will be around for a while I'm sure, you'll not hit nearly the snags you currently are, and you won't have /that/ much to learn when the later version are ready for you (and you them). also, ruby != rails ;)
- derefr 16y agoIf you're quite familiar with Ruby/Rails, but you left the ecosystem for a while (before 1.9 existed), would it be better to re-acquaint yourself with 1.8/Rails 2 first, or just dive into the new stuff?
- justinchen 16y agoHow about an RSS feed for the recent benchmarks?
- sandrot 16y agoTop of my list!
- viraptor 16y agoIt doesn't seem to throw out / properly mark insignificant results: http://rubybenchmark.com/reports/10 http://rubybenchmark.com/reports/10 (is there any reason these 2 variants could get different bytecode results?) That's the way echo -vs- print wars start...
- sandrot 16y agoLooks like there can be a more significant difference after 100,000 repetitions: http://rubybenchmark.com/reports/13 http://rubybenchmark.com/reports/13. In general, the program is pretty dumb and my math isn't that great. Accepting patches: http://github.com/sandro/bench_press http://github.com/sandro/bench_press
- ulvund 16y agoIt might be fun to add some Date tests as well. The methods are very slow to say the least. There is a 'date/performance' gem I think.
- sandrot 16y agoCool, I'll keep that in mind. The site is intended to have user-generated content so if you find the exact thing you'd like to benchmark, use the bench_press gem to test locally, then publish it to share your findings.
- pieter 16y ago> "while" is up to 92% faster over 100000 repetitions Array#each_with_index 1.3768820762634277 secs 92% Slower while 0.10703921318054199 secs Fastest Don't you mean it's around 13x fast (or 1200% faster)?
- Chirono 16y agoThis is always rather ambiguous terminology. For some people, it makes no sense for thing a to be more than 100% faster than thing b. Other people see it clearly meaning that thing a take half the time of thing b. What we really want to measure is time quantities rather than speed quantities. I find comparisons of time much easier to grasp than comparisons of speed.
- rue 16y agoI am, uh, highly sceptical of the value of this. That is, even ignoring the variance between runs of the same code, differences between implementations and within an implementation with slightly different objects. Writing code based on "what is fastest" is an overall awful strategy.
- AndrewO 16y agoThat's a great idea. I'd really like to see a "raw" format option on the server that would give the code and metadata in a structured format. With that one could write a tool that suggests improvements to code as it tests it using this as a sort of crowd-sourced benchmark suite. It may not be 100% accurate, but at least it would give verified data instead of the educated-guess strategy most of us employ now.
- Pistos2 16y agoAlthough it usually only matters when compared code scores similarly, it is important to be able to know when benchmark comparisons are statistically significant[1]. I wrote a library[2] a while back to do this. This becomes even more of a concern when the benchmarked code is running on a machine which is not well-isolated and dedicated to running the benchmark. [1]: http://en.wikipedia.org/wiki/Statistical_significance http://en.wikipedia.org/wiki/Statistical_significance [2]: http://github.com/Pistos/better-benchmark http://github.com/Pistos/better-benchmark