4 ms·

Running the test on the latest firefox nightly results in a time of ~3 seconds, so if microsoft IS cheating by doing some browser sniffing they might want to wi

by Trezoid 14y ago

Running the test on the latest firefox nightly results in a time of ~3 seconds, so if microsoft IS cheating by doing some browser sniffing they might want to widen the net somewhat, or end up looking bad (in 18 weeks...) The other option, and still a little dodgy, is that they found operations that ran poorly in V8, optimized the hell out of them in chakra specifically to say "Look, we're faster then chrome!" while just letting the fact that spidermonkey is faster slide because mozilla is way less of a threat then google.

kevingadd 14y ago
Everyone optimizes for their own benchmarks. You can say it's dodgy, but that's really just an argument in favor of real-world tests instead of microbenchmarks like this one. You don't have to 'rig' a microbenchmark to come out the winner; you just pick the right microbenchmark, because it's not a real world scenario. Given the lack of evidence to support actual cheating you should probably just apply occam's razor here and assume it's actually faster. Or you could use the dark, hidden secrets of 'view source' to read the JS yourself. It has comments!
- Trezoid 14y ago
  making a benchmark specifically to be slower on your competitors browser? Little dodgy. From everything I've read IE10 apparently is fast, and the numbers are believable from my experience doing similar things (web workers drawing fractal-like images on a canvas) but given that it's the only "pretty" benchmark in the lot, it just seems a little strange.
  - kevingadd 14y ago
    I think you misread my post. My point is that everyone optimizes their implementation so that their benchmarks run fast. Furthermore, people pick benchmarks that run fast on their implementation. Neither of these are a 'little dodgy': They are how benchmarks get used. You might note that they make for bad science, but benchmarks tend not to get used in anything resembling a scientific comparison. Benchmarks get used for comparative testing (did we make thing X faster than it was before?) and to try and prove that your product is better than the competition (in which case much larger lies are going to get trotted out either way).

by Trezoid 14y ago

kevingadd 14y ago

Everyone optimizes for their own benchmarks. You can say it's dodgy, but that's really just an argument in favor of real-world tests instead of microbenchmarks like this one. You don't have to 'rig' a microbenchmark to come out the winner; you just pick the right microbenchmark, because it's not a real world scenario. Given the lack of evidence to support actual cheating you should probably just apply occam's razor here and assume it's actually faster. Or you could use the dark, hidden secrets of 'view source' to read the JS yourself. It has comments!

Trezoid 14y ago
making a benchmark specifically to be slower on your competitors browser? Little dodgy. From everything I've read IE10 apparently is fast, and the numbers are believable from my experience doing similar things (web workers drawing fractal-like images on a canvas) but given that it's the only "pretty" benchmark in the lot, it just seems a little strange.
- kevingadd 14y ago
  I think you misread my post. My point is that everyone optimizes their implementation so that their benchmarks run fast. Furthermore, people pick benchmarks that run fast on their implementation. Neither of these are a 'little dodgy': They are how benchmarks get used. You might note that they make for bad science, but benchmarks tend not to get used in anything resembling a scientific comparison. Benchmarks get used for comparative testing (did we make thing X faster than it was before?) and to try and prove that your product is better than the competition (in which case much larger lies are going to get trotted out either way).