3 ms·
"Bad" is relative. Some tests are not as reflective of real-world workloads as some other tests. But they still test real aspects of VM performance. I don't thi
by om2 12y ago
"Bad" is relative. Some tests are not as reflective of real-world workloads as some other tests. But they still test real aspects of VM performance. I don't think these tests force you to do things that are actively counterproductive, and if you did, the weight of other tests would punish you. The scoring is geometric mean, so it's impossible for a single test to dominate.
Did Robohornet get non-Google browser vendor involvement? It doesn't seem so currently, and I don't recall any particular push to get involvement. It seems like it was single-browser-vendor in practice (and also not very well designed).
I will hedge this by mentioning that there is a totally neutral benchmark that is actually good and more realistic than any of these. JSBench is created from the JS in real websites, was not created by any browser vendor, and has had advice and input from multiple vendors. <http://jsbench.cs.purdue.edu> http://jsbench.cs.purdue.edu> It's not very good at modeling advanced / bleeding-edge webs though. Also, Safari spanks other browsers so hard on this benchmark that it's not a useful optimization tool.
- nnethercote 12y ago> Did Robohornet get non-Google browser vendor involvement? There was a stewardship committee that had lots of non-Google people on it: https://github.com/robohornet/robohornet/wiki/Committee-Membership https://github.com/robohornet/robohornet/wiki/Committee-Memb.... There was even a Mozilla person on it at one point, I can't remember who (someone I didn't know) but they asked to have their name removed once we discovered and explained how bad the benchmarks were -- it turns out that person had only had tangential involvement anyway. But I just found the technical advisory committee, which consisted of three Google Chrome people: https://github.com/robohornet/robohornet/wiki/Technical-Advisors https://github.com/robohornet/robohornet/wiki/Technical-Advi.... So it was more Google-heavy than I realized.