3 ms·
I'd also be concerned that the benchmark doesn't present a realistic performance profile, either in CPU or GPU terms. It appears to have a low entity count up
by kevingadd 13y ago
I'd also be concerned that the benchmark doesn't present a realistic performance profile, either in CPU or GPU terms.
It appears to have a low entity count up until the end, and not have any considerable amount of collision detection or other cpu-intensive entity behaviors. This means it's not really stressing a JS runtime and could probably perform great under an interpreter (as demonstrated by Ejecta getting a good framerate)
It also doesn't appear to produce a taxing GPU profile even on a canvas implementation. It draws very few objects, and most of them share the same texture and rendering state, ensuring that even a simple rendering implementation can batch them up into a single draw call with very few vertices. The framebuffer resolution is low and the overdraw is low as well.
A real-world game, even one that isn't 'too complicated' for mobile, would have a dramatically different performance profile.
I think this is a solid benchmark, but it's irresponsible to make claims like 'if sbperftest runs well, most other HTML5 games should run decently as well' and 'The next most common case is usually someone designing or porting such a complex game that even a native engine would struggle on the limited, battery-powered mobile hardware.'
You simply can't make those claims without a broader benchmark suite (which I should note most of the professional mobile/desktop benchmarking suites come closer to offering, with a handful of different benchmarks based on different workloads).
- AshleysBrain 13y agoAuthor here. Perhaps the "if sbperftest runs well, most other HTML5 games should run decently as well" statement should have been inverted, so it reads "if sbperftest doesn't run well, few other HTML5 games will". We develop a major HTML5 game engine, and this statement is backed by quite a bit of testing and profiling of real-world games. The test is deliberately based on a real-world game (it was adapted from an interactive game) so it's not a synthetic benchmark. No it is not enormously onerous for the CPU or GPU, but if you look at the results many mobile devices still can't quite manage a rock-solid 60 FPS, so I think it serves pretty well to see if mobile devices are fast enough for relatively simple games. There is actually a lot of collision testing in the laser-to-enemy intersection test - all of the objects in the game have custom collision polygons, it's not a simple bounding box test. The explosions use additive blending (or "lighter" composite mode) which actually seems to stress many canvas2d implementations but generally is a lot smoother with WebGL. So it's not that simple a test. I'm not sure why the 60 second period is important; the most intensive part is the waves of green sprites which should come well within the measured period, and it gets quieter after that when the test ends, so it's not skipping the hard bit.