4 ms·
> JIT warmup No kidding, this can be seriously misleading. IMO JS (and likely many other JIT VM) benchmarks should run: - multiple cold starts - multiple pr
by eyelidlessness 5y ago
> JIT warmup
No kidding, this can be seriously misleading.
IMO JS (and likely many other JIT VM) benchmarks should run:
- multiple cold starts
- multiple pre-warmed starts (essentially run your bench twice, discard the results, run it again)
- isolated cases
- shared runtime cases with evenly distributed ordering
All of those will reveal different performance characteristics you may want to know about, and people evaluating your results should be able to know about. Cold starts are a worst case scenario of each implementation, warmed should show a more ideal outcome. Isolated will again highlight the perf of each implementation on its own merits, whereas shared runtime should help highlight how interacting with other real-world code affects VM optimization of that implementation.
- xpl 5y agoYeah, properly benchmarking JS code is hard because of how complex is the V8's optimizing compiler infrastructure. I once made an online tool for measuring run times of JS code, with [hopefully, lol] proper warmup and protection against possible dead code elimination optimization. There are a few clickable benchmark examples in the readme: https://github.com/xpl/what-code-is-faster#what-code-is-faster https://github.com/xpl/what-code-is-faster#what-code-is-fast...
- ignoramous 5y ago> No kidding, this can be seriously misleading. Reminds me of prof. Emery Berger's talk, Performance Matters: https://www.youtube-nocookie.com/embed/r-TLSBdHe1A https://www.youtube-nocookie.com/embed/r-TLSBdHe1A