4 ms·
Ask HN: Are your code benchmarks statistically valid?
It appears that some benchmarking tools provide average or percentile values but no information about statistical validity of the result. What did you observe or do in your career?
- Magma7404 1y agoI haven't got a clue since I have never used any benchmark tool in my whole life. Could you tell us in which professional context you would use that kind of tool?
- DenisM 1y agoRun a benchmark to compare performance of my product before and after making a change in code.
- gus_massa 1y agoI run it 5 times and drop the two extremes. If the middle tree don't overlap, I'm probably fine. Not super precise, but good enough and easy. Also, add a loop so the benchmark is like 1 second long, I'm old enough to be worried about anything smaller than 1/18 seconds.