3 ms·
The fact that the 5x outlier may be qualitatively worse is something that you pin down using percentiles, which is what everyone does, either 1% or 0.1% low. Th
by sudosysgen 3y ago
The fact that the 5x outlier may be qualitatively worse is something that you pin down using percentiles, which is what everyone does, either 1% or 0.1% low. The argument about time-weighted frame staleness is not mathematically valid.
- Retric 3y agoThere’s nothing inherently mathematically special about 1% or 0.1%, you can also use 0.0042% or 0.00069% which highlights this kind of extreme outliers more. Benchmarking sites use 1% or 0.1% because they better highlight the commodity PC hardware rather than the games bugs and architecture.
- sudosysgen 3y agoThat's not true. You can't use 0.0042% because then you don't have enough frames to be statistically significant without running for a very long time. It's also true that a frame pacing issue that is so rare that it isn't even impacting the 0.1% frame time isn't going to be noticeable, really. These benchmarks are controlled for game bugs and architecture by simply averaging over ~40-50 games, on repeatable loops.