4 ms·
> Which answer do you give? Whatever your software tells you (e.g., 87.14234%) or a number made of a small and fixed number of significant digits (e.g., 87%).
by CrazyStat 3y ago
> Which answer do you give? Whatever your software tells you (e.g., 87.14234%) or a number made of a small and fixed number of significant digits (e.g., 87%).
> The latter is the right answer in almost all instances.
and
> Most human beings are happy with a 1% error margin.
While he doesn't say it as a rule outright, the author repeatedly uses a "small and fixed" two significant digits. He later says this:
> You must choose the number of significant digits deliberately.
which I agree with, but is at odds with the "small and fixed" dictum with which he leads off the post.
- YetAnotherNick 3y agoFixed doesn't mean 2. If it is statstically and practically significant. It could be 3 or 4.
- CrazyStat 3y agoIf you're adjusting the number of significant figures you report based on what is practically and statistically significant then it's no longer fixed.
- YetAnotherNick 3y agoIt doesn't need to be same for all the task, just fixed for a particular task. The creators of benchmark or the community could come up with digits of significance that should be the standard, which is based on expected error range. For imagenet it could be 4, for ARC it could be 2.