3 ms·
You can be 1% faster and be 'significantly' faster. 'Significantly' just tells you about variance of your samples. What matters is if it's usefully faster.
by chrisseaton 6y ago
You can be 1% faster and be 'significantly' faster. 'Significantly' just tells you about variance of your samples. What matters is if it's usefully faster.
- creata 6y agoI think they meant "significant" as in "of importance", not "statistically significant".
- jez 6y agoI've been trying to catch myself claiming something is "significantly faster" and instead say it's "substantially faster" for this reason. As far as I know, "substantial" doesn't have the same kind of formal definition, and still captures the informal point I was trying to make. I wish that we (collectively) did a better job of reporting statistically significant performance comparisons, and in that light using "substantially" is a reminder to myself to think about whether it would be hard to go the extra mile and collect the data to actually report significance properly.