4 ms·
30% faster is significantly faster
by guipsp 6y ago
30% faster is significantly faster
- eikenberry 6y agoIt is significantly faster, just not enough to make a difference when the baseline is so slow.
- mhh__ 6y agoDepends, 30% faster codegen is a good thing to build from but codegen is only a fraction of the actual compilation process.
- chrisseaton 6y agoYou can be 1% faster and be 'significantly' faster. 'Significantly' just tells you about variance of your samples. What matters is if it's usefully faster.
- creata 6y agoI think they meant "significant" as in "of importance", not "statistically significant".
- jez 6y agoI've been trying to catch myself claiming something is "significantly faster" and instead say it's "substantially faster" for this reason. As far as I know, "substantial" doesn't have the same kind of formal definition, and still captures the informal point I was trying to make. I wish that we (collectively) did a better job of reporting statistically significant performance comparisons, and in that light using "substantially" is a reminder to myself to think about whether it would be hard to go the extra mile and collect the data to actually report significance properly.