3 ms·
"To know if it's statically significant,it's not important if the benchmark use the arithmetic or the geometric mean. " Of course, but i was answering whether
by DannyBee 11y ago
"To know if it's statically significant,it's not important if the benchmark use the arithmetic or the geometric mean.
"
Of course, but i was answering whether 0.5% matters, not whether it is valid.
This is explicitly why i said "assuming you are accurately measuring the parts".
It seems you are just making a side point for the heck of it, despite completely agreeing with what i said.
- chrisseaton 11y agoI'm not sure what you mean by 'valid' or 'matters' or the difference between them though. I'm asking if it is more likely that the 0.5% difference is due to the change in the implementation than normal variation in the original implementation. I don't know how you can say that this is the case based only on the information that it's a geometric mean. You need to know the variation in the reference implementation, and you don't. The facts that he measured an increase of performance of 0.5% and he used a geometric mean are not enough information to say that it's statistically significant.
- DannyBee 11y agoI understand exactly what you are saying. I'm saying you are being super pedantic here, since it's pretty obvious I meant valid to mean statistically significant, and matters took mean whether improving spec by 0.5% translates into real world application performance benefit. I'm not sure why you want to be so pedantic, but it doesn't seem worthwhile to me since nobody was confused.
- zodiac 11y ago> it doesn't seem worthwhile to me since nobody was confused. fwiw I was confused about what you meant by valid and matters