3 ms·

Thank you so much! I agree that the figures can vary for many reasons and we shouldn't expect them to be exactly the same (some things we don't end up controll

Thank you so much! I agree that the figures can vary for many reasons and we shouldn't expect them to be exactly the same (some things we don't end up controlling for). At the same time, if we take the experiment of the soup for example: They measured 9,227 sales of it so the 21.1% increase is quite robust and I'd expect the error margin to be much lower than 5% either way - so in some ways the precision is warranted. I also feel that if I were to round a 21.4% to 20% I'd be miscommunicating the findings of the research :)

shoto_io 5y ago
Thanks for explaining! Your point about miscommunication makes a lot sense to me!
lupire 5y ago
You are confusing confidence intervals (used to say that you are confident the increase is positive at all) with error margin (the false precision in the 3rd significant figure).
- vntok 5y ago
  That does not bode well for the rest of the methodology...
  - johnsmith4739 5y ago
    I don't know why people downvote you: //> we finally have an absolute number of sales measured, but no way of knowing - representing all sales within a period or just cherry picked? //> for the rest of the population, did it reduced sales? //> was there any randomised test or not, because in the latter case there could be other biases we're unaware of //> the increase is compared to what exactly? //> any WHY is purely speculative as what was measured was WHAT people did. Internal motivation is in this case unproven, there is just a potential correlation. //> confuses me to hell people taking about error margins and confidence intervals for something measured directly.
  - boomboomsubban 5y ago
    Mixing up terms in casual conversation happens to everyone, and is not a sign of a meaningful problem.
- tdmckinlay 5y ago
  You are right, my bad! Thanks for catching that in the comment above, I've updated it
rahimnathwani 5y ago
"if I were to round a 21.4% to 20%" Right, but you'd round 21.4% to 21%, not to 20%.
- pushrax 5y ago
  Or ideally keep 21.4% and include the 95% CI. There’s nothing misleading about the extra precision when the CI is included.
  - yowlingcat 5y ago
    Agreed. Actually, I wish the norm was to communicate percentages with confidence intervals by default, because I feel like the tacit implication is colloquially "100% CI unless communicated otherwise."
- infogulch 5y ago
  You're assuming that you can only round to the nearest 1% which is not true at all. If you round to the nearest confidence interval and the CI is 5% then you would round to 20%. That said, I would prefer both the precise number and CI communicated together like sibling comment mentions.

shoto_io 5y ago

Thanks for explaining! Your point about miscommunication makes a lot sense to me!

lupire 5y ago

You are confusing confidence intervals (used to say that you are confident the increase is positive at all) with error margin (the false precision in the 3rd significant figure).

vntok 5y ago
That does not bode well for the rest of the methodology...
- johnsmith4739 5y ago
  I don't know why people downvote you: //> we finally have an absolute number of sales measured, but no way of knowing - representing all sales within a period or just cherry picked? //> for the rest of the population, did it reduced sales? //> was there any randomised test or not, because in the latter case there could be other biases we're unaware of //> the increase is compared to what exactly? //> any WHY is purely speculative as what was measured was WHAT people did. Internal motivation is in this case unproven, there is just a potential correlation. //> confuses me to hell people taking about error margins and confidence intervals for something measured directly.
- boomboomsubban 5y ago
  Mixing up terms in casual conversation happens to everyone, and is not a sign of a meaningful problem.
tdmckinlay 5y ago
You are right, my bad! Thanks for catching that in the comment above, I've updated it

rahimnathwani 5y ago

"if I were to round a 21.4% to 20%" Right, but you'd round 21.4% to 21%, not to 20%.

pushrax 5y ago
Or ideally keep 21.4% and include the 95% CI. There’s nothing misleading about the extra precision when the CI is included.
- yowlingcat 5y ago
  Agreed. Actually, I wish the norm was to communicate percentages with confidence intervals by default, because I feel like the tacit implication is colloquially "100% CI unless communicated otherwise."
infogulch 5y ago
You're assuming that you can only round to the nearest 1% which is not true at all. If you round to the nearest confidence interval and the CI is 5% then you would round to 20%. That said, I would prefer both the precise number and CI communicated together like sibling comment mentions.