3 ms·
If you want to prove that the new process is better on average ("Xnew > Xold") , you would have to use "Xnew <= Xold" as the null. When you determine Xold to be
by yorwba 9y ago
If you want to prove that the new process is better on average ("Xnew > Xold") , you would have to use "Xnew <= Xold" as the null. When you determine Xold to be 1C using a sample size of 100, you also get a variance that you have to take into account instead of naively substituting it into the hypothesis.
If you do the statistical significance test correctly, you won't be able to show the new process better by taking lots of samples for it, since the variance of the old measurements dominates the uncertainty. (I think, haven't done the math to show it.)
If you get wrong conclusions using correct math, your assumptions are wrong. In the case of your example, the incorrect assumption is treating an uncertain measurement as exact when comparing it to another uncertain measurement.
- BeetleB 9y agoI think people are misreading my purpose. Your response comes closest: >If you do the statistical significance test correctly, you won't be able to show the new process better by taking lots of samples for it, since the variance of the old measurements dominates the uncertainty. (I think, haven't done the math to show it.) Higher sample size means lower variance: https://onlinecourses.science.psu.edu/stat414/node/167 https://onlinecourses.science.psu.edu/stat414/node/167
- yorwba 9y ago> Higher sample size means lower variance: That's for a single population. If you are trying to compare two populations, you have two empirical means and two variances. When you do a t-test [1] to compare your measurements, the test statistic has a variance that is approximately computed as variance = variance_1/sample_size_1 + variance_2/sample_size_2 Adding more samples for only one of the populations means that this half of the sum vanishes, but the other half remains unchanged, thus guaranteeing that the variance never drops below a certain threshold. [1] https://en.wikipedia.org/wiki/Student%27s_t-test#Equal_or_unequal_sample_sizes.2C_unequal_variances https://en.wikipedia.org/wiki/Student%27s_t-test#Equal_or_un...