3 ms·
I've just had a look at the |STAT source, and it computes the variance with double M = Sum/N; /* mean */ double var
by imurray 13y ago
I've just had a look at the |STAT source, and it computes the variance with
double M = Sum/N; /* mean */
double var = (s2 - M*Sum)/(N-1); /* variance */
where s2 is the sum of squares. In most reasonable situations, this approach will work fine. It just doesn't take as wide a variety of inputs as is easily possible to achieve with normal floating point doubles. In fairness, the |STAT terms and conditions state:
|STAT PROGRAMS HAVE NOT BEEN VALIDATED FOR LARGE DATASETS, HIGHLY VARIABLE DATA, NOR VERY LARGE NUMBERS.