3 ms·
Frequentist estimators try to estimate an uncertain quantity, but they don't define what the goal is. Is the goal to get as close to the true value as possible
by jules 16y ago
Frequentist estimators try to estimate an uncertain quantity, but they don't define what the goal is. Is the goal to get as close to the true value as possible on average (i.e. minimize |t-e| where t is the true value and e is the estimator). Is the goal to minimize (t-e)^2? To minimize some other cost function? No, they don't define the goal all they do is define some desirable properties that don't fully specify the problem.
It's like asking a physicist "how do I throw a ball". He can't give a good answer unless you define what your goal is. Is the goal to throw as high as possible? Throw straight up. Is the goal to throw as far as possible? Throw at 45 degrees.
- tel 16y agoGenerally, they seem to begin with (t-e)^2 which they rapidly note the orthogonality decomposition between "bias" and variance. The decomposition appears metaphorically in other error functions as well. This is often mentioned as a supportive point, but I'm not so sure it's meaningful. Finding minimization of MSE difficult, frequentists choose to minimize bias first. The method of the MVUE produces a low MSE estimator from the class of unbiased estimators and thus, automatically, ignores low variance biased estimators which may or may not outperform the MVUE. Frankly, though, the bias-variance decomposition seems too widely interpreted, especially as an expectation over all possible data instead of something considering the problem of inference with a known, fixed set of data. The difference here is thus philosophical and hard to mathematically analyze.