3 ms·

This is the problem with frequentist estimators: it's not clear what part is math and what part is subjective guessing.

by jules 16y ago

tel 16y ago
I'm completely certain that the unbiased estimators are a class of estimators that eliminate bias. I'm completely certain that the MVUE is the one of those with the highest Fisher information subject to the Cramer-Rao bound. Those arguments are abstract, but simple. I've just never been too sure why I should prefer unbiasedness to minimizing error (or, more generally, minimizing a cost function of some variety). I've never understood when, theoretically, estimating a 'fixed' parameter is better than pruning it's possible distribution. But, I've also never doubted statements of frequentist statistics made in full rigor. I don't believe they're guessing, I just think they're playing a game that they made up the rules too but got them tragically wrong.
- jules 16y ago
  Frequentist estimators try to estimate an uncertain quantity, but they don't define what the goal is. Is the goal to get as close to the true value as possible on average (i.e. minimize |t-e| where t is the true value and e is the estimator). Is the goal to minimize (t-e)^2? To minimize some other cost function? No, they don't define the goal all they do is define some desirable properties that don't fully specify the problem. It's like asking a physicist "how do I throw a ball". He can't give a good answer unless you define what your goal is. Is the goal to throw as high as possible? Throw straight up. Is the goal to throw as far as possible? Throw at 45 degrees.
  - tel 16y ago
    Generally, they seem to begin with (t-e)^2 which they rapidly note the orthogonality decomposition between "bias" and variance. The decomposition appears metaphorically in other error functions as well. This is often mentioned as a supportive point, but I'm not so sure it's meaningful. Finding minimization of MSE difficult, frequentists choose to minimize bias first. The method of the MVUE produces a low MSE estimator from the class of unbiased estimators and thus, automatically, ignores low variance biased estimators which may or may not outperform the MVUE. Frankly, though, the bias-variance decomposition seems too widely interpreted, especially as an expectation over all possible data instead of something considering the problem of inference with a known, fixed set of data. The difference here is thus philosophical and hard to mathematically analyze.

This is the problem with frequentist estimators: it's not clear what part is math and what part is subjective guessing.

by jules 16y ago

This is the problem with frequentist estimators: it's not clear what part is math and what part is subjective guessing.

tel 16y ago

I'm completely certain that the unbiased estimators are a class of estimators that eliminate bias. I'm completely certain that the MVUE is the one of those with the highest Fisher information subject to the Cramer-Rao bound. Those arguments are abstract, but simple. I've just never been too sure why I should prefer unbiasedness to minimizing error (or, more generally, minimizing a cost function of some variety). I've never understood when, theoretically, estimating a 'fixed' parameter is better than pruning it's possible distribution. But, I've also never doubted statements of frequentist statistics made in full rigor. I don't believe they're guessing, I just think they're playing a game that they made up the rules too but got them tragically wrong.

jules 16y ago
Frequentist estimators try to estimate an uncertain quantity, but they don't define what the goal is. Is the goal to get as close to the true value as possible on average (i.e. minimize |t-e| where t is the true value and e is the estimator). Is the goal to minimize (t-e)^2? To minimize some other cost function? No, they don't define the goal all they do is define some desirable properties that don't fully specify the problem. It's like asking a physicist "how do I throw a ball". He can't give a good answer unless you define what your goal is. Is the goal to throw as high as possible? Throw straight up. Is the goal to throw as far as possible? Throw at 45 degrees.
- tel 16y ago
  Generally, they seem to begin with (t-e)^2 which they rapidly note the orthogonality decomposition between "bias" and variance. The decomposition appears metaphorically in other error functions as well. This is often mentioned as a supportive point, but I'm not so sure it's meaningful. Finding minimization of MSE difficult, frequentists choose to minimize bias first. The method of the MVUE produces a low MSE estimator from the class of unbiased estimators and thus, automatically, ignores low variance biased estimators which may or may not outperform the MVUE. Frankly, though, the bias-variance decomposition seems too widely interpreted, especially as an expectation over all possible data instead of something considering the problem of inference with a known, fixed set of data. The difference here is thus philosophical and hard to mathematically analyze.