4 ms·

So say you have 3 predictors that have high intra predictor correlation. Can you still pick one of them, and discard the remaning 2? Or you cant pick any one of

So say you have 3 predictors that have high intra predictor correlation. Can you still pick one of them, and discard the remaning 2? Or you cant pick any one of them?

cocoablazing 8y ago
You can, but why trash information that is present when you can leverage it with a different approach?
- MichailP 8y ago
  Like PCA? But that way you loose physical meaning of the predictors.
  - closed 8y ago
    PCA is a special case of factor analysis, so you are representing them as observations of a latent variable (which is often a narrative people use when explaining why two x variables are correlated)
beagle3 8y ago
Using ridge regression (mentioned in TFA) would prefer a (1/3,1/3,1/3) average of those predictors (or a better combination, depending on their respective noises). Using lasso (also mentioned in TFA) would prefer to pick the best of the three and drop the others. Using elastic net would be a combination of both. Note, though, that any method other than simple regression has tuning parameters -- depending on those, you could still end with result equivalent to plain least squares.