> This is a good article, but I think it leaves out an important option, which is "do nothing." I don't see how "do nothing" is a good option. If the co-linear

> This is a good article, but I think it leaves out an important option, which is "do nothing." I don't see how "do nothing" is a good option. If the co-linearity between two variables is high enough, then the linear regression equation will not have a solution at all. But this is an edge case. The more common problem with co-linearity is that it leads to unstable regression parameters. For example, all variables in a dataset could have very high, positive correlation, but one of the regression parameters could come out as a negative number due to co-linearity (which would counter-intuitive). If you are basing decisions on the regression parameters then it could lead to some very silly decisions. Moreover, if you resample the data and re-fit the regression model, you could get very different regression parameters (i.e. the parameters will be unstable). Stability of the parameters over time (or over different training sets) is one of the most important properties of a good model. To see why "do nothing" is not a logical option, you can imagine that your data points are plotted within a 3-dimensional cube (1 response variable, and 2 predictor variables). If the data is co-linear, then your data points will lie along a straight line that cuts across the diagonal of the cube. If you "do nothing," then it is directly equivalent to trying to fit a PLANE through a LINE in 3 dimensions. The problem is that points along a LINE do not uniquely identify a plane. The slope along one axis of the plane will be stable, but the slope along the other axis will be fitted arbitrarily by the regression model and will be unstable. The proper thing to do here is to model the line in 2 dimensions. In order to do this, you can either eliminate one of the two predictor variables from the model, or you can project the line in 3-dimensions onto a 2-dimensional subspace (which is what PCA does). Of course, a line formed by data points in 2 dimensions (x and y) is very nicely modeled by ordinary linear regression and it will give very a stable slope (Beta) parameter.