The conditional mean E(y|x), which exists under quite general conditions, is the best predictor in the conventional sense of MSE. The linear projection, assumi

The conditional mean E(y|x), which exists under quite general conditions, is the best predictor in the conventional sense of MSE. The linear projection, assuming finite second moments and a positive definite expectation of the design matrix, is the best linear predictor in the sense of MSE. Its error is by construction (not by assumption) uncorrelated with the regressands. The linear regression comes in when one additionally assumes that the conditional expectation of the error given the explanatory variables is zero. This then gives unbiased estimates. No distributions required. To characterize the distributions and do testing/intervals, one usually employs large sample theory. This also then shows when the probability limit of the estimator converges to the true value (consistency) and at which speed. The idea of requiring Gaussian noise is, except for edge cases, depreciated. Textbooks that build on this should not be used anymore. It is misleading and it leads to the wide spread misconception that normality of the error term is in some sense essential for linear regression. It's fine to come from the CS angle of algorithms and optimizations, as opposed to theoretical distributions and all that, but semi-parametric statistics in quite useful to understand why and when linear regression can do really well.