The main difference between classical regression using ordinary least squares (OLS) and robust regression using iterative re-weighted least squares (IRLS) is th

The main difference between classical regression using ordinary least squares (OLS) and robust regression using iterative re-weighted least squares (IRLS) is that with OLS, all observations are given equal weight and with IRLS, observations may or may not be given equal weight. Essentially, IRLS gives outliers and/or influential [1] data points less weight, which may improve the performance of the overall model since these outliers/influential data would otherwise cause assumption violations using classical regression. If there are no outliers, then results from robust and classical regression converge. I would disagree with SagelyGuru in recommending robust regression for non-statisticians, though I can see where he or she is coming from. With robust regression, you don't have to worry as much about assumptions as with classical regression. But with robust regression, you need to be aware that the underlying analytical method is different and what that means. For example, the standard robust regression implementation in R (i.e, the rlm function in the MASS package) doesn't produce t-statistics or p-values. There're also warnings that especially at lower sample sizes, the standard errors produced by rlm may be unreliable. One recommended way to obtain those p-values would be to get bootstrapped standard error estimates, so that normal-theory approximation would apply. [1] There are different types of robust estimators (e.g., M, S, MM, etc.) that have different robustness properties.