4 ms·

I have a hard time believing this, as a linear combination of features doesn’t add any information for linear models. c(f527 - f528) = cf527 - cf528 which is a

I have a hard time believing this, as a linear combination of features doesn’t add any information for linear models. c(f527 - f528) = cf527 - cf528 which is a constrained version of a linear model af527 + b*f528. If they saw their std err of the feature shoot way down thats very different than improving model performance. For explanations, key is having the input features be as intuitive as possible. So if a subtraction of features is more intuitive than the two feature separately then the explanations will be more intuitive as a whole

rm999 8y ago
Not to disagree because you're totally correct, but the winning model used GBMs (a non-linear model) and a bunch of subtracted correlated features to great success. When using GBMs, subtracting two features can add a lot of value. https://storage.googleapis.com/kaggle-forum-message-attachments/41007/1148/Josef_Feigl_Documentation.pdf https://storage.googleapis.com/kaggle-forum-message-attachme...