4 ms·

What if you simply add department and department*salary interaction terms to a standard regression model?

wodenokoto 6y ago
Then for any given data point, salary for all other departments will be zero (dep1salery = dep2salary = ... = 0) so data for each department will become separate and you will in effect have i number of different models, although they all share the same bias term / intersect.
- plouffy 6y ago
  You could argue that a hierarchical model is already multiple models. If each department has a dummy variable then they wouldn't share the same intersect. The only difference is how you interpret that intercept. The department intercept would now not be the salary of an individual with 0 experience, but the difference between the salary of an individual with 0 experience in a department compared to the all other employees with 0 experience.
  - wodenokoto 6y ago
    Well, two things here: You wouldn't actually add salary as an interaction, as it is salary you are trying to predict. You'd use an interaction between experience and department to predict salary, as you correctly point out. But usually when doing interactions, you don't apply it to the bias term, so you will get this: salary = b0 + b1*d1*exp + b2*d2*exp ... And this is essentially a model for each department, but all models share the bias term, b0. If you also interact the bias term with the dummy variable for department, you will in effect have completely independent models, salary = b0,1*d1 + b1*d1*exp + b0,2*d2 * b2*d2*exp ... But I don't think that is what is normally done when applying interactions between features. Anyway, I think we both agree, once we get our terminology aligned.
aseerdbnarng 6y ago
You'll run out of data pretty quickly. https://statmodeling.stat.columbia.edu/2018/03/15/need-16-times-sample-size-estimate-interaction-estimate-main-effect/ https://statmodeling.stat.columbia.edu/2018/03/15/need-16-ti...
- alisonkisk 6y ago
  Hierarchial model has the same problem, since it splits up the data into categories.
alisonkisk 6y ago
I think it gives you the same answer without having the intuitive picture of your kernel that people like for toy problems.

Then for any given data point, salary for all other departments will be zero (dep1salery = dep2salary = ... = 0) so data for each department will become separate and you will in effect have i number of different models, although they all share the same bias term / intersect.

plouffy 6y ago
You could argue that a hierarchical model is already multiple models. If each department has a dummy variable then they wouldn't share the same intersect. The only difference is how you interpret that intercept. The department intercept would now not be the salary of an individual with 0 experience, but the difference between the salary of an individual with 0 experience in a department compared to the all other employees with 0 experience.
- wodenokoto 6y ago
  Well, two things here: You wouldn't actually add salary as an interaction, as it is salary you are trying to predict. You'd use an interaction between experience and department to predict salary, as you correctly point out. But usually when doing interactions, you don't apply it to the bias term, so you will get this: salary = b0 + b1*d1*exp + b2*d2*exp ... And this is essentially a model for each department, but all models share the bias term, b0. If you also interact the bias term with the dummy variable for department, you will in effect have completely independent models, salary = b0,1*d1 + b1*d1*exp + b0,2*d2 * b2*d2*exp ... But I don't think that is what is normally done when applying interactions between features. Anyway, I think we both agree, once we get our terminology aligned.

aseerdbnarng 6y ago

You'll run out of data pretty quickly. https://statmodeling.stat.columbia.edu/2018/03/15/need-16-times-sample-size-estimate-interaction-estimate-main-effect/ https://statmodeling.stat.columbia.edu/2018/03/15/need-16-ti...

alisonkisk 6y ago
Hierarchial model has the same problem, since it splits up the data into categories.

alisonkisk 6y ago

I think it gives you the same answer without having the intuitive picture of your kernel that people like for toy problems.