3 ms·

In (2) it's actually worse than that; any relevant predictor left out of the model affects the betas of all other predictors, even if it's uncorrelated to all o

In (2) it's actually worse than that; any relevant predictor left out of the model affects the betas of all other predictors, even if it's uncorrelated to all other predictors. This is a big thing most people miss when trying to interpret logistic regression the way they do linear regression. Logistic regression estimates are _conditional_ on the model spec in a way linear regression estimates are not.

gbrown 8y ago
I'm not sure I follow your second point. In both multiple linear regression and multiple logistic regression, the coefficients are interpreted conditionally - a one unit increase in X^{(i)} is associated with a B^{(i)} unit change in f(E(Y)), conditional on the values of the other covariates. Using a logit link function does remove the result from the raw scale of Y (thus, f), but in my mind the "model" is more than just the data distribution - it's whatever form your linear predictor takes. This is the problem the poster originally pointed to - interpreting a variable conditional on the fixed values of all others doesn't always make sense when there's strong correlation among predictors.
- zwaps 8y ago
  I still don't get your issue at all. There are tons of different models and implications, and it depends entirely on your question how you interpret the model. Often you will go after average partial effects in your sample. Or you have some correlated variables in mind such that you can just plot the partial effects. Sometimes you have different models, where you will plot a distribution of partial effects around a mean based on some prior assumption... I mean, it really depends. If you impose a more complex relationship, then of course you can not put everything in a single number. But no matter what you are interested in, such a model will give you the possibility to exactly determine the measure. And what you are saying is also not correct. I may very often be interested in fixed values. Doing this regression, and not a quantile regression, for example, means I am somehow interested in a conditional expectation. That means I probably care about some sort of average impact, perhaps for some fixed subgroups. But those averages are fixed values... I think the point is that in these models, we know exactly what we go after, how to get there, and what it means. We know exactly when our inference may fail. If we both care about average effects, and I can convince you of my identification assumptions, then there is really no mystery left as to what my estimates mean. In with Deep Learning, this is still more difficult.

gbrown 8y ago

I'm not sure I follow your second point. In both multiple linear regression and multiple logistic regression, the coefficients are interpreted conditionally - a one unit increase in X^{(i)} is associated with a B^{(i)} unit change in f(E(Y)), conditional on the values of the other covariates. Using a logit link function does remove the result from the raw scale of Y (thus, f), but in my mind the "model" is more than just the data distribution - it's whatever form your linear predictor takes. This is the problem the poster originally pointed to - interpreting a variable conditional on the fixed values of all others doesn't always make sense when there's strong correlation among predictors.

zwaps 8y ago
I still don't get your issue at all. There are tons of different models and implications, and it depends entirely on your question how you interpret the model. Often you will go after average partial effects in your sample. Or you have some correlated variables in mind such that you can just plot the partial effects. Sometimes you have different models, where you will plot a distribution of partial effects around a mean based on some prior assumption... I mean, it really depends. If you impose a more complex relationship, then of course you can not put everything in a single number. But no matter what you are interested in, such a model will give you the possibility to exactly determine the measure. And what you are saying is also not correct. I may very often be interested in fixed values. Doing this regression, and not a quantile regression, for example, means I am somehow interested in a conditional expectation. That means I probably care about some sort of average impact, perhaps for some fixed subgroups. But those averages are fixed values... I think the point is that in these models, we know exactly what we go after, how to get there, and what it means. We know exactly when our inference may fail. If we both care about average effects, and I can convince you of my identification assumptions, then there is really no mystery left as to what my estimates mean. In with Deep Learning, this is still more difficult.