5 ms·
Very interesting seeing people in the comments debate what is a very basic thing taught in any stats/econometrics class. The idea behind binary regression ( Y
by curiousgal 6y ago
Very interesting seeing people in the comments debate what is a very basic thing taught in any stats/econometrics class.
The idea behind binary regression ( Y is 0 or 1) is that you use a latent variable Y* = beta X + epsilon.
X is the matrix of indenpent variables, beta is the vector of coefficients and epsilon is an error term that sums the rest of what X can't explain.
Y thus becomes 1 if Y* >0 and 0 otherwise.
Seeing how Y is binary, we can model it using a Bernoulli distribution with a success probability P(Y=1) = P(Y* >0) = 1 - P(Y* <=0) = 1 - P(epsilon <= -beta X) = 1 - CDF_epsilon(-beta X)
Technically you can use any function that maps R to [0, 1] as a CDF. If its density is symmetrical then you can directly write the above probability as CDF(beta X). The two usual choices are either the normal CDF which gives the Probit model or the Logistic function (Sigmoid) which gives the Logit model. With the CDF known you can calculate the likelihood and use it to estimate the coefficients.
People prefer the Logit model because the coefficients of the model are interpretable in terms of log-odds and all and the fucntion has some nice numerical properties.
That's all there is to it really.
- platz 6y agofyi, generally when explaining things to non-practitioners, it's a detractor to add qualifiers like - this is a very basic thing - that's all there is to it because although it is a basic thing to you, it's not a basic thing to someone who hasn't spent the same time studying all the concepts beforehand. This is generally why you have to take a class to grok stats rather than just read some reference material and definitions. It's similar to programming environments when the senior dev says some thing that requires a lot of context is just really simple. Monads are Simply Just monoids in the category of endofunctors, after all, they are really Simple, that's all there is to it! (what the subject hears: This is so simple to me, why aren't you smart enough to get this simple concept? You should know this already—I shouldn't even have to make this comment! Why haven't you been properly educated, like I have? )
- curiousgal 6y agoI totally agree but I was explaining that to practionners, or at least people who consider themselves data scientists. I agree that it is by no means a beginner friendly explanation.
- kelvin0 6y agoIs that what some people call the 'mansplain' method?
- jiaaro 6y agono. all explanations must make some assumption about what readers already know, or else every explanation would begin with teaching english. also, mansplaining generally refers to assuming people (women, usually) have less pre-existing knowledge than they actually do.
- kelvin0 6y agoThanks for clarifying. So condescending is not necessarily mansplaining. But mansplaining IS condescending.
- ImaCake 6y agoThe author of the parent explains their view well I think (not directed at statistics students). But I would like to make the more general complaint that this attitude seems pervasive in maths. People often fail to include enough detail in their derivations of proofs or their explanations of things and the students are left confused and unsure because they don't see how they got from step 3 to step 4.
- curiousgal 6y agoI wholeheartedly agree with you, in my case I was typing my comment on my phone while waiting on the train. I wanted to get into as many details as possible but not having access to easily typed math notation made my task harder.
- reactspa 6y agoI've always found it a little simplistic that the default cut-off, in most statistical software, for whether something should be 0 or 1, is 0.5. (i.e. > 0.5 equals 1, and < 0.5 equals 0). This seems to be a "rarely-questioned assumption". Is there a reason why this is considered reasonable? And is there a name for the cut-off (i.e., if I were to want to change the cut-off, what keyword should I search for inside the software's manual?)?
- curiousgal 6y agoFrom a stats perspective the cutoff is included in the coefficients. If you use a design matrix (add a column of 1s to your variables) you get in a non matrix notation (beta_01 + beta_1 X_1 +...) So the threshold can be considered beta_0. In the software, you can get classification models to output class probabilities instead of class labels. You can then use whatever threshold you like for to transform those probabilities to labels. You may see it refered to as "discrimination threshold". Varying that threshold is how ROC curves are constructed.
- cuchoi 6y agoThe threshold would be beta_0 on every case or only when you have subtracted the mean from your data?
- vhhn 6y agoYou don't want to demean your dependent (response) binary variable. So you almost always want to keep beta0 to control for any imbalance in your dependent var.
- cuchoi 6y agoI meant demeaning the independent variables. My understanding is that the beta_0 will have the meaning the curiousgal attach it only if you demean your independent variables.
- 6y ago
- avs733 6y agoso I literally teach statistics to undergraduates...a couple thoughts: 1) agree with others comments that stating with 'very basic' ain't going to work 2) logistic regression actually isn't a common topic in an undergraduate statistics course. I cover it on the second to last day as an 'here is where you can extend it' 3) this isn't a very effective explanation. 4) a big background factor is the misconceptions about statistics that are ingrained in human beings. Learning statistical tests is hard, especially for many engineers and similar STEM types because it is typically the first (and sometimes only) class that deals with concepts of error and variance. Everything else presents an absolute theoretical model. On average, it breaks their brain...