4 ms·
The statistical model is a Bernoulli distribution where the expected value p is parameterized by the logistic sigmoid function applied to a linear model of the
by ogrisel 10y ago
The statistical model is a Bernoulli distribution where the expected value p is parameterized by the logistic sigmoid function applied to a linear model of the input variables.
The choice of the use of the sigmoid function does not stem from applying MLE to the model: instead it is an priori and arbitrary modeling decision that has been done even before starting to think about the estimation of the parameters of the model.
The intuitive justification with respect to the choice of this link function given in this blog post seems quite standard to me. This text-book chapter gives a similar intuition: http://www.stat.cmu.edu/~cshalizi/uADA/12/lectures/ch12.pdf http://www.stat.cmu.edu/~cshalizi/uADA/12/lectures/ch12.pdf (section 12.2).
Also in practice we don't use pure MLE but l2-penalized (or l1-penalized) MLE to fit a logistic regression model as otherwise it might be very sensitive to noisy data if the number of features very large (compared to the number of training samples).
- oddthink 10y agoYep, you're right. I was playing too fast and loose there. The Bernoulli likelihood definitely suggests the odds as a natural space, but it's not required.
- krosaen 10y agoThanks for pointing out the similarity in intuitive justification in Shalizi's text, I have that book on my to-read list already, fun to see I wasn't far off.