3 ms·
I was being sloppy in my description. You are right that they are applying a non-linear function (step-function or logistic) to the output of a linear model, a
by bravura 14y ago
I was being sloppy in my description.
You are right that they are applying a non-linear function (step-function or logistic) to the output of a linear model, and then aggregating. I am not sure if this has equivalent modeling power as a linear model or single logreg model. I suspect it does, but haven't done the math.
I am not aware of results that say that an ensemble of linear or logreg models, trained over partitions of the data, is better than a single linear or logreg model trained over all the data. My understanding was that the opposite was true. Most of the positive results for ensemble methods and bagging are when the base classifier is a decision splits and decision trees, in which case it is clear that the ensemble has more modeling power.
- ot 14y agoI don't see why it should have equivalent modeling power, the set of decision surfaces that it can represent is strictly larger.
- bravura 14y agoputs on theoretician's hat You are correct, it can represent more decision surfaces. takes off theoretician's hat In practice, my intuition tells me that aggregating over an ensemble of logreg models trained on subsets of the data is inferior to training a single logreg model over the entire data. I apologize for being sloppy tho.
- ced 14y agoIt looks like a linear combination of shifted step functions/logistics. That's a complete set - it can represent any function to arbitrary accuracy. I agree that in practice you won't necessarily get better results, though.