3 ms·
Always willing to learn more. I was going off of https://en.wikipedia.org/wiki/AdaBoost https://en.wikipedia.org/wiki/AdaBoost which mentions "Unlike neural net
by SubiculumCode 9y ago
Always willing to learn more. I was going off of https://en.wikipedia.org/wiki/AdaBoost https://en.wikipedia.org/wiki/AdaBoost
which mentions "Unlike neural networks and SVMs, the AdaBoost training process selects only those features known to improve the predictive power of the model, reducing dimensionality and potentially improving execution time as irrelevant features need not be computed."
edit: When I read about ensemble theory, you receive support for the increased computational cost.
Evaluating the prediction of an ensemble typically requires more computation than evaluating the prediction of a single model, so ensembles may be thought of as a way to compensate for poor learning algorithms by performing a lot of extra computation. Fast algorithms such as decision trees are commonly used in ensemble methods (for example Random Forest), although slower algorithms can benefit from ensemble techniques as well.
But I found this interesting:
Empirically, ensembles tend to yield better results when there is a significant diversity among the models.[4][5] Many ensemble methods, therefore, seek to promote diversity among the models they combine.[6][7] Although perhaps non-intuitive, more random algorithms (like random decision trees) can be used to produce a stronger ensemble than very deliberate algorithms (like entropy-reducing decision trees).[8] Using a variety of strong learning algorithms, however, has been shown to be more effective than using techniques that attempt to dumb-down the models in order to promote diversity.[9]
..which under my analogy would seem to suggest that exact replication is less productive than having a diversity of study designs.
- AstralStorm 9y agoNot quite. If you read the latter paragraph closely, you would note that an ensemble of stronger models performs better than a multitude of dumb models. Then there is a problem that an ensemble of biased classifiers (mostly in the same ditection in this case - positivity bias) will magnify the bias. This is a reason why metaanalyses have to get at the actual raw data to pool and analyse as well as correct for multiple biases. Even then the process is not perfect.