"Vote tallying" is basically taking a mode/weighted mode response rather than a mean/median/trimmed mean/weighted mean response. There are contexts where this i

"Vote tallying" is basically taking a mode/weighted mode response rather than a mean/median/trimmed mean/weighted mean response. There are contexts where this is ideal (for example, classification in multiple unordered classes where mode is the only measure of central tendency that is even reasonable); cases where it's superfluous (in 2-class classification the mode is the median is the sign of the mean); and cases where it's bad (in regression with a continuous outcome where the modal prediction has probability exactly 0). So really it depends on the space where you want to use it. Typically in a regression setting with an ensemble learner you're using a kind of weighted mean, where the weights are selected based on cross-validation performance. This is sometimes called a "super learner". See van der Laan, Polley, and Hubbard 2007. Note that this suffers from being similarly awful in terms of theory as a lot of other ML stuff. It is not, for example, the case that two apparently similar datasets or problem domains will produce similar super-learner weights. Which is disturbing, because it's easy to believe that say SVM does better at X and penalized ordered logit at Y, but it's hard to believe that they both do better seemingly at random.