3 ms·

Anyone know why blending seems to work so well?

• mattj 17y ago
  It's a standard ml technique that's been shown to work well, although no ones shown a proof of this. This is related to the observation "more data beats better models." This does seem to have it's limits - at some point you do need better features before you can expect your data to pay off.
• gaika 17y ago
  Only because each individual algorithm is slightly overfit. Blending it all together keeps the good predictive part and reduces the overfit noise.
• hundredwatt 17y ago
  It seems like just a consequence of basic statistics (central limit theorem?). Any algorithm you create has some random bias. By blending, you take several algorithms with random bias and average them together, reducing the overall bias (assuming the algorithms converge toward the correct result).
  • eru 17y ago
    That's as far from the central limit theorem as you can go..
• lliiffee 17y ago
  I believe it is a consequence of A) netflix evaluating the algorithms in terms of the squared error and B) different classification algorithms making different types of mistakes. Take a simple example, with two movies (A and B), and two algorithms (X and Y). Say Alg. X rates movie A perfectly, but is off by 2 for movie B. Meanwhile, Alg. Y rates movie B perfectly, but is off by 2 for movie A. The squared errors for the different algorithms will be: Alg X: 4 (=0^2 + 2^2) Alg Y: 4 (=2^2 + 0^2) Mean of Alg X and Alg Y: 2 (=1^2 + 1^2)