4 ms·
Wow! I wonder though how they account for over fitting of the data. Is it a real solution or a statistical anomaly? I ask because it seems that the progress ove
by SapphireSun 17y ago
Wow! I wonder though how they account for over fitting of the data. Is it a real solution or a statistical anomaly? I ask because it seems that the progress over the last year(s) has been small increments within the 9-10% range.
- Eliezer 17y agoI'm also worried about this. I believe that Netflix has a separate data set, not used in the previous reports of mean squared error, which validates the $ prize. I also believe that the teams have been use the Netflix-reported squared errors from standard test data to combine their estimates. If so, in a month we're going to learn that the Prize has not actually been won yet.
- paraschopra 17y agoBut I wonder whether the effect of an unknown dataset slowly seeps into the algorithms as they are tweaked, especially when 100s of algorithms are competing to fit that data. It might be akin to the fact that if you correlate more than 25-30 sets of 25-30 random numbers, you would find at least one statistifcally significant correlation.
- bravura 17y agoThey are not granted access to the data set, so it would be statistically unlikely to achieve 10% RMSE improvement on the test set solely by chance.