The “people who rate a whole slew of movies at one time tend to be rating movies they saw a long time ago” example is wonderful, actually: it indicates exactly

The “people who rate a whole slew of movies at one time tend to be rating movies they saw a long time ago” example is wonderful, actually: it indicates exactly the reason that humans can guide choices of algorithms in a way better than machines can: the data that a human uses to to come up with that hypothesis is quite literally unavailable to the machine. It's completely outside the dataset that's under analysis, and comes from a human's experience dealing with humans, and his assumptions about how they act. Most humans would probably mark that statement as "probably true" without even investigating the data, and that's an extremely valuable prior that a ML algorithm has no access to (unless we explicitly program it in). Sure, you might argue that the hypothesis is implicit in the data set, and (though I'm not familiar with the actual Netflix data, so I'm not sure) that might be true - if it's in there in some form, then it's even conceivable that some algorithm might eventually pick it up. But a human would likely never even dream of advancing that hypothesis without at least some vague sense that other humans would probably act that way, and in many cases, without that high prior probability that comes from our knowledge of psychology it wouldn't be proper to consider that factor. So in a sense, we're cheating every time we use our external domain knowledge to push our ML algos to a better spot in hypothesis space. This doesn't say that generic ML fails; it merely says that "the sum total of human knowledge + ML algo applied to data set" > "ML algo applied to data set", especially when "data set" has something to do with shit that humans know very well, like ourselves.