The problem is that mathematically when you partition your data into sub-set and analyse those subsets you are no longer analysing the original data set. Your

The problem is that mathematically when you partition your data into sub-set and analyse those subsets you are no longer analysing the original data set. Your data is now composed of the original set of data [n1, n2, n3... nn] + a NEW DATA SET [s1, s2, s3 ... sn] which is your selection criteria. This is often masked by the fact that your selection data set is a function f(x) ("we'll just split them up by week and partition by gender"), so it doesn't look like your actually combining two dataset, but you are. This new set of data S, biases the result of the analysis, and the more complex it is, the closer you are to over-fitting to find the 'golden correlation' you're looking for in your data. Humans are pattern matching machines. We see patterns in random noise, and hear voices in random tones. It's easy for us to see patterns (or what we think are patterns) in data sets, but its much harder to statistically substantiate those patterns as distinct from randomness. It's provable, I imagine, that any data pattern can be found in a data set if we have sufficiently complex selection criteria for the samples, and sufficiently random raw data. dont do this If you do, at the very least acknowledge that you have selectively modified the original dataset and specifically test your partitioning rules. Its much, much better to search the dataset for a target pattern, and then investigate the partitioning rules that your search matches against (because you can examine the partitioning rules for complexity, and data-content and easily spot over-fitting).