I'm kind of worried that they checked so many types of things (64) on small outlier samples (top 300): > We ran eight different queries and sorted the question

I'm kind of worried that they checked so many types of things (64) on small outlier samples (top 300): > We ran eight different queries and sorted the questions according to: [8 criteria]. Then we compared the top 300 questions in each section to 300 questions that received an average score on the parameter we focused on. Would results of that be statistically significant, if you picked the apparently-best 5 resulting from that process, or would they just be noise? (Isn't ten thousand questions not even two days' worth of questions?)