I've though a lot about your comment. I don't think that they're dummies. This is one of the big unsolved problems in statistics and empiricism.[1] There is no

I've though a lot about your comment. I don't think that they're dummies. This is one of the big unsolved problems in statistics and empiricism.[1] There is no general solution to this problem, and the practical solutions to this problem are all very particular to their field. The geneticists have certain ways for controlling for certain known problems; the neuro imagers have certain ways for controlling for certain known problems; same goes for ML, particle physics, economics, ect. I'm personally familiar with a few of these, and I know that they are very particular to their field and that they each have their own history (which is why I'm hesitant to present myself as an expert on statistic empiricism in the context of trans-neptunian planet hunting). And those histories (the ones I'm familiar with) are a parade of over confidence followed by the community establishing new things that need to be controlled for. The red flag in my mind isn't that they are following the established rigor for trans-neptunian planet hunting (I'm sure that they are, because peer-review), or that they aren't following a theoretically valid methodology (I'm sure that they are not, because it doesn't exist). The red flag in my head is that trans-neptunian planet hunting, as a field, looks like the kind of place that there would be a lot of false positives (few historical true-positives, relatively small field, relatively novel methods). I don't doubt this experiment the way a expert in trans-neptunian planet hunting would: "They should control for X, Y, and Z, but didn't!" I don't have the expertise to do that. I'm criticizing it the way a philosopher of science might: "This sort of question in this sort of field is likely to have such-and-such problems and not yet have good ways to dealing with it yet." And I do think that know enough to log that criticism. Love to know what you think of that explanation. [1] https://en.wikipedia.org/wiki/List_of_unsolved_problems_in_statistics https://en.wikipedia.org/wiki/List_of_unsolved_problems_in_s...