3 ms·

> but if they didn't do their multiple comparisons correction right (which I am in no position to determine), they're basically reading tea leaves. Please don'

by gaur 11y ago

> but if they didn't do their multiple comparisons correction right (which I am in no position to determine), they're basically reading tea leaves. Please don't spread FUD about other people's research and then try to soften it with a disclaimer about how you're "in no position to determine" anything about the research. If you want to offer criticism of the research, you should do the legwork to find out what the researchers did and did not do.

deleted 11y ago
[deleted]
dr_zoidberg 11y ago
Well said. There are people reviewing the papers, who are in a position to determine if the work is correct, who are enthusiastic about this.
tdaltonc 11y ago
I've though a lot about your comment. I don't think that they're dummies. This is one of the big unsolved problems in statistics and empiricism.[1] There is no general solution to this problem, and the practical solutions to this problem are all very particular to their field. The geneticists have certain ways for controlling for certain known problems; the neuro imagers have certain ways for controlling for certain known problems; same goes for ML, particle physics, economics, ect. I'm personally familiar with a few of these, and I know that they are very particular to their field and that they each have their own history (which is why I'm hesitant to present myself as an expert on statistic empiricism in the context of trans-neptunian planet hunting). And those histories (the ones I'm familiar with) are a parade of over confidence followed by the community establishing new things that need to be controlled for. The red flag in my mind isn't that they are following the established rigor for trans-neptunian planet hunting (I'm sure that they are, because peer-review), or that they aren't following a theoretically valid methodology (I'm sure that they are not, because it doesn't exist). The red flag in my head is that trans-neptunian planet hunting, as a field, looks like the kind of place that there would be a lot of false positives (few historical true-positives, relatively small field, relatively novel methods). I don't doubt this experiment the way a expert in trans-neptunian planet hunting would: "They should control for X, Y, and Z, but didn't!" I don't have the expertise to do that. I'm criticizing it the way a philosopher of science might: "This sort of question in this sort of field is likely to have such-and-such problems and not yet have good ways to dealing with it yet." And I do think that know enough to log that criticism. Love to know what you think of that explanation. [1] https://en.wikipedia.org/wiki/List_of_unsolved_problems_in_statistics https://en.wikipedia.org/wiki/List_of_unsolved_problems_in_s...

by gaur 11y ago

deleted 11y ago

[deleted]

dr_zoidberg 11y ago

Well said. There are people reviewing the papers, who are in a position to determine if the work is correct, who are enthusiastic about this.

tdaltonc 11y ago

I've though a lot about your comment. I don't think that they're dummies. This is one of the big unsolved problems in statistics and empiricism.[1] There is no general solution to this problem, and the practical solutions to this problem are all very particular to their field. The geneticists have certain ways for controlling for certain known problems; the neuro imagers have certain ways for controlling for certain known problems; same goes for ML, particle physics, economics, ect. I'm personally familiar with a few of these, and I know that they are very particular to their field and that they each have their own history (which is why I'm hesitant to present myself as an expert on statistic empiricism in the context of trans-neptunian planet hunting). And those histories (the ones I'm familiar with) are a parade of over confidence followed by the community establishing new things that need to be controlled for. The red flag in my mind isn't that they are following the established rigor for trans-neptunian planet hunting (I'm sure that they are, because peer-review), or that they aren't following a theoretically valid methodology (I'm sure that they are not, because it doesn't exist). The red flag in my head is that trans-neptunian planet hunting, as a field, looks like the kind of place that there would be a lot of false positives (few historical true-positives, relatively small field, relatively novel methods). I don't doubt this experiment the way a expert in trans-neptunian planet hunting would: "They should control for X, Y, and Z, but didn't!" I don't have the expertise to do that. I'm criticizing it the way a philosopher of science might: "This sort of question in this sort of field is likely to have such-and-such problems and not yet have good ways to dealing with it yet." And I do think that know enough to log that criticism. Love to know what you think of that explanation. [1] https://en.wikipedia.org/wiki/List_of_unsolved_problems_in_statistics https://en.wikipedia.org/wiki/List_of_unsolved_problems_in_s...