What you are saying is reasonable but it is not what the author proposed. This is your rephrasing: > The point being made isn't that large effects are impossib

What you are saying is reasonable but it is not what the author proposed. This is your rephrasing: > The point being made isn't that large effects are impossible, or that high-d results should be dismissed outright. But what the author actually says in the concluding paragraph is that certain results are to be dismissed outright and moreover the burden of proposing a plausible mechanism is wholly on the experimenter: > Implausibility is not a reason to completely dismiss empirical findings, but impossibility is. It is up to authors to interpret the effect size in their study, and to show the mechanism through which an effect that is impossibly large, becomes plausible. Without such an explanation, the finding should simply be dismissed. He does not say, "other experimenters should attempt to reproduce the result" or "theoreticians should explore the phenomena mathematically" but only "the finding should simply be dismissed." Dismissed! Note the stark contrast: while you emphasize that such results should not be "dismissed outright," that is exactly what the author calls for! No follow up studies, no novel theoretical models proposed, just blanket dismissal; furthermore, in the authors own words, the test used to divide those results which are dismissed is "implausible" vs. "impossible." But who can so finely demarcate the implausible from the impossible? Certainly not the contemporary scientists, who in every age we find railing against newly discovered theories and inconvenient facts. And why would the experimenter who first notices a phenomenon also be required to at once to provide a satisfying theoretical explanation? The role of the experimenter is careful lab work. Was the mechanism behind penicillin understood immediately? Was the quantum physics of the depletion zone fully understood when the strange electrical properties of doped silicon were first observed? When half of the neutrinos from the sun go missing, the experimenter must simply attend to his instruments to ensure he has not made an error, and then throw up his hands and let the theoreticians get to the bottom of the thing. This is not the only way science can proceed, but it is an important way: surprising empirical observations, outside the realm of current theory or even directly in contradiction to it are announced decades, or centuries, or in a few cases (such as ancient supernovas) millennia before they are fully understood. https://en.wikipedia.org/wiki/Solar_neutrino_problem https://en.wikipedia.org/wiki/Solar_neutrino_problem https://en.wikipedia.org/wiki/History_of_supernova_observation https://en.wikipedia.org/wiki/History_of_supernova_observati... If the author had been content with making a specific criticism of this one study, I would not have objected. The study is almost certainly flawed, and various good reasons why it is flawed have been discussed. But the author seeks to elevates this principle to that of a general maxim (as the concluding paragraph makes clear), recommending it to one and all as as a valid and useful form of inference. But it is not (in the general case) a valid "form" of inference, as we see by swapping out the details of this one flawed psychological studies for several others, such as penicillin, the field effect, or neutrinos. In those cases, arguments of the exact same form would have resulted in the dismissal of legitimate data. It is often not a good idea to reject experimental results just because they seem "impossible." Concretely, we may analyze the authors proposed method as a three step process: 1. Before the experiment, decide what is possible and what is impossible. (We fix a hypothesis space. Anything outside of this space is given zero a priori probability.) 2. We conduct the experiment and observe the outcome. 3. If the outcome of the experiment is impossible, we "dismiss" the result. (We update no posterior probabilities.) It is difficult to see how science could have progressed if such a rule were universally applied. For example, Rutherford was absolutely shocked when a certain percentage of alpha particles were reflected from a thin sheet of gold foil back at an angle near 180 degrees: > "It was quite the most incredible event that has ever happened to me in my life. It was almost as incredible as if you fired a 15-inch shell at a piece of tissue paper and it came back and hit you." Yet, because he did not follow the author's advice and dismiss these incredible observations, the nucleus was discovered. Kuhn argues that when this situation occurs in the history of science, what actually happens is that the old theories are discarded and new ones are proposed and validated until even the "impossible" observations are now explained. (A paradigm shift.) I would suggest, and I think Kuhn would agree, that in step #3 we should not discard the observation, but rather discard the theory used in step #1 that led us to conclude that it was "impossible." https://en.wikipedia.org/wiki/Paradigm_shift https://en.wikipedia.org/wiki/Paradigm_shift