4 ms·
> You seem to be attacking the notion of replication here, rather than anything I said. No, I was only pointing out that our evaluation of a given study cannot
by lutusp 13y ago
> You seem to be attacking the notion of replication here, rather than anything I said.
No, I was only pointing out that our evaluation of a given study cannot depend on how many other similar studies exist. I'm not saying that doesn't happen -- it does -- I'm only saying it contradicts scientific best practice.
> Random preprint on arXiv.org is going to be different than a study from a reputable researcher that's been peer reviewed.
Not in science. Peer review and the imprimatur of a "reputable researcher" cannot grant extra authority to a study, because there is no authority in science -- to the degree that a reliance on authority is a logical error.
> That doesn't mean "assume it's flawed" is correct advice.
In science? Of course it's correct advice. If studies are flawed, and if we don't make the default scientific assumption that they're broken and meaningless, we may not exercise the effort required to sift the few properly conducted studies from the chaff.
- dllthomas 13y ago"No, I was only pointing out that our evaluation of a given study cannot depend on how many other similar studies exist." 1) That's replication, and 2) that's not what I was suggesting. So I don't understand your "no" at all. To clarify a little: First, by "similar studies" I mean "studies drawn from the same population of studies", not "studies that are looking at the same thing in the same way". Second, I said that the study has a similar chance of being flawed as those other studies, not that more studies means less chance of being wrong (which would be true to that degree the chances of error are uncorrelated, but wasn't at all what I was saying). 'Not in science. Peer review and the imprimatur of a "reputable researcher" cannot grant extra authority to a study, because there is no authority in science -- to the degree that a reliance on authority is a logical error.' So wait... first, peer review is not a part of science? Second, a paper having passed peer review tells us nothing about whether it's flawed? Seriously? So if I pick two papers at random, one from "the set of papers that have not been peer reviewed" and one from "the set of papers that have been peer reviewed in a reputable journal and not found wanting", they are equally likely to be erroneous?
- lutusp 13y ago>> 'Not in science. Peer review and the imprimatur of a "reputable researcher" cannot grant extra authority to a study, because there is no authority in science -- to the degree that a reliance on authority is a logical error.' > So wait... first, peer review is not a part of science? It's part of how science is practiced, but it is not part of science's fundamentals -- the latter honors only direct evidence -- see below. > Second, a paper having passed peer review tells us nothing about whether it's flawed? Well, since I never said that, I don't need to defend it. Instead, I will refer you to the motto of the Royal Society, science's oldest professional society: http://en.wikipedia.org/wiki/Nullius_in_verba http://en.wikipedia.org/wiki/Nullius_in_verba Quote: "Nullius in verba (Latin for "on the word of no one" or "Take nobody's word for it") is the motto of the Royal Society. John Evelyn and other Royal Society fellows chose the motto soon after the founding of the Society.[1] The current Royal Society website explains the motto thus: 'It is an expression of the determination of Fellows to withstand the domination of authority and to verify all statements by an appeal to facts determined by experiment.'[2]"
- dllthomas 13y agoIf you're going to persist in flatly denying you said the things you clearly say, I'm done talking with you.