3 ms·

I think you missed my 2nd paragraph. What I was saying was: no, this article didn't show that. The only thing it's exposed so far is that "science journalists d

I think you missed my 2nd paragraph. What I was saying was: no, this article didn't show that. The only thing it's exposed so far is that "science journalists don't watch out for this vulnerability". Well, so what if they don't? There are a million things people don't watch out for. Is this one actually causing problems in reality, or is this a hypothetical thing that doesn't occur in practice? To show this is a real problem requires providing evidence of real-world p-hacking in the wild that has had significant actual adverse consequences.

mixologic 4y ago
That was the whole point. Science journalists dont do the jobs we think they do. Our presumption is that they do not trust anything as true facts until they verify and vet the information. I.e do what a journalists do. Report whats true But thats not actually their job anymore. Their job is to get eyeballs. Doesnt really matter whats true.
- lupire 4y ago
  In no sense is this limited to science journalism. Bizarrely, the most reputable journalist are probably at TMZ, because they get sued for libel if they are wrong. When there is no cost for being wrong, there is no incentive for truth.
imgabe 4y ago
No, the prank pointed out multiple failures in different stages of the process. > But even if we had been careful to avoid p-hacking, our study was doomed by the tiny number of subjects, which amplifies the effects of uncontrolled factors. ... Which is why you need to use a large number of people, and balance age and gender across treatment groups. (We didn’t bother.) This poor design should have been reason enough not to publish the study - p-hacking or not. But... > Our paper was accepted for publication by multiple journals within 24 hours. Needless to say, we faced no peer review at all. The eager suitor we ultimately chose was the the International Archives of Medicine. It got published anyway in a "journal" with a fancy sounding name that doesn't actually do any rigorous peer review. The team then issued a press release that the press regurgitated without any scrutiny. If they had bothered to look and if they were good at their jobs, they might have noticed that the journal was not reputable. Anybody familiar with any academic field can recite off the top of their head what the reputable journals they'd want to be published in are. If it's not one of those, it deserves a closer look. That's without even doing some basic googling about the team behind the research which should have been able to uncover that they were nothing but a website.
- lupire 4y ago
  The problem here is a breakdown in society. Why do sham journals exist? Why do their press release reach the ears of journalists? Why is there no web of trust of credibility? Why have all our institutions (flawed as they may have been) collapsed?

mixologic 4y ago

That was the whole point. Science journalists dont do the jobs we think they do. Our presumption is that they do not trust anything as true facts until they verify and vet the information. I.e do what a journalists do. Report whats true But thats not actually their job anymore. Their job is to get eyeballs. Doesnt really matter whats true.

lupire 4y ago
In no sense is this limited to science journalism. Bizarrely, the most reputable journalist are probably at TMZ, because they get sued for libel if they are wrong. When there is no cost for being wrong, there is no incentive for truth.

imgabe 4y ago

No, the prank pointed out multiple failures in different stages of the process. > But even if we had been careful to avoid p-hacking, our study was doomed by the tiny number of subjects, which amplifies the effects of uncontrolled factors. ... Which is why you need to use a large number of people, and balance age and gender across treatment groups. (We didn’t bother.) This poor design should have been reason enough not to publish the study - p-hacking or not. But... > Our paper was accepted for publication by multiple journals within 24 hours. Needless to say, we faced no peer review at all. The eager suitor we ultimately chose was the the International Archives of Medicine. It got published anyway in a "journal" with a fancy sounding name that doesn't actually do any rigorous peer review. The team then issued a press release that the press regurgitated without any scrutiny. If they had bothered to look and if they were good at their jobs, they might have noticed that the journal was not reputable. Anybody familiar with any academic field can recite off the top of their head what the reputable journals they'd want to be published in are. If it's not one of those, it deserves a closer look. That's without even doing some basic googling about the team behind the research which should have been able to uncover that they were nothing but a website.

lupire 4y ago
The problem here is a breakdown in society. Why do sham journals exist? Why do their press release reach the ears of journalists? Why is there no web of trust of credibility? Why have all our institutions (flawed as they may have been) collapsed?