3 ms·

Ok, then let's funding psychology like we fund physics. I would love to run 1000+ patient studies to test psychotherapies, and in fact we'd be able to answer so

Ok, then let's funding psychology like we fund physics. I would love to run 1000+ patient studies to test psychotherapies, and in fact we'd be able to answer some really interesting questions if we did, but there is currently no way of doing this.

tokenadult 16y ago
Let's fund psychology like that when the psychologists define their hypotheses before the experiment begins as well as physicists do.
- bigfudge 16y ago
  That's a cheap shot. Our trial will publish a detailed protocol and analysis plan, as do most large, publicly funded trials. Small-scale experimental work is a different matter. I personally agree that all experiments which could end up in peer reviewed journals should be registered before participants are recruited. This would be simple to do by submitting ethics applications and an analysis plan to a trusted third party which would only release them once the author is in a position to publish, or at a pre-agreed cutoff (perhaps 2 years), whichever is the shorter (to avoid scooping). Perhaps I should set something up...
Eliezer 16y ago
I repeat, you do not need 1000 times as many subjects to get results that are 1000 times as significant! If 40 subjects gets you results with p < 0.05, then 100 subjects should get you results with p < 0.001. Doing half as many experiments and having nearly all the published results being real effects, instead of most of them failing to replicate when tested, sounds like a great tradeoff to me. And I suspect the ultimate reason it's not done this way... is that scientists in certain fields would publish a lot fewer papers, not slightly fewer but a lot fewer, if all the effects they were studying had to be real.
- saurik 16y ago
  "1000+", not "1000x". Also, I'm assuming bigfudge was talking about p < 0.0001, given the comparison made to physicists.
  - bigfudge 16y ago
    Yes - thanks. The current norm for a 'suitably powered' trial of a psychotherapy is about 300. We've just got a trial funded for that number (admittedly in a challenging patient population) which will cost about £2.5m in research and treatment costs. We would love to run 1000 patients and start looking at therapist-client interactions, individual differences in treatment suitability but that's out of the question.

tokenadult 16y ago

Let's fund psychology like that when the psychologists define their hypotheses before the experiment begins as well as physicists do.

bigfudge 16y ago
That's a cheap shot. Our trial will publish a detailed protocol and analysis plan, as do most large, publicly funded trials. Small-scale experimental work is a different matter. I personally agree that all experiments which could end up in peer reviewed journals should be registered before participants are recruited. This would be simple to do by submitting ethics applications and an analysis plan to a trusted third party which would only release them once the author is in a position to publish, or at a pre-agreed cutoff (perhaps 2 years), whichever is the shorter (to avoid scooping). Perhaps I should set something up...

Eliezer 16y ago

I repeat, you do not need 1000 times as many subjects to get results that are 1000 times as significant! If 40 subjects gets you results with p < 0.05, then 100 subjects should get you results with p < 0.001. Doing half as many experiments and having nearly all the published results being real effects, instead of most of them failing to replicate when tested, sounds like a great tradeoff to me. And I suspect the ultimate reason it's not done this way... is that scientists in certain fields would publish a lot fewer papers, not slightly fewer but a lot fewer, if all the effects they were studying had to be real.

saurik 16y ago
"1000+", not "1000x". Also, I'm assuming bigfudge was talking about p < 0.0001, given the comparison made to physicists.
- bigfudge 16y ago
  Yes - thanks. The current norm for a 'suitably powered' trial of a psychotherapy is about 300. We've just got a trial funded for that number (admittedly in a challenging patient population) which will cost about £2.5m in research and treatment costs. We would love to run 1000 patients and start looking at therapist-client interactions, individual differences in treatment suitability but that's out of the question.