3 ms·

I'm curious - and no expert here - but if what you say is correct (I haven't checked the paper), I thought this is the sort of thing peer review screened for?

I'm curious - and no expert here - but if what you say is correct (I haven't checked the paper), I thought this is the sort of thing peer review screened for? If it's the case there was no control, why would peer review allow it and why would that not be mentioned in the article about it?

true_religion 4y ago
“Passing” peer review is a bit of a misnomer. Peer review doesn’t prevent you from publishing anything. It just prevents you from publishing obviously flawed, or obviously fraudulent experimental results in this particular journal. You can always shop around to different journals, use disclaimers, get the name of a prominent peer on tour paper, or find a journal with no reviewer. If you really want to publish, you will. Disclaimer: I haven’t read the actual experimental paper. I don’t claim they are flawed. Lack of a control doesn’t mean you are flawed. You could be using baseline results from another study as a comparison.
- dTal 4y ago
  >Lack of a control doesn’t mean you are flawed. You could be using baseline results from another study as a comparison. No you can't! This is naughty! There are too many uncontrolled variables between studies. You need to run your own control group, every time, to have any experimental power. To be sure, it's common to cheat in the way that you describe, for budgetary reasons. That doesn't make it good science.
dTal 4y ago
There is a group they label "control", but they simply get fed flicker at frequencies they predict to be non-optimal. They did measure differences, so the experiment does have power (unlike an experiment with no control group at all). It's a valid experiment, on its own terms. But without a 0-flicker control group we can't absolutely say whether it improves learning over the 0-flicker baseline we all experience in daily life, only that some flicker frequencies/phases are better than others (and by a wide margin). This is still an interesting result! There may be other results I'm not aware of that argue against my discombobulation theory. I'm sure the experimenters have a deeper grasp of the field than I do.
yorwba 4y ago
It's still an interesting effect, even if it isn't useful. The reviewers were presumably evaluating the paper on its merits as a neuroscience paper, not its suitability as an educational treatment. The reason the article hypes the results so much nonetheless is that it's a press release by the university where the researchers work.

true_religion 4y ago

“Passing” peer review is a bit of a misnomer. Peer review doesn’t prevent you from publishing anything. It just prevents you from publishing obviously flawed, or obviously fraudulent experimental results in this particular journal. You can always shop around to different journals, use disclaimers, get the name of a prominent peer on tour paper, or find a journal with no reviewer. If you really want to publish, you will. Disclaimer: I haven’t read the actual experimental paper. I don’t claim they are flawed. Lack of a control doesn’t mean you are flawed. You could be using baseline results from another study as a comparison.

dTal 4y ago
>Lack of a control doesn’t mean you are flawed. You could be using baseline results from another study as a comparison. No you can't! This is naughty! There are too many uncontrolled variables between studies. You need to run your own control group, every time, to have any experimental power. To be sure, it's common to cheat in the way that you describe, for budgetary reasons. That doesn't make it good science.

dTal 4y ago

There is a group they label "control", but they simply get fed flicker at frequencies they predict to be non-optimal. They did measure differences, so the experiment does have power (unlike an experiment with no control group at all). It's a valid experiment, on its own terms. But without a 0-flicker control group we can't absolutely say whether it improves learning over the 0-flicker baseline we all experience in daily life, only that some flicker frequencies/phases are better than others (and by a wide margin). This is still an interesting result! There may be other results I'm not aware of that argue against my discombobulation theory. I'm sure the experimenters have a deeper grasp of the field than I do.

yorwba 4y ago

It's still an interesting effect, even if it isn't useful. The reviewers were presumably evaluating the paper on its merits as a neuroscience paper, not its suitability as an educational treatment. The reason the article hypes the results so much nonetheless is that it's a press release by the university where the researchers work.