5 ms·

Thanks gwern - I work at Posit Science, and that's been our approach since our beginning. When we got started, we ran an initial randomized controlled trial[1]

• lutusp 14y ago
  > I work at Posit Science, and that's been our approach since our beginning. When we got started, we ran an initial randomized controlled trial[1] which showed our cognitive training program improved memory, then ran a larger study[2], and then ran the largest study that had ever been done with commercially available brain training program[3]. Problems obvious to a scientist: 1. Poor controls. Both the experimental subjects and the researchers know, or can easily establish, which are experimental subjects and which are controls. A built-in bias. 2. The study was conducted by the company that stood to benefit from a positive result. An obvious procedural flaw. 3. Inadequate precautions against counting a placebo effect (an imaginary benefit) or Hawthorne effect (any attention paid to the subjects improves their state) as a meaningful outcomes correlated with the specific treatment rather then any treatment. But I am sure you already know this -- it presents the usual litany of legitimate objections to this kind of study. > Different cognitive training programs are different - just like different anti-cancer molecules are different. Some are going to work, and some aren't. That's true but irrelevant. The only meaningful question is which methods work in principle. There are no methods that consistently work, and no one can explain the outcomes, only describe them. Which means it's not science.
  • hmahncke 14y ago
    You're right - those are the usual litany of legitimate objections to this kind of study - and they are good ones. So we addressed them: 1. The trials are blinded, with more advanced studies using active controls (e.g., ordinary computer games[6], adult education[1,2,3]). Study staff administering cognitive assessments are always blind, and participants are blind in studies with active controls. The human consent forms describe the studies as comparing two forms of cognitive training. This is a good blind - in one study the wife of a control participant called my to tell me her husband loved the cognitive training because the art history education made him go to the museum more with her :-) 2. Study [3] in my post was conducted by Mayo and USC (and funded by Posit - the money has to come from somewhere), studies [4],[5],[6] were conducted entirely by independent academic investigators. 3. Active control groups control for the placebo/Hawthorne effect. As I see it, the only meaningful question is which methods work in practice - and the studies show us that. If what you mean is do we know what the mechanism of action is, we are speeding up information processing flow through the brain and making it more efficient - speaking loosely, we're improving signal/noise ratios. Reference [1] in my post has an extensive conceptual background to this approach. And we've seen that with studies of EEG/working memory [7] [7] http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2904363/ http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2904363/
    • gwern 14y ago
      > You're right - those are the usual litany of legitimate objections to this kind of study - and they are good ones. So we addressed them: But you guys were apparently unable to address Robert's questions: http://blog.sethroberts.net/2013/03/09/a-few-answers-from-posit-science/ http://blog.sethroberts.net/2013/03/09/a-few-answers-from-po...
      • hmahncke 14y ago
        Seth Roberts sent me a list of questions and I answered all of them. He then selectively quoted my answers. It's his blog, and there's nothing I can do about it.
    • tel 14y ago