5 ms·
The companies do have scientific strategies; Posit Science, for example, has a bunch of associated studies. They're an integral part of the marketing. > Lumino
by gwern 14y ago
The companies do have scientific strategies; Posit Science, for example, has a bunch of associated studies. They're an integral part of the marketing.
> Luminosity should be fighting alongside those meta-analyses and investors should be demanding proof points in terms of p-values.
How many $$$ is a p-value or meta-analysis worth?
> Fail (scientifically) fast.
That doesn't sound very profitable. How does that put more $$$ into the founders' pockets?
- tel 14y agoIt's product-market fit if your market includes risk-bearing customers. This is, admittedly, less a problem with nebulous mental claims and more of one when it comes to, say, claiming to prevent the onset of Alzheimer's. I'd say healthcare companies in the behavioral/digital/connected health segments that have scientific strategies are outliers. Without regulation it's easy to justify trying to sneak by... but hard to cross to the b2b market.
- hmahncke 14y agoThanks gwern - I work at Posit Science, and that's been our approach since our beginning. When we got started, we ran an initial randomized controlled trial[1] which showed our cognitive training program improved memory, then ran a larger study[2], and then ran the largest study that had ever been done with commercially available brain training program[3]. Other research groups have gone on to show benefits in chemobrain[4], HIV associated neurocognitive impairment[5], and schizophrenia[6]. Different cognitive training programs are different - just like different anti-cancer molecules are different. Some are going to work, and some aren't. [1] http://www.ncbi.nlm.nih.gov/pubmed/17046669 http://www.ncbi.nlm.nih.gov/pubmed/17046669 [2] http://www.ncbi.nlm.nih.gov/pubmed/16888038 http://www.ncbi.nlm.nih.gov/pubmed/16888038 [3] http://www.ncbi.nlm.nih.gov/pubmed/19220558 http://www.ncbi.nlm.nih.gov/pubmed/19220558 [4] http://www.ncbi.nlm.nih.gov/pubmed/22918524 http://www.ncbi.nlm.nih.gov/pubmed/22918524 [5] http://www.ncbi.nlm.nih.gov/pubmed/22579081 http://www.ncbi.nlm.nih.gov/pubmed/22579081 [6] http://www.ncbi.nlm.nih.gov/pubmed/19448187 http://www.ncbi.nlm.nih.gov/pubmed/19448187
- lutusp 14y ago> I work at Posit Science, and that's been our approach since our beginning. When we got started, we ran an initial randomized controlled trial[1] which showed our cognitive training program improved memory, then ran a larger study[2], and then ran the largest study that had ever been done with commercially available brain training program[3]. Problems obvious to a scientist: 1. Poor controls. Both the experimental subjects and the researchers know, or can easily establish, which are experimental subjects and which are controls. A built-in bias. 2. The study was conducted by the company that stood to benefit from a positive result. An obvious procedural flaw. 3. Inadequate precautions against counting a placebo effect (an imaginary benefit) or Hawthorne effect (any attention paid to the subjects improves their state) as a meaningful outcomes correlated with the specific treatment rather then any treatment. But I am sure you already know this -- it presents the usual litany of legitimate objections to this kind of study. > Different cognitive training programs are different - just like different anti-cancer molecules are different. Some are going to work, and some aren't. That's true but irrelevant. The only meaningful question is which methods work in principle. There are no methods that consistently work, and no one can explain the outcomes, only describe them. Which means it's not science.
- hmahncke 14y agoYou're right - those are the usual litany of legitimate objections to this kind of study - and they are good ones. So we addressed them: 1. The trials are blinded, with more advanced studies using active controls (e.g., ordinary computer games[6], adult education[1,2,3]). Study staff administering cognitive assessments are always blind, and participants are blind in studies with active controls. The human consent forms describe the studies as comparing two forms of cognitive training. This is a good blind - in one study the wife of a control participant called my to tell me her husband loved the cognitive training because the art history education made him go to the museum more with her :-) 2. Study [3] in my post was conducted by Mayo and USC (and funded by Posit - the money has to come from somewhere), studies [4],[5],[6] were conducted entirely by independent academic investigators. 3. Active control groups control for the placebo/Hawthorne effect. As I see it, the only meaningful question is which methods work in practice - and the studies show us that. If what you mean is do we know what the mechanism of action is, we are speeding up information processing flow through the brain and making it more efficient - speaking loosely, we're improving signal/noise ratios. Reference [1] in my post has an extensive conceptual background to this approach. And we've seen that with studies of EEG/working memory [7] [7] http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2904363/ http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2904363/