6 ms·

No gain from brain training

38 points by robg 17y ago

tokenadult 17y ago
This link http://www.nature.com/news/2010/100420/full/4641111a.html?s=news_rss http://www.nature.com/news/2010/100420/full/4641111a.html?s=... may be out from behind the paywall for many readers. The story is interesting and important.
- carbocation 17y ago
  Fortunately, neither appear to be behind the paywall. In fact, it appears that the original article is free, as well: http://www.nature.com/nature/journal/vnfv/ncurrent/pdf/nature09042.pdf http://www.nature.com/nature/journal/vnfv/ncurrent/pdf/natur...
orborde 17y ago
Owen concedes that his findings don't necessarily mean that training in young children or elderly patients is pointless. But "the evidence is not strong", he says. "And someone needs to go and test it." This guy is a true scientist and a man after my own heart. He admits that he could be wrong, and responds with an honest analysis of the limitations of his results and a suggestion to go do some more tests to explore those dark corners.
goodside 17y ago
This paragraph from the original paper explains beautifully how little benefit brain-training really provides: "To illustrate the size of the transfer effects observed in this study, consider the following representative example from the data. The increase in the number of digits that could be remembered following training on tests designed, at least in part, to improve memory (for example, in experimental group 2) was three-hundredths of a digit. Assuming a linear relationship between time spent training and improvement, it would take almost four years of training to remember one extra digit. Moreover, the control group improved by two-tenths of a digit, with no formal memory training at all." This is far from the first study of its kind. People have been trying for decades to find any sort of training task that can improve general cognitive ability, and there is simply no evidence that one exists. Tests with sufficiently high g-loading are virtually impervious to any form of preparation or training. The closest anyone has ever come to finding one that works is the discovery that the dual n-back task improves scores on Raven's Progressive Matrices (which is very much the gold standard of g-loaded tests), and that's highly suspect given that both n-back and RPM involve the very idiosyncratic task of holding three-by-three matrices in working memory. As important as discovering a general IQ-raising task would be to psychology, there's nothing especially depressing about one not existing. Remember: You can still learn things. All this research tells you is that if you want to get good at some particular task, you have to train either at that task specifically, or something that's in some way related to it. Playing Tetris, memorizing chess openings, or practicing Bach's Goldberg Variations is simply not going to help you become a better programmer. For that, you need to program.
- silentbicycle 17y ago
  > Playing Tetris, memorizing chess openings, or practicing Bach's Goldberg Variations is simply not going to help you become a better programmer. For that, you need to program. That's leaving out the benefits from practicing general problem solving skills, logical reasoning, analyzing design trade-offs, etc., though. You make it sound like programming is all about remembering where to put the semicolons! Meditation also seems to have positive cognitive effects, though they probably come from improved focus rather than intelligence.
  - goodside 17y ago
    "That's leaving out the benefits from practicing general problem solving skills, logical reasoning, analyzing design trade-offs, etc." Yes. I do so explicitly and intentionally, as there is no evidence these benefits exist. That was my whole point.
CWuestefeld 17y ago
a series of online tasks for a minimum of ten minutes a day, three times a week, for six weeks. How much benefit can one expect from three hours of investment? If this were about, say, weight resistance training, would you expect to have bigger muscles after this program?
- jules 17y ago
  > If this were about, say, weight resistance training, would you expect to have bigger muscles after this program? Yes, absolutely, and you will have better muscles. You can have big changes in performance in a few days if you spend more time per day (but whether your muscles look bigger maybe not).
  - gaius 17y ago
    Most of the initial quick gains in weight training are related to gaining the skill needed to coordinate your muscles correctly to do say a bench press. Adding muscle requires patience - gaining 10-15lbs of muscle per year is about as good as it gets without steroids, and that's with training hard and eating right all year too.
    - jules 17y ago
      Sure that is part of it but that isn't the only thing that happens. For example: when I cycled 4 days about 60 km/day at age ~10 I was completely amazed how much easier & faster it was when I cycled to school after this vacation. This is not due to better coordination, as I had cycled every day for many years. I'm not saying that you get significantly bigger muscles quickly, but you can get better muscles quickly (or perhaps it's due to the lungs/heart?). BTW 10 lbs per year is a lot. It's more than 12 grams per day which is about 12 cm^3 per day. This is definitely noticeable. Where do you have this information from? Do you have other information like this? I'd be interested to know what exactly makes muscle tissue grow.
      - gaius 17y ago
        Loads of great info on www.crossfit.com www.t-nation.com www.gymjones.com
crux_ 17y ago
Does this mean my coffee break go games haven't been making me smarter? :( (They've actually made me worse at go; blitz builds bad habits.)
hmahncke 17y ago
The interpretation of this study is astonishingly over-reaching. The BBC researchers designed their own cognitive stimulation program, applied it at a very low intensity in healthy young people, and then saw no effect on cognitive function. To then claim that their data shows that "computerized mental workouts don't boost mental skills" is akin to saying "sugar doesn't help with a headache, and sugar and aspirin are both molecules, so aspirin must not help with headaches either." This is an elementary logical fallacy. There is a tremendous body of published evidence showing that certain specifically designed cognitive training programs drive real benefits: http://www.ncbi.nlm.nih.gov/pubmed/19220558 http://www.ncbi.nlm.nih.gov/pubmed/19220558 http://www.ncbi.nlm.nih.gov/pubmed/17565162 http://www.ncbi.nlm.nih.gov/pubmed/17565162 If the BBC researchers want to contest those results, they should use the programs and methods used in those studies. This is a fundamental principle of the scientific method, and its surprising to have to point it out to working scientists. The only conclusion from the BBC study is that very limited amounts of everyday cognitive stimulation does not improve cognitive function. This is an interesting conclusion, and the study should have reported it as such. Henry Mahncke http://www.positscience.com http://www.positscience.com I am a researcher at Posit Science, where I design and test cognitive training programs.
- hmahncke 17y ago
  I always thought it would be fun to have my work picked up at hacker news; but just not quite like this...
- tokenadult 17y ago
  Thanks for sharing the links with your brand-new Hacker News account. What about the confidence intervals in the link http://www.ncbi.nlm.nih.gov/pubmed/19220558 http://www.ncbi.nlm.nih.gov/pubmed/19220558 you kindly shared? The overlapping confidence intervals show that the effect size is meager, even if there is "statistically significant" differences in outcome between the treatment group and the control group, don't they? That's always an important issue in interpreting research, http://norvig.com/experiment-design.html http://norvig.com/experiment-design.html as pointed out in Warning Sign I5 of Peter Norvig's famous article.
techiferous 17y ago
I wonder if this applies to Dual N-Back? http://brainworkshop.sourceforge.net/ http://brainworkshop.sourceforge.net/ This claims to have the support of a scientific study.

38 points by robg 17y ago

This link http://www.nature.com/news/2010/100420/full/4641111a.html?s=news_rss http://www.nature.com/news/2010/100420/full/4641111a.html?s=... may be out from behind the paywall for many readers. The story is interesting and important.

carbocation 17y ago
Fortunately, neither appear to be behind the paywall. In fact, it appears that the original article is free, as well: http://www.nature.com/nature/journal/vnfv/ncurrent/pdf/nature09042.pdf http://www.nature.com/nature/journal/vnfv/ncurrent/pdf/natur...

orborde 17y ago

Owen concedes that his findings don't necessarily mean that training in young children or elderly patients is pointless. But "the evidence is not strong", he says. "And someone needs to go and test it." This guy is a true scientist and a man after my own heart. He admits that he could be wrong, and responds with an honest analysis of the limitations of his results and a suggestion to go do some more tests to explore those dark corners.

goodside 17y ago

This paragraph from the original paper explains beautifully how little benefit brain-training really provides: "To illustrate the size of the transfer effects observed in this study, consider the following representative example from the data. The increase in the number of digits that could be remembered following training on tests designed, at least in part, to improve memory (for example, in experimental group 2) was three-hundredths of a digit. Assuming a linear relationship between time spent training and improvement, it would take almost four years of training to remember one extra digit. Moreover, the control group improved by two-tenths of a digit, with no formal memory training at all." This is far from the first study of its kind. People have been trying for decades to find any sort of training task that can improve general cognitive ability, and there is simply no evidence that one exists. Tests with sufficiently high g-loading are virtually impervious to any form of preparation or training. The closest anyone has ever come to finding one that works is the discovery that the dual n-back task improves scores on Raven's Progressive Matrices (which is very much the gold standard of g-loaded tests), and that's highly suspect given that both n-back and RPM involve the very idiosyncratic task of holding three-by-three matrices in working memory. As important as discovering a general IQ-raising task would be to psychology, there's nothing especially depressing about one not existing. Remember: You can still learn things. All this research tells you is that if you want to get good at some particular task, you have to train either at that task specifically, or something that's in some way related to it. Playing Tetris, memorizing chess openings, or practicing Bach's Goldberg Variations is simply not going to help you become a better programmer. For that, you need to program.

silentbicycle 17y ago
> Playing Tetris, memorizing chess openings, or practicing Bach's Goldberg Variations is simply not going to help you become a better programmer. For that, you need to program. That's leaving out the benefits from practicing general problem solving skills, logical reasoning, analyzing design trade-offs, etc., though. You make it sound like programming is all about remembering where to put the semicolons! Meditation also seems to have positive cognitive effects, though they probably come from improved focus rather than intelligence.
- goodside 17y ago
  "That's leaving out the benefits from practicing general problem solving skills, logical reasoning, analyzing design trade-offs, etc." Yes. I do so explicitly and intentionally, as there is no evidence these benefits exist. That was my whole point.

CWuestefeld 17y ago

a series of online tasks for a minimum of ten minutes a day, three times a week, for six weeks. How much benefit can one expect from three hours of investment? If this were about, say, weight resistance training, would you expect to have bigger muscles after this program?

jules 17y ago
> If this were about, say, weight resistance training, would you expect to have bigger muscles after this program? Yes, absolutely, and you will have better muscles. You can have big changes in performance in a few days if you spend more time per day (but whether your muscles look bigger maybe not).
- gaius 17y ago
  Most of the initial quick gains in weight training are related to gaining the skill needed to coordinate your muscles correctly to do say a bench press. Adding muscle requires patience - gaining 10-15lbs of muscle per year is about as good as it gets without steroids, and that's with training hard and eating right all year too.
  - jules 17y ago
    Sure that is part of it but that isn't the only thing that happens. For example: when I cycled 4 days about 60 km/day at age ~10 I was completely amazed how much easier & faster it was when I cycled to school after this vacation. This is not due to better coordination, as I had cycled every day for many years. I'm not saying that you get significantly bigger muscles quickly, but you can get better muscles quickly (or perhaps it's due to the lungs/heart?). BTW 10 lbs per year is a lot. It's more than 12 grams per day which is about 12 cm^3 per day. This is definitely noticeable. Where do you have this information from? Do you have other information like this? I'd be interested to know what exactly makes muscle tissue grow.
    - gaius 17y ago
      Loads of great info on www.crossfit.com www.t-nation.com www.gymjones.com

crux_ 17y ago

Does this mean my coffee break go games haven't been making me smarter? :( (They've actually made me worse at go; blitz builds bad habits.)

hmahncke 17y ago

The interpretation of this study is astonishingly over-reaching. The BBC researchers designed their own cognitive stimulation program, applied it at a very low intensity in healthy young people, and then saw no effect on cognitive function. To then claim that their data shows that "computerized mental workouts don't boost mental skills" is akin to saying "sugar doesn't help with a headache, and sugar and aspirin are both molecules, so aspirin must not help with headaches either." This is an elementary logical fallacy. There is a tremendous body of published evidence showing that certain specifically designed cognitive training programs drive real benefits: http://www.ncbi.nlm.nih.gov/pubmed/19220558 http://www.ncbi.nlm.nih.gov/pubmed/19220558 http://www.ncbi.nlm.nih.gov/pubmed/17565162 http://www.ncbi.nlm.nih.gov/pubmed/17565162 If the BBC researchers want to contest those results, they should use the programs and methods used in those studies. This is a fundamental principle of the scientific method, and its surprising to have to point it out to working scientists. The only conclusion from the BBC study is that very limited amounts of everyday cognitive stimulation does not improve cognitive function. This is an interesting conclusion, and the study should have reported it as such. Henry Mahncke http://www.positscience.com http://www.positscience.com I am a researcher at Posit Science, where I design and test cognitive training programs.

hmahncke 17y ago
I always thought it would be fun to have my work picked up at hacker news; but just not quite like this...
tokenadult 17y ago
Thanks for sharing the links with your brand-new Hacker News account. What about the confidence intervals in the link http://www.ncbi.nlm.nih.gov/pubmed/19220558 http://www.ncbi.nlm.nih.gov/pubmed/19220558 you kindly shared? The overlapping confidence intervals show that the effect size is meager, even if there is "statistically significant" differences in outcome between the treatment group and the control group, don't they? That's always an important issue in interpreting research, http://norvig.com/experiment-design.html http://norvig.com/experiment-design.html as pointed out in Warning Sign I5 of Peter Norvig's famous article.

techiferous 17y ago

I wonder if this applies to Dual N-Back? http://brainworkshop.sourceforge.net/ http://brainworkshop.sourceforge.net/ This claims to have the support of a scientific study.