5 ms·

All of these brain training products are suspect. Evidence for far transfer (training in one task transferring to a different domain task) is surprisingly hard

All of these brain training products are suspect. Evidence for far transfer (training in one task transferring to a different domain task) is surprisingly hard to find, and empirical findings otherwise tend to disappear or diminish when replicated. Many of the pro-brain-training camp have already begun to shift the goal posts. First it was 'simple games increase IQ,' which turned out to be difficult to prove when well controlled studies were performed. Now it's more along the lines of 'These simple games might have preventative effects against age related declines!,' which is an even harder claim to actually prove given the difficulties performing well controlled studies on aged participants. In the cognitive science world, if we discovered a solid far transfer paradigm, especially one which transferred to something like G(eneral Intelligence), it would be our anti-baldness pill\flying car\4-day cellphone battery. People thought that these working memory transfer effects were the real deal and got very excited about it, money poured in, and the water got muddied by all these scientists with conflicts. I obviously don't put much stock in working memory training. I wish it worked like they said, but I don't think it does. If far-transfer shows up at all, it's tiny, and doesn't persist after delay.

bberrry 11y ago
I was under the impression that Dual-n-Back[0] had real benefits. Is that not the case? [0] http://brainworkshop.sourceforge.net/ http://brainworkshop.sourceforge.net/
- w1ntermute 11y ago
  Gwern's done extensive research into this, and his meta-analysis has shown there's "a net gain (medium-sized) on the post-training IQ tests"[0, 1]. 0: http://www.gwern.net/DNB%20meta-analysis http://www.gwern.net/DNB%20meta-analysis 1: http://www.gwern.net/DNB%20FAQ http://www.gwern.net/DNB%20FAQ
  - knowaveragejoe 11y ago
    Why is this downvoted? I was under the impression that Gwern did solid resarch and was well respected in the HN community.
    - deleted 11y ago
      [deleted]
    - iaw 11y ago
      Because JumpCrisscross's post [0] implies that w1ntermute cherry picked a component of the Gwern meta-analysis to prove his point while the next paragraph refuted it. [0] : https://news.ycombinator.com/item?id=10846443 https://news.ycombinator.com/item?id=10846443
  - ryanferg 11y ago
    This is really great! I'll have to look into this when I have a little while. Thanks for passing it on. Just on a glance, I disagree that it really shows much of an increase though. Just glancing at the forest plots of those effect sizes tells me that this isn't a very strong effect at all (if it's there). And there is also the so called 'file drawer' effect. I was at one of the big cog psych conferences a few years ago when a colleague was asking around -'Do you have any failed to replicates for WM training?' Everyone was so excited with the original Jaeggi 2008 paper, went out and tried it, and had a tough time replicating what turned out to be a severely flawed study.
    - ryanferg 11y ago
      Not to say that I'm not open to the idea. I'd love for it to be true, I just think large-effect-size effects are not often mired in the controversy that this one is. They're hard to find.
  - JumpCrisscross 11y ago
    Literally the next paragraph: "The size of this increase on IQ test score correlates highly with the methodological concern of whether a study used active or passive control groups. This indicates that the medium effect size is due to methodological problems and that n-back training does not increase subjects’ underlying fluid intelligence but the gains are due to the motivational effect of passive control groups (who did not train on anything) not trying as hard as the n-back-trained experimental groups on the post-tests. The remaining studies using active control groups find a small positive effect (but this may be due to matrix-test-specific training, undetected publication bias, smaller motivational effects, etc.)" http://www.gwern.net/DNB%20meta-analysis#analysis http://www.gwern.net/DNB%20meta-analysis#analysis
- ryanferg 11y ago
  So it definitely transfers 'near'. You'll get better at the dual-n-back test. And possibly other visio-spatial working memory tests. But the question of far transfer- will it make you smarter- is probably not. Here is a link to a 2015 meta-analysis that looks at the question of working memory training (including dual-n-backs) transferring to other working memory tasks: https://www.dropbox.com/s/084fvteji1tyz8t/2015-schwaighofer.pdf https://www.dropbox.com/s/084fvteji1tyz8t/2015-schwaighofer.... Melby-Lervag & Hulme, 2013, is also pretty damning.
  - cLeEOGPw 11y ago
    That's exactly, if my understanding of changes happening to brain while performing any activity in general is correct, what should happen - you get better at doing that activity. So, unless your work contains some kind of dual-n-back activity, you are not getting better at it. You get better at work by doing work, not by playing brain game. I am really happy they got fined for this deception.
- cant_kant 11y ago
  From a meta-analysis: " The 20 studies included here were all completed between 2008 and 2013....Sample sizes of treatment groups varied between 7 and 36 participants, and control groups between 8 and 43" "net effect of n-back training on Gf outcome measures, about the equivalent of 3–4 points on a standardized IQ test" ie: very small groups, tiny effect. Sounds dubious to me, like much research in the social "sciences" Improving fluid intelligence with training on working memory:a meta-analysis http://scottbarrykaufman.com/wp-content/uploads/2014/08/au-et-al2014_nback-training-gf-meta-analysis.pdf http://scottbarrykaufman.com/wp-content/uploads/2014/08/au-e...
  - freehunter 11y ago
    Considering the standard deviation for IQ is 15 points, I wouldn't drink an extra glass of water per day for another 3-4 IQ points, let alone play a training game for any extended amount of time. 3-4 points is statistically meaningless.
    - cant_kant 11y ago
      "Considering the standard deviation for IQ is 15 points" The standard deviation for the distribution of IQ is 15 points. The standard deviation of the errors of measurement associated with measuring IQ is 3 points.( ie: the standard error of measurement aka SEM) I would be willing to have 3 fingers of my left hand ( not including my thumb ) amputated to improve my IQ by 1 standard deviation. For more on Standard Error of Measurement, see http://www.csus.edu/indiv/b/brocks/Courses/EDS%20245/Handouts/Standard%20Error%20of%20Measurement.ITEM.pdf http://www.csus.edu/indiv/b/brocks/Courses/EDS%20245/Handout...
      - cbd1984 11y ago
        That link is a PDF.
      - corndoge 11y ago
        What is your point? The parent made no error in his language. The inferred meaning of 'standard deviation' in this context is 'standard deviation for distribution'. The parent referred to an improvement of 3-4 IQ points, not standard deviations.
pocketstar 11y ago
4-day smartphone* battery, I can remember when I only charged my mobile telephone once a week.
- function_seven 11y ago
  I'm sure that same mobile telephone was capable of far less than a smartphone. I remember my Nokias being able to last for days on end (unless I played a lot of Snake!) While I wouldn't trade my current smartphone for one of those, I do miss the lack of battery anxiety, and wish the manufacturers would make models for those of us who aren't obsessed with thinness. An iPhone 6 at 5/8" (15mm) would be awesome.
  - spooningtamarin 11y ago
    Nope, it's just bad software engineers. Africa has those pretty very battery friendly "smart phones" and they do everything that yours does, probably without the fancy games. I've been tweaking my Android tablet's code (10" screen), profiling, setting up all of the possible GPU optimizations (whole UI is whenever possible displayed using it) and this is the result. http://imgur.com/QLN17SS http://imgur.com/QLN17SS This is about a one year worth of work but I guess Android engineers don't have time to optimize per device. You could probably squeeze out the same for any mobile device, although iPhone is probably not that configurable.
    - raverbashing 11y ago
      Africa's "smart phones" (whatever that means) are usually a feature-phone with a web browser. Much less capable than a smartphone Turn off wifi and 3G/4G see how long your smartphone battery lasts. Oh I did this recently with an old phone, it lasts around 4 days as well (and that's with an old battery) Also a lot of apps abuse smartphone processing updating how many times per minute But screens and processing power still cost a lot of energy
    - _nedR 11y ago
      Really want to read more about it. You should do blog or do a write-up on it on xda-developers about it. Hell you should release a mod for whatever tablet you are using, and make a killing.

bberrry 11y ago

I was under the impression that Dual-n-Back[0] had real benefits. Is that not the case? [0] http://brainworkshop.sourceforge.net/ http://brainworkshop.sourceforge.net/

w1ntermute 11y ago
Gwern's done extensive research into this, and his meta-analysis has shown there's "a net gain (medium-sized) on the post-training IQ tests"[0, 1]. 0: http://www.gwern.net/DNB%20meta-analysis http://www.gwern.net/DNB%20meta-analysis 1: http://www.gwern.net/DNB%20FAQ http://www.gwern.net/DNB%20FAQ
- knowaveragejoe 11y ago
  Why is this downvoted? I was under the impression that Gwern did solid resarch and was well respected in the HN community.
  - deleted 11y ago
    [deleted]
  - iaw 11y ago
    Because JumpCrisscross's post [0] implies that w1ntermute cherry picked a component of the Gwern meta-analysis to prove his point while the next paragraph refuted it. [0] : https://news.ycombinator.com/item?id=10846443 https://news.ycombinator.com/item?id=10846443
- ryanferg 11y ago
  This is really great! I'll have to look into this when I have a little while. Thanks for passing it on. Just on a glance, I disagree that it really shows much of an increase though. Just glancing at the forest plots of those effect sizes tells me that this isn't a very strong effect at all (if it's there). And there is also the so called 'file drawer' effect. I was at one of the big cog psych conferences a few years ago when a colleague was asking around -'Do you have any failed to replicates for WM training?' Everyone was so excited with the original Jaeggi 2008 paper, went out and tried it, and had a tough time replicating what turned out to be a severely flawed study.
  - ryanferg 11y ago
    Not to say that I'm not open to the idea. I'd love for it to be true, I just think large-effect-size effects are not often mired in the controversy that this one is. They're hard to find.
- JumpCrisscross 11y ago
  Literally the next paragraph: "The size of this increase on IQ test score correlates highly with the methodological concern of whether a study used active or passive control groups. This indicates that the medium effect size is due to methodological problems and that n-back training does not increase subjects’ underlying fluid intelligence but the gains are due to the motivational effect of passive control groups (who did not train on anything) not trying as hard as the n-back-trained experimental groups on the post-tests. The remaining studies using active control groups find a small positive effect (but this may be due to matrix-test-specific training, undetected publication bias, smaller motivational effects, etc.)" http://www.gwern.net/DNB%20meta-analysis#analysis http://www.gwern.net/DNB%20meta-analysis#analysis
ryanferg 11y ago
So it definitely transfers 'near'. You'll get better at the dual-n-back test. And possibly other visio-spatial working memory tests. But the question of far transfer- will it make you smarter- is probably not. Here is a link to a 2015 meta-analysis that looks at the question of working memory training (including dual-n-backs) transferring to other working memory tasks: https://www.dropbox.com/s/084fvteji1tyz8t/2015-schwaighofer.pdf https://www.dropbox.com/s/084fvteji1tyz8t/2015-schwaighofer.... Melby-Lervag & Hulme, 2013, is also pretty damning.
- cLeEOGPw 11y ago
  That's exactly, if my understanding of changes happening to brain while performing any activity in general is correct, what should happen - you get better at doing that activity. So, unless your work contains some kind of dual-n-back activity, you are not getting better at it. You get better at work by doing work, not by playing brain game. I am really happy they got fined for this deception.
cant_kant 11y ago
From a meta-analysis: " The 20 studies included here were all completed between 2008 and 2013....Sample sizes of treatment groups varied between 7 and 36 participants, and control groups between 8 and 43" "net effect of n-back training on Gf outcome measures, about the equivalent of 3–4 points on a standardized IQ test" ie: very small groups, tiny effect. Sounds dubious to me, like much research in the social "sciences" Improving fluid intelligence with training on working memory:a meta-analysis http://scottbarrykaufman.com/wp-content/uploads/2014/08/au-et-al2014_nback-training-gf-meta-analysis.pdf http://scottbarrykaufman.com/wp-content/uploads/2014/08/au-e...
- freehunter 11y ago
  Considering the standard deviation for IQ is 15 points, I wouldn't drink an extra glass of water per day for another 3-4 IQ points, let alone play a training game for any extended amount of time. 3-4 points is statistically meaningless.
  - cant_kant 11y ago
    "Considering the standard deviation for IQ is 15 points" The standard deviation for the distribution of IQ is 15 points. The standard deviation of the errors of measurement associated with measuring IQ is 3 points.( ie: the standard error of measurement aka SEM) I would be willing to have 3 fingers of my left hand ( not including my thumb ) amputated to improve my IQ by 1 standard deviation. For more on Standard Error of Measurement, see http://www.csus.edu/indiv/b/brocks/Courses/EDS%20245/Handouts/Standard%20Error%20of%20Measurement.ITEM.pdf http://www.csus.edu/indiv/b/brocks/Courses/EDS%20245/Handout...
    - cbd1984 11y ago
      That link is a PDF.
    - corndoge 11y ago
      What is your point? The parent made no error in his language. The inferred meaning of 'standard deviation' in this context is 'standard deviation for distribution'. The parent referred to an improvement of 3-4 IQ points, not standard deviations.

pocketstar 11y ago

4-day smartphone* battery, I can remember when I only charged my mobile telephone once a week.

function_seven 11y ago
I'm sure that same mobile telephone was capable of far less than a smartphone. I remember my Nokias being able to last for days on end (unless I played a lot of Snake!) While I wouldn't trade my current smartphone for one of those, I do miss the lack of battery anxiety, and wish the manufacturers would make models for those of us who aren't obsessed with thinness. An iPhone 6 at 5/8" (15mm) would be awesome.
- spooningtamarin 11y ago
  Nope, it's just bad software engineers. Africa has those pretty very battery friendly "smart phones" and they do everything that yours does, probably without the fancy games. I've been tweaking my Android tablet's code (10" screen), profiling, setting up all of the possible GPU optimizations (whole UI is whenever possible displayed using it) and this is the result. http://imgur.com/QLN17SS http://imgur.com/QLN17SS This is about a one year worth of work but I guess Android engineers don't have time to optimize per device. You could probably squeeze out the same for any mobile device, although iPhone is probably not that configurable.
  - raverbashing 11y ago
    Africa's "smart phones" (whatever that means) are usually a feature-phone with a web browser. Much less capable than a smartphone Turn off wifi and 3G/4G see how long your smartphone battery lasts. Oh I did this recently with an old phone, it lasts around 4 days as well (and that's with an old battery) Also a lot of apps abuse smartphone processing updating how many times per minute But screens and processing power still cost a lot of energy
  - _nedR 11y ago
    Really want to read more about it. You should do blog or do a write-up on it on xda-developers about it. Hell you should release a mod for whatever tablet you are using, and make a killing.