4 ms·

The game looks like a gamified version of the test they're using to measure concentration. It seems likely that the subjects are getting better at that specific

by ssfrr 8y ago

The game looks like a gamified version of the test they're using to measure concentration. It seems likely that the subjects are getting better at that specific task through practice, but I'd be skeptical that the results would generalize to other tasks requiring concentration.

sam_goody 8y ago
Goodhart's law: "When a measure becomes a target, it ceases to be a good measure." If it is just training to become better at the test to measure ADHD (I don't know enough about the game to be sure) than it would be a classic application of that "law".
- WhompingWindows 8y ago
  I don't know much about the product either, but there are not good quantitative tests to measure ADHD. ADHD is a multifactorial diagnosis that can present with myriad symptoms, and it's diagnosed by examining a list of symptoms and their effect on the patient's life. For instance, do you lose things you need, are you attentive to details, do you daydream or space out to your own detriment, do you fidget or move when it's inappropriate, etc. It'd be very lackluster for a single test, or a single person, to judge these symptoms. The gold standard ADHD diagnosis would involve examining the child (or adult) in multiple settings with multiple people's evidence, and then comparing symptoms against those for anxiety, depression, and other MH diagnoses. If they have enough symptoms from the official list, and they've been carefully examined, they may qualify. A simple game/test will never be able to replace that process.
  - maceurt 8y ago
    Furthermore, many tests for adhd also work as IQ tests, meaning if you have a loq iq you are more likely to get a lower score, and bice versa. I remember one of the tests for adhd involved response time to visual stimuli and how fast a person responded. Response time has a direct correlation to IQ.
  - paulie_a 8y ago
    I generally agree but there are circumstances where you can just tell. My experience as an adult being rediagnosed was honestly answering a survey of questions. Half way through the psychiatrist already knew it. There was no question. I was originally diagnosed in college. And have managed it with a combo of effort and medication. Life has been good. Now I have two family member adolescents. One was diagnosed, and I knew he absolutely did not have it. It took two years before the psychiatrist admitted it was anxiety. Meanwhile the younger one definitely does, but personally I feel he is too young for medication or even be diagnosed. He will eventually be properly diagnosed and be prescribed. I know there is a lot of misdiagnosis, but sometimes it's incredibly obvious even on a first interaction. I am not a psychiatrist but sometimes you just recognize your people.
  - ThrustVectoring 8y ago
    One of the big issues with these distress-based diagnosis metrics is that it fails to diagnose people whose social environments are particularly well suited for their mental health issues. If an autistic child has parents who manage their sensory sensitivities and work well with their communicative difficulties, the system sees a child who isn't experiencing a lot of the distress-based symptoms used to diagnose autism. So like, your gold standard diagnosis ends up depending a lot on the organizational and attention demands the patient is required to meet. A business analyst who is required to sit at a desk for eight hours a day and pay attention to boring numbers on a business spreadsheet is much more likely to get diagnosed with ADHD, while a barber that hangs out and does short haircuts while interacting with clients is much less likely to merit such diagnosis.
eranation 8y ago
Exactly. About the game: "To meet the objective, users have to identify different combinations of number strings in missions littered with distraction." About the test (used to prove the game does improve cognitive abilities): "Those tested were asked to detect digit sequences (such as 2-4 or 6-8), and then hit a button once detected as quickly as they could — and multiple sequences could appear at the same time" This feels like a paid article to promote the Peak app. No science here unfortunately. I don't understand how public funds pay for studies like this.
Bartweiss 8y ago
Dual N-back is currently the premiere brain-training task, by which I mean it's the only one that's ever looked at all promising for general benefit. Unfortunately, the newest and best studies suggest that even n-back performance is basically just a training effect on a single test, plus possibly improvement in visual processing. (In layman's terms, your memory doesn't improve, but you do get better at carefully watching for a dot.) This isn't being advertised as more substantive than dual n-back, just more gamified and playable. Given that n-back doesn't seem to work even if you're absolutely diligent about doing it, my expectation for "less rigorous but more playable" is that it will have no value whatsoever. https://www.gwern.net/docs/dnb/2013-colom.pdf https://www.gwern.net/docs/dnb/2013-colom.pdf
systemBuilder 8y ago
While this may be true, I have found with my own son's that the number one most popular soccer skills test (MacDonald test, 1960) is also a good indication of soccer skills and you can train for it. It's a good adjunct to regular participation in a recreational soccer program. My sons achieved competitive A and B league performance (nerd kids, sons of a PhD computer scientists).

by ssfrr 8y ago

sam_goody 8y ago

Goodhart's law: "When a measure becomes a target, it ceases to be a good measure." If it is just training to become better at the test to measure ADHD (I don't know enough about the game to be sure) than it would be a classic application of that "law".

WhompingWindows 8y ago
I don't know much about the product either, but there are not good quantitative tests to measure ADHD. ADHD is a multifactorial diagnosis that can present with myriad symptoms, and it's diagnosed by examining a list of symptoms and their effect on the patient's life. For instance, do you lose things you need, are you attentive to details, do you daydream or space out to your own detriment, do you fidget or move when it's inappropriate, etc. It'd be very lackluster for a single test, or a single person, to judge these symptoms. The gold standard ADHD diagnosis would involve examining the child (or adult) in multiple settings with multiple people's evidence, and then comparing symptoms against those for anxiety, depression, and other MH diagnoses. If they have enough symptoms from the official list, and they've been carefully examined, they may qualify. A simple game/test will never be able to replace that process.
- maceurt 8y ago
  Furthermore, many tests for adhd also work as IQ tests, meaning if you have a loq iq you are more likely to get a lower score, and bice versa. I remember one of the tests for adhd involved response time to visual stimuli and how fast a person responded. Response time has a direct correlation to IQ.
- paulie_a 8y ago
  I generally agree but there are circumstances where you can just tell. My experience as an adult being rediagnosed was honestly answering a survey of questions. Half way through the psychiatrist already knew it. There was no question. I was originally diagnosed in college. And have managed it with a combo of effort and medication. Life has been good. Now I have two family member adolescents. One was diagnosed, and I knew he absolutely did not have it. It took two years before the psychiatrist admitted it was anxiety. Meanwhile the younger one definitely does, but personally I feel he is too young for medication or even be diagnosed. He will eventually be properly diagnosed and be prescribed. I know there is a lot of misdiagnosis, but sometimes it's incredibly obvious even on a first interaction. I am not a psychiatrist but sometimes you just recognize your people.
- ThrustVectoring 8y ago
  One of the big issues with these distress-based diagnosis metrics is that it fails to diagnose people whose social environments are particularly well suited for their mental health issues. If an autistic child has parents who manage their sensory sensitivities and work well with their communicative difficulties, the system sees a child who isn't experiencing a lot of the distress-based symptoms used to diagnose autism. So like, your gold standard diagnosis ends up depending a lot on the organizational and attention demands the patient is required to meet. A business analyst who is required to sit at a desk for eight hours a day and pay attention to boring numbers on a business spreadsheet is much more likely to get diagnosed with ADHD, while a barber that hangs out and does short haircuts while interacting with clients is much less likely to merit such diagnosis.

eranation 8y ago

Exactly. About the game: "To meet the objective, users have to identify different combinations of number strings in missions littered with distraction." About the test (used to prove the game does improve cognitive abilities): "Those tested were asked to detect digit sequences (such as 2-4 or 6-8), and then hit a button once detected as quickly as they could — and multiple sequences could appear at the same time" This feels like a paid article to promote the Peak app. No science here unfortunately. I don't understand how public funds pay for studies like this.

Bartweiss 8y ago

Dual N-back is currently the premiere brain-training task, by which I mean it's the only one that's ever looked at all promising for general benefit. Unfortunately, the newest and best studies suggest that even n-back performance is basically just a training effect on a single test, plus possibly improvement in visual processing. (In layman's terms, your memory doesn't improve, but you do get better at carefully watching for a dot.) This isn't being advertised as more substantive than dual n-back, just more gamified and playable. Given that n-back doesn't seem to work even if you're absolutely diligent about doing it, my expectation for "less rigorous but more playable" is that it will have no value whatsoever. https://www.gwern.net/docs/dnb/2013-colom.pdf https://www.gwern.net/docs/dnb/2013-colom.pdf

systemBuilder 8y ago

While this may be true, I have found with my own son's that the number one most popular soccer skills test (MacDonald test, 1960) is also a good indication of soccer skills and you can train for it. It's a good adjunct to regular participation in a recreational soccer program. My sons achieved competitive A and B league performance (nerd kids, sons of a PhD computer scientists).