10 ms·

First physiological test for autism proves high accuracy in second trial

159 points by just_saying_it 8y ago

wu-ikkyu 8y ago
Perhaps I missed it, but I did not see anything about controlling for the potential effects of those persons with ASD having taken drugs to mitigate symptoms. Couldn't the biochemical effects of ASD drugs themselves be the primary indicating factor? I've seen studies on biochemical indicators for schizophrenia also make this same mistake.
- himom 8y ago
  No, read the source article. The validation data are taken at baseline from three previously published studies investigating pharmaceutical interventions to normalize metabolic abnormalities of children with ASD31: (1) a combination of methylcobalamin and low dose folinic acid32, 33 (2) high dose folinic acid,34 and (3) sapropternin.35 Given that these studies all focused on evaluating treatment strategies for ASD, all participants had a confirmed diagnosis of ASD. FOCM/TS metabolites were available for 154 (76% male) participants with ASD with a mean age of 8.8 years (range 2–17 years). These ages are different than reported by Delhey et al.34 because this study only required that measurements be available at baseline, rather than both at baseline as well as the conclusion of the treatment phase. Furthermore, stratifying patients by age or gender did not reveal any differences in the univariate metabolite distributions. The first two studies were approved by the IRB at the University of Arkansas for Medical Sciences and the third study was approved by the IRB at the University of Texas Health Science Center at Houston. All parents gave written, signed consent and patients provided assent when appropriate. 2.3 Metabolites The metabolites under investigation are presented in Table 1 and additional details of these measurements and derivations are presented in Melnyk, et al.30 This is only a subset of the measurements investigated previously29 because “% DNA methylation” and “8‐OHG” were absent from the validation data set and were therefore removed from this study to ensure that a consistent set of metabolites are used for training and testing. (Table 1 - 22 metabolic variables considered) https://onlinelibrary.wiley.com/doi/full/10.1002/btm2.10095 https://onlinelibrary.wiley.com/doi/full/10.1002/btm2.10095 I’m curious if these metabolic abnormalities apply to adults as well. I had an ADOS at the UCD MIND Institute come back with a 7 (1-14 scale IIRC) about 10 years ago. PS: PKU treatment sapropterin (Kuvan) costs $100k USD/yr
scythe 8y ago
Very surprising result. I don't think I've ever heard of a blood test for a mental disorder -- at least, not one that actually works.
- jmcgough 8y ago
  Be careful about that kind of wording. I and friends who're on the spectrum don't consider it a "mental disorder". It's simply a physiological difference, one that theoretically can be tested for.
  - cosmojg 8y ago
    Doesn't that reasoning apply to any disorder? Medically, a disorder is simply "a disruption of normal physical or mental functions." So, having any disorder makes one abnormal, unique, special, and possibly above average or below average, but certainly not average. "Disorder" and "special physiological difference" are synonyms.
    - arkades 8y ago
      “Disorder” is explicitly pejorative. I’m not sure where I stand wrt the position that autism is just a non-inferior neurovariant, but let’s not pretend we don’t understand the difference between asserting it’s a value-neutral variant and a disease.
      - throwawaymath 8y ago
        No, "disorder" is explicitly clinical. The words "retarded", "dumb", and "stupid" are pejorative. Calling autism a disorder is an accurate paychiatric description, not a value judgement designed to insult a person's identity. An insult can be targeted at a person's autism, but not simply by describing it according to clinical terminology. I sympathize with people who don't believe autism should be considered a disorder, but that doesn't mean discussion needs to be encumbered by walking on eggshells. In particular, admonishing someone for using established clinical terminology is unproductive.
amelius 8y ago
Does this also work in reverse? I.e. take supplements/drugs until the test is negative, and then be (temporarily) autism-free?
- smt88 8y ago
  I don't know, but there's preliminary evidence that autism can be reversed by dosing children with oxytocin and by using TMS on adults. Because those approaches are considered to be fairly low-risk, I suspect research will continue in those directions until they hit a wall of some kind.
  - magic_beans 8y ago
    What does it even mean to “reverse autism”?
    - olh 8y ago
      it is like moonwalk but more autistic /s
    - smt88 8y ago
      Most famous example: https://well.blogs.nytimes.com/2016/03/18/an-experimental-autism-treatment-cost-me-my-marriage/ https://well.blogs.nytimes.com/2016/03/18/an-experimental-au... It allows people to read faces and emotional context in a way they couldn't before. Some people find that their decision-making becomes less logical and more emotional. For others, nothing changes. If you search "tms autism" there are other firsthand accounts that you might find interesting.
      - amelius 8y ago
        Have there been any scientific experiments already to validate this method?
        smt88 8y ago
        As of 2 years ago, there were more than a dozen clinical trials. Evidence is preliminary -- populations are small, and measuring how autistic someone is is not easy. https://www.autismspeaks.org/blog/2016/03/25/transcranial-magnetic-stimulation-autism-evidence-benefit https://www.autismspeaks.org/blog/2016/03/25/transcranial-ma...
amelius 8y ago
> Hahn said the difference between the original accuracy rate and that of the new study can likely be attributed to several factors, the most important being that two of the metabolites were unavailable in the second dataset. Each of the two metabolites had been strong indicators in the previous study. Another issue could be overfitting.
- BadCookie 8y ago
  It sounds like they were using leave-one-out cross validation. My machine learning is a bit rusty, but I don't think that overfitting is the issue.
bcatanzaro 8y ago
If this test becomes widespread, I'd love to take it. I bet many of us in the tech industry have some sort of autism markers. It is a spectrum, after all.
- sidlls 8y ago
  I doubt most (vast majority) have more than a statistically meaningless set of any hypothetical autism markers. A predilection for technology, math or science and social immaturity are commonly identified items techies use to contend they "may" be on the spectrum, but the reality is even mild autism includes so much more abnormality than that.
  - wst_ 8y ago
    And yet tech hubs are crawling with people who are on the spectrum, and have autistic children. Of course not everyone on the spectrum works in tech industry and not working in tech is on the spectrum, but correlation can be seen.
- hosh 8y ago
  There is also this recently published work: http://stm.sciencemag.org/content/9/393/eaag2882 http://stm.sciencemag.org/content/9/393/eaag2882 This one uses machine learning and brain scans to predict the diagnoses for someone as young as 6 months old. It also predicts the severity. What's also interesting is how it differentiate between people who are at-risk, but without a diagnoses, and those who are at-risk, and have a diagnoses. I also wonder if there are more to be studied with the Broad Autism Phenotype (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2746421/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2746421/)
hhh 8y ago
This is quite interesting. Being on the spectrum myself, and many of my friends being so as well, it is very important how different everyone on the spectrum truly is. For high functioning people like myself I see Autism more akin to a mental super-power that can be crippled by small things. The difference in thinking is magnificent, and it aids in my success in most things that I do, but is an absolute disaster when it comes to socializing with people near me. In a town of 300 people, there aren't really any tech nerds or otherwise. Which makes socializing awkward since technology and game are a massive portion of my life. Even to just go to the doctor the interaction is anxiety inducing that my heart rate can raise by a good amount. (I've seen 133 on a bad day, when normally I hang out around 100.) Having to rely on analysis of people and memory of what people think is acceptable is absolutely exhausting. Having a clear test for autism is wonderful, but the spectrum is of infinite width and infinite height.
- deleted 8y ago
  [deleted]
- TheSpiceIsLife 8y ago
  > spectrum is of infinite width and infinite height If this were true, everyone is diagnosable.
  - hhh 8y ago
    It was more for the aesthetic of the sentence, but if you have a spectrum including 'normalcy' that is infinite width and infinite height, with x being ability (a combined measurement of communication ability, sensitivities, etc) and y being level of function, it is possible to include 'normalcy' as it is defined at the farthest top right corner. This could even have more dimensions to it, represented as 0-1 on each. If we're creating a diagnosis of a spectrum that directly affects function, does it not make sense for the general acceptance of normalcy to be represented as either all 1's or a number in-between? If being a savant gives someone greater individual knowledge, should this not be represented in a diagnosis of autism? Do they not instead become above that of normalcy by having this increased ability for specialized knowledge? This almost extends the idea of the spectrum into an extensive representation of a human's ability to function. With the idea of the spectrum being so wide, having a system to classify a human's ability to function over certain categories can aid in the ability to define normalcy and otherwise. The more pieces of information we have in this essentially matrix allows us to have a clear-cut understanding of what most people fall under. To limit this to 3-dimensional space helps as humans to visualize this, but to limit a tool to what we can visualize can hamper its ability. (I am not a doctor)
robrenaud 8y ago
Statistical reporting is often pretty frustrating. > “We are able to predict with 88 percent accuracy whether children have autism,” said Juergen Hahn, lead author > It is estimated that approximately 1.7 percent of all children are diagnosed with ASD def has_autism(child): return False apparently has 98% accuracy.
- vorotato 8y ago
  I would hope that there's more rigor than that. I mean if you're willing to be that dishonest with your sample selection you could select only people who don't have autism and get 100% accuracy.
- icegreentea2 8y ago
  It's 88% correct classification on a validation data set consisting of 154 patients diagnosed with ASD. Their validation dataset did not contain a "typically developing" population since there's only one dataset for that data and it was consumed in their model generation. Their training data resulted in a 5% misclassification rate of their controls - that is 5% of patients that were not diagnosed with ASD in current tests would be classified as having ASD in this scheme. The full paper is here: https://onlinelibrary.wiley.com/doi/10.1002/btm2.10095 https://onlinelibrary.wiley.com/doi/10.1002/btm2.10095
  - perl4ever 8y ago
    If you give the test to everyone, your results are: ~4.9% incorrectly diagnosed with autism (0.05 x .983) ~1.5% correctly diagnosed with autism (0.88 x 0.017) Thus, 76% of those diagnosed would be diagnosed incorrectly.
    - alexandercrohde 8y ago
      That's true in a statistical sense. Or it's possible that the human-observers are less accurate about autism than this brain scan. It's unclear if those who have the odd brain structure but aren't diagnosed are entirely asymptomatic. Also they may be non-autistic in spite of this brain pattern, but nonetheless have a genetic predisposition to pass it on. Big step.
perl4ever 8y ago
Does nobody care about specificity?
- deleted 8y ago
  [deleted]
jeffdavis 8y ago
Isn't it hard to define the accuracy without knowing more about the condition?
- timb07 8y ago
  The generally accepted definition of autism spectrum disorder is that in DSM-V. This physiological test is compared against ASD diagnoses based on DSM-V, and based on that comparison, the accuracy of the test (for the given sample tested) is calculated. As more is known about the condition, the diagnostic criteria listed in the DSM could be updated, which would mean that this test would then need to be evaluated again. But that doesn't prevent its current accuracy being measured.

Perhaps I missed it, but I did not see anything about controlling for the potential effects of those persons with ASD having taken drugs to mitigate symptoms. Couldn't the biochemical effects of ASD drugs themselves be the primary indicating factor? I've seen studies on biochemical indicators for schizophrenia also make this same mistake.

himom 8y ago
No, read the source article. The validation data are taken at baseline from three previously published studies investigating pharmaceutical interventions to normalize metabolic abnormalities of children with ASD31: (1) a combination of methylcobalamin and low dose folinic acid32, 33 (2) high dose folinic acid,34 and (3) sapropternin.35 Given that these studies all focused on evaluating treatment strategies for ASD, all participants had a confirmed diagnosis of ASD. FOCM/TS metabolites were available for 154 (76% male) participants with ASD with a mean age of 8.8 years (range 2–17 years). These ages are different than reported by Delhey et al.34 because this study only required that measurements be available at baseline, rather than both at baseline as well as the conclusion of the treatment phase. Furthermore, stratifying patients by age or gender did not reveal any differences in the univariate metabolite distributions. The first two studies were approved by the IRB at the University of Arkansas for Medical Sciences and the third study was approved by the IRB at the University of Texas Health Science Center at Houston. All parents gave written, signed consent and patients provided assent when appropriate. 2.3 Metabolites The metabolites under investigation are presented in Table 1 and additional details of these measurements and derivations are presented in Melnyk, et al.30 This is only a subset of the measurements investigated previously29 because “% DNA methylation” and “8‐OHG” were absent from the validation data set and were therefore removed from this study to ensure that a consistent set of metabolites are used for training and testing. (Table 1 - 22 metabolic variables considered) https://onlinelibrary.wiley.com/doi/full/10.1002/btm2.10095 https://onlinelibrary.wiley.com/doi/full/10.1002/btm2.10095 I’m curious if these metabolic abnormalities apply to adults as well. I had an ADOS at the UCD MIND Institute come back with a 7 (1-14 scale IIRC) about 10 years ago. PS: PKU treatment sapropterin (Kuvan) costs $100k USD/yr

scythe 8y ago

Very surprising result. I don't think I've ever heard of a blood test for a mental disorder -- at least, not one that actually works.

jmcgough 8y ago
Be careful about that kind of wording. I and friends who're on the spectrum don't consider it a "mental disorder". It's simply a physiological difference, one that theoretically can be tested for.
- cosmojg 8y ago
  Doesn't that reasoning apply to any disorder? Medically, a disorder is simply "a disruption of normal physical or mental functions." So, having any disorder makes one abnormal, unique, special, and possibly above average or below average, but certainly not average. "Disorder" and "special physiological difference" are synonyms.
  - arkades 8y ago
    “Disorder” is explicitly pejorative. I’m not sure where I stand wrt the position that autism is just a non-inferior neurovariant, but let’s not pretend we don’t understand the difference between asserting it’s a value-neutral variant and a disease.
    - throwawaymath 8y ago
      No, "disorder" is explicitly clinical. The words "retarded", "dumb", and "stupid" are pejorative. Calling autism a disorder is an accurate paychiatric description, not a value judgement designed to insult a person's identity. An insult can be targeted at a person's autism, but not simply by describing it according to clinical terminology. I sympathize with people who don't believe autism should be considered a disorder, but that doesn't mean discussion needs to be encumbered by walking on eggshells. In particular, admonishing someone for using established clinical terminology is unproductive.

amelius 8y ago

Does this also work in reverse? I.e. take supplements/drugs until the test is negative, and then be (temporarily) autism-free?

smt88 8y ago
I don't know, but there's preliminary evidence that autism can be reversed by dosing children with oxytocin and by using TMS on adults. Because those approaches are considered to be fairly low-risk, I suspect research will continue in those directions until they hit a wall of some kind.
- magic_beans 8y ago
  What does it even mean to “reverse autism”?
  - olh 8y ago
    it is like moonwalk but more autistic /s
  - smt88 8y ago
    Most famous example: https://well.blogs.nytimes.com/2016/03/18/an-experimental-autism-treatment-cost-me-my-marriage/ https://well.blogs.nytimes.com/2016/03/18/an-experimental-au... It allows people to read faces and emotional context in a way they couldn't before. Some people find that their decision-making becomes less logical and more emotional. For others, nothing changes. If you search "tms autism" there are other firsthand accounts that you might find interesting.
    - amelius 8y ago
      Have there been any scientific experiments already to validate this method?
      - smt88 8y ago
        As of 2 years ago, there were more than a dozen clinical trials. Evidence is preliminary -- populations are small, and measuring how autistic someone is is not easy. https://www.autismspeaks.org/blog/2016/03/25/transcranial-magnetic-stimulation-autism-evidence-benefit https://www.autismspeaks.org/blog/2016/03/25/transcranial-ma...

amelius 8y ago

> Hahn said the difference between the original accuracy rate and that of the new study can likely be attributed to several factors, the most important being that two of the metabolites were unavailable in the second dataset. Each of the two metabolites had been strong indicators in the previous study. Another issue could be overfitting.

BadCookie 8y ago
It sounds like they were using leave-one-out cross validation. My machine learning is a bit rusty, but I don't think that overfitting is the issue.

bcatanzaro 8y ago

If this test becomes widespread, I'd love to take it. I bet many of us in the tech industry have some sort of autism markers. It is a spectrum, after all.

sidlls 8y ago
I doubt most (vast majority) have more than a statistically meaningless set of any hypothetical autism markers. A predilection for technology, math or science and social immaturity are commonly identified items techies use to contend they "may" be on the spectrum, but the reality is even mild autism includes so much more abnormality than that.
- wst_ 8y ago
  And yet tech hubs are crawling with people who are on the spectrum, and have autistic children. Of course not everyone on the spectrum works in tech industry and not working in tech is on the spectrum, but correlation can be seen.
hosh 8y ago
There is also this recently published work: http://stm.sciencemag.org/content/9/393/eaag2882 http://stm.sciencemag.org/content/9/393/eaag2882 This one uses machine learning and brain scans to predict the diagnoses for someone as young as 6 months old. It also predicts the severity. What's also interesting is how it differentiate between people who are at-risk, but without a diagnoses, and those who are at-risk, and have a diagnoses. I also wonder if there are more to be studied with the Broad Autism Phenotype (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2746421/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2746421/)

hhh 8y ago

This is quite interesting. Being on the spectrum myself, and many of my friends being so as well, it is very important how different everyone on the spectrum truly is. For high functioning people like myself I see Autism more akin to a mental super-power that can be crippled by small things. The difference in thinking is magnificent, and it aids in my success in most things that I do, but is an absolute disaster when it comes to socializing with people near me. In a town of 300 people, there aren't really any tech nerds or otherwise. Which makes socializing awkward since technology and game are a massive portion of my life. Even to just go to the doctor the interaction is anxiety inducing that my heart rate can raise by a good amount. (I've seen 133 on a bad day, when normally I hang out around 100.) Having to rely on analysis of people and memory of what people think is acceptable is absolutely exhausting. Having a clear test for autism is wonderful, but the spectrum is of infinite width and infinite height.

deleted 8y ago
[deleted]
TheSpiceIsLife 8y ago
> spectrum is of infinite width and infinite height If this were true, everyone is diagnosable.
- hhh 8y ago
  It was more for the aesthetic of the sentence, but if you have a spectrum including 'normalcy' that is infinite width and infinite height, with x being ability (a combined measurement of communication ability, sensitivities, etc) and y being level of function, it is possible to include 'normalcy' as it is defined at the farthest top right corner. This could even have more dimensions to it, represented as 0-1 on each. If we're creating a diagnosis of a spectrum that directly affects function, does it not make sense for the general acceptance of normalcy to be represented as either all 1's or a number in-between? If being a savant gives someone greater individual knowledge, should this not be represented in a diagnosis of autism? Do they not instead become above that of normalcy by having this increased ability for specialized knowledge? This almost extends the idea of the spectrum into an extensive representation of a human's ability to function. With the idea of the spectrum being so wide, having a system to classify a human's ability to function over certain categories can aid in the ability to define normalcy and otherwise. The more pieces of information we have in this essentially matrix allows us to have a clear-cut understanding of what most people fall under. To limit this to 3-dimensional space helps as humans to visualize this, but to limit a tool to what we can visualize can hamper its ability. (I am not a doctor)

robrenaud 8y ago

Statistical reporting is often pretty frustrating. > “We are able to predict with 88 percent accuracy whether children have autism,” said Juergen Hahn, lead author > It is estimated that approximately 1.7 percent of all children are diagnosed with ASD def has_autism(child): return False apparently has 98% accuracy.

vorotato 8y ago
I would hope that there's more rigor than that. I mean if you're willing to be that dishonest with your sample selection you could select only people who don't have autism and get 100% accuracy.
icegreentea2 8y ago
It's 88% correct classification on a validation data set consisting of 154 patients diagnosed with ASD. Their validation dataset did not contain a "typically developing" population since there's only one dataset for that data and it was consumed in their model generation. Their training data resulted in a 5% misclassification rate of their controls - that is 5% of patients that were not diagnosed with ASD in current tests would be classified as having ASD in this scheme. The full paper is here: https://onlinelibrary.wiley.com/doi/10.1002/btm2.10095 https://onlinelibrary.wiley.com/doi/10.1002/btm2.10095
- perl4ever 8y ago
  If you give the test to everyone, your results are: ~4.9% incorrectly diagnosed with autism (0.05 x .983) ~1.5% correctly diagnosed with autism (0.88 x 0.017) Thus, 76% of those diagnosed would be diagnosed incorrectly.
  - alexandercrohde 8y ago
    That's true in a statistical sense. Or it's possible that the human-observers are less accurate about autism than this brain scan. It's unclear if those who have the odd brain structure but aren't diagnosed are entirely asymptomatic. Also they may be non-autistic in spite of this brain pattern, but nonetheless have a genetic predisposition to pass it on. Big step.

perl4ever 8y ago

Does nobody care about specificity?

deleted 8y ago
[deleted]

jeffdavis 8y ago

Isn't it hard to define the accuracy without knowing more about the condition?

timb07 8y ago
The generally accepted definition of autism spectrum disorder is that in DSM-V. This physiological test is compared against ASD diagnoses based on DSM-V, and based on that comparison, the accuracy of the test (for the given sample tested) is calculated. As more is known about the condition, the diagnostic criteria listed in the DSM could be updated, which would mean that this test would then need to be evaluated again. But that doesn't prevent its current accuracy being measured.