5 ms·

5-HTTLPR: A Pointed Review

119 points by gbear605 7y ago

michaelbuckbee 7y ago
Does everyone remember Theranos? Aka the billion dollar company built on the shaky basis of being able to do a full blood test on demand from a single drop of your blood? The first 2/3 of this article describe the unstable foundation of genetic testing for depression and the last 1/3 the frightening consequences. It really starts at "Third, antidepressant pharmacogenomic testing." and ends with this brutal conclusion: "But I think we should take a second to remember that yes, this is really bad. That this is a rare case where methodological improvements allowed a conclusive test of a popular hypothesis, and it failed badly. How many other cases like this are there, where there’s no geneticist with a 600,000 person sample size to check if it’s true or not? How many of our scientific edifices are built on air? How many useless products are out there under the guise of good science? We still don’t know."
gioele 7y ago
«I love this paper because it is ruthless. The authors know exactly what they are doing, and they are clearly enjoying every second of it. They explain that given what we now know about polygenicity, the highest-effect-size genes require samples of about 34,000 people to detect, and so any study with fewer than 34,000 people that says anything about specific genes is almost definitely a false positive; they go on to show that the median sample size for previous studies in this area was 345.»
ummonk 7y ago
>They explain that given what we now know about polygenicity, the highest-effect-size genes require samples of about 34,000 people to detect, and so any study with fewer than 34,000 people that says anything about specific genes is almost definitely a false positive; they go on to show that the median sample size for previous studies in this area was 345. This is a misleading paraphrasing of what the study said. There are plenty of effects which can be easily demonstrated with smaller sample sizes; it's just not complex effects like depression. What the authors said: >genome-wide association studies (GWASs), which agnostically examine associations at millions of common single-nucleotide polymorphisms (SNPs) across the genome in large samples, have consistently found that individual SNPs exert small effects on genetically complex traits such as depression (18–20). For example, in the most recent GWAS of depression, which utilized a sample of 135,458 case subjects and 344,901 control subjects, the strongest individual signal detected (rs12552; odds ra- tio=1.044, p=6.07310219) would require a sample of ap- proximately 34,100 individuals to be detected with 80% power at an alpha level of 0.05, assuming a balanced case- control design (18).
- abecedarius 7y ago
  FWIW, one of the authors also said "I have never in my career read a synopsis of a paper I've (co-)written that is better than the original paper. Until now." https://twitter.com/matthewckeller/status/1126380891243188224 https://twitter.com/matthewckeller/status/112638089124318822...
outlace 7y ago
I wonder if this has anything to do with the problem with multiple comparisons. In a GWAS since you're comparing possibly thousands of genes against controls, the probability of getting a false positive is higher just by the multiple tests you're running. So you have to correct for this often by dividing the chosen p-value threshold (usually 0.05) by the number of comparisons (this is the simple Bonferonni correction). If you looked at 10,000 genes your new significance threshold would be 0.05/10000=0.000005. Therefore any gene difference between groups must have a p-value of less than 0.000005 to be considered significant if you're testing that many genes. However, if you chose to only look at 5-HTTLPR, then you don't need this correction and you only need p < 0.05.
zby 7y ago
Related: https://library.mpib-berlin.mpg.de/ft/gg/GG_Null_2004.pdf https://library.mpib-berlin.mpg.de/ft/gg/GG_Null_2004.pdf "The Null Ritual What You Always Wanted to Know About Significance Testing but Were Afraid to Ask"
tlynchpin 7y ago
came here for this in the comments of TFA: >In retrospect, it should have been obvious that hypertext transport over line printers just wasn’t a good idea.

119 points by gbear605 7y ago

Does everyone remember Theranos? Aka the billion dollar company built on the shaky basis of being able to do a full blood test on demand from a single drop of your blood? The first 2/3 of this article describe the unstable foundation of genetic testing for depression and the last 1/3 the frightening consequences. It really starts at "Third, antidepressant pharmacogenomic testing." and ends with this brutal conclusion: "But I think we should take a second to remember that yes, this is really bad. That this is a rare case where methodological improvements allowed a conclusive test of a popular hypothesis, and it failed badly. How many other cases like this are there, where there’s no geneticist with a 600,000 person sample size to check if it’s true or not? How many of our scientific edifices are built on air? How many useless products are out there under the guise of good science? We still don’t know."

gioele 7y ago

«I love this paper because it is ruthless. The authors know exactly what they are doing, and they are clearly enjoying every second of it. They explain that given what we now know about polygenicity, the highest-effect-size genes require samples of about 34,000 people to detect, and so any study with fewer than 34,000 people that says anything about specific genes is almost definitely a false positive; they go on to show that the median sample size for previous studies in this area was 345.»

ummonk 7y ago

>They explain that given what we now know about polygenicity, the highest-effect-size genes require samples of about 34,000 people to detect, and so any study with fewer than 34,000 people that says anything about specific genes is almost definitely a false positive; they go on to show that the median sample size for previous studies in this area was 345. This is a misleading paraphrasing of what the study said. There are plenty of effects which can be easily demonstrated with smaller sample sizes; it's just not complex effects like depression. What the authors said: >genome-wide association studies (GWASs), which agnostically examine associations at millions of common single-nucleotide polymorphisms (SNPs) across the genome in large samples, have consistently found that individual SNPs exert small effects on genetically complex traits such as depression (18–20). For example, in the most recent GWAS of depression, which utilized a sample of 135,458 case subjects and 344,901 control subjects, the strongest individual signal detected (rs12552; odds ra- tio=1.044, p=6.07310219) would require a sample of ap- proximately 34,100 individuals to be detected with 80% power at an alpha level of 0.05, assuming a balanced case- control design (18).

abecedarius 7y ago
FWIW, one of the authors also said "I have never in my career read a synopsis of a paper I've (co-)written that is better than the original paper. Until now." https://twitter.com/matthewckeller/status/1126380891243188224 https://twitter.com/matthewckeller/status/112638089124318822...

outlace 7y ago

I wonder if this has anything to do with the problem with multiple comparisons. In a GWAS since you're comparing possibly thousands of genes against controls, the probability of getting a false positive is higher just by the multiple tests you're running. So you have to correct for this often by dividing the chosen p-value threshold (usually 0.05) by the number of comparisons (this is the simple Bonferonni correction). If you looked at 10,000 genes your new significance threshold would be 0.05/10000=0.000005. Therefore any gene difference between groups must have a p-value of less than 0.000005 to be considered significant if you're testing that many genes. However, if you chose to only look at 5-HTTLPR, then you don't need this correction and you only need p < 0.05.

zby 7y ago

Related: https://library.mpib-berlin.mpg.de/ft/gg/GG_Null_2004.pdf https://library.mpib-berlin.mpg.de/ft/gg/GG_Null_2004.pdf "The Null Ritual What You Always Wanted to Know About Significance Testing but Were Afraid to Ask"

tlynchpin 7y ago

came here for this in the comments of TFA: >In retrospect, it should have been obvious that hypertext transport over line printers just wasn’t a good idea.