11 ms·

Depression-Gene Studies Were Built on Shaky Foundations

138 points by wyclif 7y ago

hairytrog 7y ago
This happens when you have a boatload of "educated" people with degrees in psychology/neuroscience who have to do "research." There's something like 100,000 new psych grads each year - and there's only so many coffee shops.
- dang 7y ago
  Please don't post shallow dismissals to HN, and please don't snark. These are two of the site guidelines. If you'd read them, and follow them when commenting here, we'd be grateful. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
scribu 7y ago
The Slatestar Codex blog post linked in the article is a much more entertaining read, I think: https://slatestarcodex.com/2019/05/07/5-httlpr-a-pointed-review/ https://slatestarcodex.com/2019/05/07/5-httlpr-a-pointed-rev...
- matt4077 7y ago
  Scott is always a good read, and rarely not entertaining. The Atlantic’s piece is a bit less snarky, but has other redeeming factors. It is quite obvious when comparing the two that a journalist’s first instinct is always to call a bunch of experts and incorporate their views. It’s also laudable that the author rejects the easy cynicism of accusing these scientists of individual intentional deception, and instead redirects our scorn onto the publish-or-perish dynamic.
- deleted 7y ago
  [deleted]
emily2749 7y ago
Even you can't believe there is a hen in world's biggest animals See the video how they looks and what they do https://cutt.ly/1rOU7Q https://cutt.ly/1rOU7Q Very funny dog using toilet Try to control your laugh http://bit.ly/2LVSNSG http://bit.ly/2LVSNSG Cute dog doing exercise like humans http://bit.ly/2Jq3rz1 http://bit.ly/2Jq3rz1 Funny gorilla acting like humans see his video on link below http://bit.ly/2JmTNxr http://bit.ly/2JmTNxr Best way to get rid from dark spots on face just follow this https://zoomtips.blogspot.com/2019/05/Dark-spots.html https://zoomtips.blogspot.com/2019/05/Dark-spots.html Funny panda is playing with friends they are sho cute http://bit.ly/2JvDQF4 http://bit.ly/2JvDQF4
AstralStorm 7y ago
Statistics. Garbage in, garbage out. This shows why you have to explicitly quantify statistical power and error model for your study design. Tools are good, but not good enough to do it for you. Then, do calculate effect size instead of a binary answer. You will get answer with a digit in fifth place with a proper size unbiased sample. Note that for proper genome study, the sample size for observational studies is in tend of thousands barring inbred model studies. (Mice or men, with easily detectable disease process.)
- wallace_f 7y ago
  It's amazing how convincing statistics are--even for supposed experts--and not just for the reason you elaborate on. In 2019, they really are the best way to lie.
  - lm28469 7y ago
    Science became a new religion. Instead of saying "shut up god made it that way" you say "shut up science says it's that way". You can find hundred of studies to support your arguments, no matter what they are, it's even easier if you don't actually read the studies and stop at the summaries made by popular medias.
    - ekianjo 7y ago
      what you refer to is not Science,its pseudoscience. Pretty much any crap gets published these days, review standards are abysmal and there is hardly any replication. Only thru replication can you actually claim there is Science.
      - coldtea 7y ago
        >what you refer to is not Science,its pseudoscience. Pretty much any crap gets published these days, review standards are abysmal and there is hardly any replication. That's like the argument that USSR was not real communism, etc. At some point, science is as science does. There's no some holier, better checked, domain of practice. It is what it is, and it sometimes has replication, more often than not it doesn't.
raverbashing 7y ago
It's just naive to pin something as complex as depression on a single variable (but that's Popperian scientificism for you) Now it might be that the 1st study identified an effect on a (very specific) subpopulation and in the bigger study case that subpopulation is not present or not identifiable. But who knows Now of course it's not wrong to study a certain gene, but to go "all in on" a very narrow study path is stupid. Now, if on the course of the depression studies they found this gene had everything to do with a different disease that would be "p-hacking" or something, right? RIGHT? More statistical power and "rigour" won't lead research anywhere because it's not the studies that were necessarily flawed, it's that the effects might be small, or dependent on a chain of other factors, so unless the effect of a single factor is predominant it might not even matter (as an isolated factor).
- raxxorrax 7y ago
  > that's Popperian scientificism I believe his criticisms would be very close to yours. For me, this is on the border between medicine, anthropology and in general social sciences. I wouldn't list hubris as a strength for anyone in these fields to underline popular prejudices. But aside from that, even if statistics are vigorously checked for correctness, it still leaves a lot of room to get different interpretations.
- fromthestart 7y ago
  >It's just naive to pin something as complex as depression on a single variable Why? There are other complex diseases with purely genetic causes, e.g. Huntington's Disease; depression isn't necessarily different. Hindsight is 20/20.
  - thanatropism 7y ago
    Yes, but Huntington's is an explicit cluster of symptoms at diagnosis + a specific prognosis. Uppercase D "Depression" is a diagnosis built around a single symptom, lowercase d "depression". But many people experience depression, both in "core mental illnesses" like bipolar and outside. (Note how bipolar is much better constructed -- indeed the method of differential diagnosis was invented by Kraepelin to differentiate bipolar from schizophrenia) If it was up to me, big D Depression would be redefined as "responds to Prozac" or something.
unityByFreedom 7y ago
Odd that this article cites Scott Alexander as if that is the name of a real psychiatrist, > “What bothers me isn’t just that people said [the gene] mattered and it didn’t,” wrote the psychiatrist Scott Alexander in a widely shared blog post. Scott Alexander is the pen name of a person running a blog who claims to be a psychiatrist, and AFAIK, his authenticity is not proven due to his anonymity.
- Lazare 7y ago
  His identity is hardly a deep secret; anyone who really cares can find it out, and verify his credentials. (Not that those credentials are really relevant here, no?)
  - atomical 7y ago
    I care about reading something written by a professional instead of a Tim Ferriss type. I've found several suspect items in his blog articles. I would like to know his credentials.
    - kerbalspacepro 7y ago
      You can look them up though? It's not difficult?
      - atomical 7y ago
        Where?
  - unityByFreedom 7y ago
    The blog actively misleads readers into thinking Scott Alexander is his real name, > SSC is the project of Scott Alexander, a psychiatrist on the US West Coast. You can email him at scott[at]slatestarcodex[dot]com. Note that emailing bloggers who say they are psychiatrists is a bad way to deal with your psychiatric emergencies, and you might wish to consider talking to your doctor or going to a hospital instead. [1] Journalists citing that blog should note it is an alias, and deeply question whether or not he really is a psychiatrist. And no, I don't find it easy to verify his identity, and yes, it is completely relevant here as he's being cited as a real psychiatrist. That's not verifiable without knowing a real name. [1] https://slatestarcodex.com/about/ https://slatestarcodex.com/about/
gnoppa 7y ago
Now please let the same epiphany come about cancer research or personalized medicine based on genes.
- icegreentea2 7y ago
  Let's not throw the baby out with the bath water. No doubt there's a lot of sketchy stuff going on, but also the fields are very wide with lots of "verticals" going on. Just cause one vertical might be built on a shakey foundation, doesn't mean all of them are (and the converse is also true).
  - gnoppa 7y ago
    You might be right that not everything in those fields is bad, but most is. Nevertheless, what is so terrible is that all the money is put into research (gene mutation causing cancer) even though it has been proven wrong many years ago. That becomes very clear after reading Thomas Seyfried's work (cell fermentation causing cancer). Sadly, academics have the incentives to publish papers in prestigious journals that make them sound smart and not to find a simple cure. Likewise, the free market does not have the incentive to easily and cheaply cure people but to maximze profits. Hence, so much research is absolutely abysmal, wrong, misleading and harmful. Sadly the human mind is constructed in such a way that we self-deceive to gain advantages. That is why there is no need for a conspiracy but just the wrong kind of incentive structure.
    - icegreentea2 7y ago
      I know you're getting downvoted. I don't think things are as "bad" as you think they are, but it's also impossible to deny that our current incentive structure is not ideal and can lead us to bad results. That said, with respect with Seyfried's work, I urge you to give something like https://sciencebasedmedicine.org/ketogenic-diets-for-cancer-hype-versus-science/ https://sciencebasedmedicine.org/ketogenic-diets-for-cancer-... a read. Go ahead and skip all of the bits about his chosen associations and focus on just the science parts if you would like. I think you'll find that quite a lot of research as been done on cancer metabolism, and that Seyfriend's work at best can be considered incomplete. If you find the blog's arguments about lack of clinical trial data to be circular (I guess they kind of are), I don't have any answers for you now, but I suggest you keep on eye out for this guy: https://clinicaltrials.gov/ct2/show/study/NCT01754350 https://clinicaltrials.gov/ct2/show/study/NCT01754350 This is one of the studies in early stages that the blog links to (it's been a few years). They just wrapped up the study a few months ago, and hopefully there will be results posted soon. You can also search for that study number (NCT01754350) or study name (ERGO2) for papers when they do come out (again, expect at least a few more months).
hmd_imputer 7y ago
imagine how many times a valid counter argument was silenced because of the "are you denying science?" bull* argument
thanatropism 7y ago
I think pop sci rags going wild over papers that claim radical breaks are a part of the problem, even if a small one. There's simply no reason that people who are not technical enough to read scientific papers would be up to date on the latest cancer research. People have a right to knowledge? Then open up the journals.
- mattkrause 7y ago
  It’s on a slightly different level. There are strong incentives to find ‘sexy’ results, but this is mostly aimed at getting a paper into Nature, Science, or Cell. Publishing in these journals can have an outsized effect on one’s career, even if the results don’t actually hold up. In contrast, there’s not a huge pay off for getting something into Popular Science or the New York Times science section. Publicity is good and can help show the relevance of your research. It’s also fun to show your mom (who still wants you to go to med school), but people tend not to chase it nearly as hard as a “glam” paper.
  - thanatropism 7y ago
    That's probably a bigger effect. People have a huge trouble accepting that their wonderful science is a social process.
    - rossdavidh 7y ago
      Yes, and also that it's a very economic one. If you need to churn out (and get accepted) research in quantity, then "hold on there, do we know that for sure? let's try to replicate that, with a much bigger sample size..." is downright unwelcome. Scientists are mostly good people, but they're not angels, and if we put them in a system that rewards the wrong thing, we will get the wrong thing.
      - AstralStorm 7y ago
        Unfortunately neither economics nor sociology are good enough science branches to even fix themselves. ;)
searine 7y ago
As someone who works on depression GWAS, this isn't the end of finding causal genes for depression, it's the beginning. In the next few years we're going to see a waterfall of new rare variants linked to disease, all which have a much higher chance of causing functional change. As WGS comes online for association studies it will both validate and more deeply explore the genetic nature of every disease. It is going to be mindblowing.
- eggie 7y ago
  > As WGS comes online for association studies it will both validate and more deeply explore the genetic nature of every disease. It is going to be mindblowing. But there is another layer of confusion, because WGS implies resequencing, and resequencing is only as good as your reference genome, and will distort results when the genome you're inferring is to far from the reference (reference bias). The real mind bend will come when we have thousands or millions of whole genome de novo assemblies and we compare these to each other to do our GWAS. Only then are we going to have a hope of knowing what is actually causal in a genomic sense. Until then we remain in the land of association. In nature, most adaptive variation appears to be large and structural. All the recent studies that have used whole genome assemblies to look at this have found the same thing. I would be surprised if this isn't the case for humans too. If it is, then much of current perspective on GWAS (both based on chips and WGS) will need to be rewritten.
  - asdff 7y ago
    GWAS is nothing more than correlations and by themselves aren't very remarkable. What is remarkable is when someone does a functional analysis on the variant they found in their GWAS and is able to describe the mechanism that it uses to affect the disease or phenotype. The way you do that isn't more sequencing, but with model systems that you can easily manipulate in the lab. GWAS also doesn't compare to an arbitrary reference sequence. Some good numbers for a GWAS are >1000 cases and >2000 controls from the same population. You have to match populations, or else all your association study is going to find is the differences between east asians and europeans, for example. You need a lot of samples to get enough stats power to even see these rare variants.
mindgam3 7y ago
“Nor... should his work be taken to mean that genes don’t affect depression. They do, and with newer, bigger studies, researchers are finally working out which ones do.” This part really needs some context/sources. Anyone know which studies he’s talking about?
- bobcostas55 7y ago
  Here's one: https://www.nature.com/articles/s41593-018-0326-7 https://www.nature.com/articles/s41593-018-0326-7
  - mindgam3 7y ago
    Thanks
DuskStar 7y ago
I love the quote from the Slate Star Codex article: > ...what bothers me isn’t just that people said 5-HTTLPR mattered and it didn’t. It’s that we built whole imaginary edifices, whole castles in the air on top of this idea of 5-HTTLPR mattering. We “figured out” how 5-HTTLPR exerted its effects, what parts of the brain it was active in, what sorts of things it interacted with, how its effects were enhanced or suppressed by the effects of other imaginary depression genes. This isn’t just an explorer coming back from the Orient and claiming there are unicorns there. It’s the explorer describing the life cycle of unicorns, what unicorns eat, all the different subspecies of unicorn, which cuts of unicorn meat are tastiest, and a blow-by-blow account of a wrestling match between unicorns and Bigfoot. https://slatestarcodex.com/2019/05/07/5-httlpr-a-pointed-review/ https://slatestarcodex.com/2019/05/07/5-httlpr-a-pointed-rev...
lisper 7y ago
> Between them, these 18 genes have been the subject of more than 1,000 research papers, on depression alone. And for what? If the new study is right, these genes have nothing to do with depression. The article doesn't actually say anything about what these 1000 research papers actually claimed to show. It's entirely possible that most of them were negative results, there's no way to tell from the text of the article. It strongly implies that this is the case, but it doesn't actually say so. This is a significant omission for an article whose thesis is that these 1000 papers constituted a "house of cards." [UPDATE] "Sometimes the gene was linked to depression; sometimes it wasn’t. And crucially, the better the methods, the less likely he was to see such a link." Seems to me like this is science working exactly as it is supposed to (except for the possible suppression of earlier negative results, but that's a known broad problem that has nothing to do with this particular gene study).
- rossdavidh 7y ago
  It worked as it was supposed to...eventually. Instead of double-checking the initial results, hundreds of research papers by many research teams looked into questions that would be very important, if the genes in question were connected to depression. Sure, it is great that it's not (quite) religious dogma, and it eventually gets revisited. But, a couple decades worth of work and ~1000 research papers, many of which took place AFTER the earlier 2005 paper that called into question the link to depression, is by no means what we should shoot for. More fundamentally, there was nothing in the system that motivated anyone to do replication of the original link. The problem is not that the original link turned out to be spurious, and it's not that anyone did anything unethical. More worrisomely, nobody appears to have done anything unethical, but because of the systemic incentives for research grants, tenure, etc., replication of the foundational result wasn't attempted in a serious enough way until many times that much work had been done on something spurious.
pfdietz 7y ago
What's needed is for the Ignoble Prize people to focus on the people who start this sort of crap, not on humorous low value targets. Really call out the people who contributed negatively to the progress of science.

138 points by wyclif 7y ago

This happens when you have a boatload of "educated" people with degrees in psychology/neuroscience who have to do "research." There's something like 100,000 new psych grads each year - and there's only so many coffee shops.

dang 7y ago
Please don't post shallow dismissals to HN, and please don't snark. These are two of the site guidelines. If you'd read them, and follow them when commenting here, we'd be grateful. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

scribu 7y ago

The Slatestar Codex blog post linked in the article is a much more entertaining read, I think: https://slatestarcodex.com/2019/05/07/5-httlpr-a-pointed-review/ https://slatestarcodex.com/2019/05/07/5-httlpr-a-pointed-rev...

matt4077 7y ago
Scott is always a good read, and rarely not entertaining. The Atlantic’s piece is a bit less snarky, but has other redeeming factors. It is quite obvious when comparing the two that a journalist’s first instinct is always to call a bunch of experts and incorporate their views. It’s also laudable that the author rejects the easy cynicism of accusing these scientists of individual intentional deception, and instead redirects our scorn onto the publish-or-perish dynamic.
deleted 7y ago
[deleted]

emily2749 7y ago

Even you can't believe there is a hen in world's biggest animals See the video how they looks and what they do https://cutt.ly/1rOU7Q https://cutt.ly/1rOU7Q Very funny dog using toilet Try to control your laugh http://bit.ly/2LVSNSG http://bit.ly/2LVSNSG Cute dog doing exercise like humans http://bit.ly/2Jq3rz1 http://bit.ly/2Jq3rz1 Funny gorilla acting like humans see his video on link below http://bit.ly/2JmTNxr http://bit.ly/2JmTNxr Best way to get rid from dark spots on face just follow this https://zoomtips.blogspot.com/2019/05/Dark-spots.html https://zoomtips.blogspot.com/2019/05/Dark-spots.html Funny panda is playing with friends they are sho cute http://bit.ly/2JvDQF4 http://bit.ly/2JvDQF4

AstralStorm 7y ago

Statistics. Garbage in, garbage out. This shows why you have to explicitly quantify statistical power and error model for your study design. Tools are good, but not good enough to do it for you. Then, do calculate effect size instead of a binary answer. You will get answer with a digit in fifth place with a proper size unbiased sample. Note that for proper genome study, the sample size for observational studies is in tend of thousands barring inbred model studies. (Mice or men, with easily detectable disease process.)

wallace_f 7y ago
It's amazing how convincing statistics are--even for supposed experts--and not just for the reason you elaborate on. In 2019, they really are the best way to lie.
- lm28469 7y ago
  Science became a new religion. Instead of saying "shut up god made it that way" you say "shut up science says it's that way". You can find hundred of studies to support your arguments, no matter what they are, it's even easier if you don't actually read the studies and stop at the summaries made by popular medias.
  - ekianjo 7y ago
    what you refer to is not Science,its pseudoscience. Pretty much any crap gets published these days, review standards are abysmal and there is hardly any replication. Only thru replication can you actually claim there is Science.
    - coldtea 7y ago
      >what you refer to is not Science,its pseudoscience. Pretty much any crap gets published these days, review standards are abysmal and there is hardly any replication. That's like the argument that USSR was not real communism, etc. At some point, science is as science does. There's no some holier, better checked, domain of practice. It is what it is, and it sometimes has replication, more often than not it doesn't.

raverbashing 7y ago

It's just naive to pin something as complex as depression on a single variable (but that's Popperian scientificism for you) Now it might be that the 1st study identified an effect on a (very specific) subpopulation and in the bigger study case that subpopulation is not present or not identifiable. But who knows Now of course it's not wrong to study a certain gene, but to go "all in on" a very narrow study path is stupid. Now, if on the course of the depression studies they found this gene had everything to do with a different disease that would be "p-hacking" or something, right? RIGHT? More statistical power and "rigour" won't lead research anywhere because it's not the studies that were necessarily flawed, it's that the effects might be small, or dependent on a chain of other factors, so unless the effect of a single factor is predominant it might not even matter (as an isolated factor).

raxxorrax 7y ago
> that's Popperian scientificism I believe his criticisms would be very close to yours. For me, this is on the border between medicine, anthropology and in general social sciences. I wouldn't list hubris as a strength for anyone in these fields to underline popular prejudices. But aside from that, even if statistics are vigorously checked for correctness, it still leaves a lot of room to get different interpretations.
fromthestart 7y ago
>It's just naive to pin something as complex as depression on a single variable Why? There are other complex diseases with purely genetic causes, e.g. Huntington's Disease; depression isn't necessarily different. Hindsight is 20/20.
- thanatropism 7y ago
  Yes, but Huntington's is an explicit cluster of symptoms at diagnosis + a specific prognosis. Uppercase D "Depression" is a diagnosis built around a single symptom, lowercase d "depression". But many people experience depression, both in "core mental illnesses" like bipolar and outside. (Note how bipolar is much better constructed -- indeed the method of differential diagnosis was invented by Kraepelin to differentiate bipolar from schizophrenia) If it was up to me, big D Depression would be redefined as "responds to Prozac" or something.

unityByFreedom 7y ago

Odd that this article cites Scott Alexander as if that is the name of a real psychiatrist, > “What bothers me isn’t just that people said [the gene] mattered and it didn’t,” wrote the psychiatrist Scott Alexander in a widely shared blog post. Scott Alexander is the pen name of a person running a blog who claims to be a psychiatrist, and AFAIK, his authenticity is not proven due to his anonymity.

Lazare 7y ago
His identity is hardly a deep secret; anyone who really cares can find it out, and verify his credentials. (Not that those credentials are really relevant here, no?)
- atomical 7y ago
  I care about reading something written by a professional instead of a Tim Ferriss type. I've found several suspect items in his blog articles. I would like to know his credentials.
  - kerbalspacepro 7y ago
    You can look them up though? It's not difficult?
    - atomical 7y ago
      Where?
- unityByFreedom 7y ago
  The blog actively misleads readers into thinking Scott Alexander is his real name, > SSC is the project of Scott Alexander, a psychiatrist on the US West Coast. You can email him at scott[at]slatestarcodex[dot]com. Note that emailing bloggers who say they are psychiatrists is a bad way to deal with your psychiatric emergencies, and you might wish to consider talking to your doctor or going to a hospital instead. [1] Journalists citing that blog should note it is an alias, and deeply question whether or not he really is a psychiatrist. And no, I don't find it easy to verify his identity, and yes, it is completely relevant here as he's being cited as a real psychiatrist. That's not verifiable without knowing a real name. [1] https://slatestarcodex.com/about/ https://slatestarcodex.com/about/

gnoppa 7y ago

Now please let the same epiphany come about cancer research or personalized medicine based on genes.

icegreentea2 7y ago
Let's not throw the baby out with the bath water. No doubt there's a lot of sketchy stuff going on, but also the fields are very wide with lots of "verticals" going on. Just cause one vertical might be built on a shakey foundation, doesn't mean all of them are (and the converse is also true).
- gnoppa 7y ago
  You might be right that not everything in those fields is bad, but most is. Nevertheless, what is so terrible is that all the money is put into research (gene mutation causing cancer) even though it has been proven wrong many years ago. That becomes very clear after reading Thomas Seyfried's work (cell fermentation causing cancer). Sadly, academics have the incentives to publish papers in prestigious journals that make them sound smart and not to find a simple cure. Likewise, the free market does not have the incentive to easily and cheaply cure people but to maximze profits. Hence, so much research is absolutely abysmal, wrong, misleading and harmful. Sadly the human mind is constructed in such a way that we self-deceive to gain advantages. That is why there is no need for a conspiracy but just the wrong kind of incentive structure.
  - icegreentea2 7y ago
    I know you're getting downvoted. I don't think things are as "bad" as you think they are, but it's also impossible to deny that our current incentive structure is not ideal and can lead us to bad results. That said, with respect with Seyfried's work, I urge you to give something like https://sciencebasedmedicine.org/ketogenic-diets-for-cancer-hype-versus-science/ https://sciencebasedmedicine.org/ketogenic-diets-for-cancer-... a read. Go ahead and skip all of the bits about his chosen associations and focus on just the science parts if you would like. I think you'll find that quite a lot of research as been done on cancer metabolism, and that Seyfriend's work at best can be considered incomplete. If you find the blog's arguments about lack of clinical trial data to be circular (I guess they kind of are), I don't have any answers for you now, but I suggest you keep on eye out for this guy: https://clinicaltrials.gov/ct2/show/study/NCT01754350 https://clinicaltrials.gov/ct2/show/study/NCT01754350 This is one of the studies in early stages that the blog links to (it's been a few years). They just wrapped up the study a few months ago, and hopefully there will be results posted soon. You can also search for that study number (NCT01754350) or study name (ERGO2) for papers when they do come out (again, expect at least a few more months).

hmd_imputer 7y ago

imagine how many times a valid counter argument was silenced because of the "are you denying science?" bull* argument

thanatropism 7y ago

I think pop sci rags going wild over papers that claim radical breaks are a part of the problem, even if a small one. There's simply no reason that people who are not technical enough to read scientific papers would be up to date on the latest cancer research. People have a right to knowledge? Then open up the journals.

mattkrause 7y ago
It’s on a slightly different level. There are strong incentives to find ‘sexy’ results, but this is mostly aimed at getting a paper into Nature, Science, or Cell. Publishing in these journals can have an outsized effect on one’s career, even if the results don’t actually hold up. In contrast, there’s not a huge pay off for getting something into Popular Science or the New York Times science section. Publicity is good and can help show the relevance of your research. It’s also fun to show your mom (who still wants you to go to med school), but people tend not to chase it nearly as hard as a “glam” paper.
- thanatropism 7y ago
  That's probably a bigger effect. People have a huge trouble accepting that their wonderful science is a social process.
  - rossdavidh 7y ago
    Yes, and also that it's a very economic one. If you need to churn out (and get accepted) research in quantity, then "hold on there, do we know that for sure? let's try to replicate that, with a much bigger sample size..." is downright unwelcome. Scientists are mostly good people, but they're not angels, and if we put them in a system that rewards the wrong thing, we will get the wrong thing.
    - AstralStorm 7y ago
      Unfortunately neither economics nor sociology are good enough science branches to even fix themselves. ;)

searine 7y ago

As someone who works on depression GWAS, this isn't the end of finding causal genes for depression, it's the beginning. In the next few years we're going to see a waterfall of new rare variants linked to disease, all which have a much higher chance of causing functional change. As WGS comes online for association studies it will both validate and more deeply explore the genetic nature of every disease. It is going to be mindblowing.

eggie 7y ago
> As WGS comes online for association studies it will both validate and more deeply explore the genetic nature of every disease. It is going to be mindblowing. But there is another layer of confusion, because WGS implies resequencing, and resequencing is only as good as your reference genome, and will distort results when the genome you're inferring is to far from the reference (reference bias). The real mind bend will come when we have thousands or millions of whole genome de novo assemblies and we compare these to each other to do our GWAS. Only then are we going to have a hope of knowing what is actually causal in a genomic sense. Until then we remain in the land of association. In nature, most adaptive variation appears to be large and structural. All the recent studies that have used whole genome assemblies to look at this have found the same thing. I would be surprised if this isn't the case for humans too. If it is, then much of current perspective on GWAS (both based on chips and WGS) will need to be rewritten.
- asdff 7y ago
  GWAS is nothing more than correlations and by themselves aren't very remarkable. What is remarkable is when someone does a functional analysis on the variant they found in their GWAS and is able to describe the mechanism that it uses to affect the disease or phenotype. The way you do that isn't more sequencing, but with model systems that you can easily manipulate in the lab. GWAS also doesn't compare to an arbitrary reference sequence. Some good numbers for a GWAS are >1000 cases and >2000 controls from the same population. You have to match populations, or else all your association study is going to find is the differences between east asians and europeans, for example. You need a lot of samples to get enough stats power to even see these rare variants.

mindgam3 7y ago

“Nor... should his work be taken to mean that genes don’t affect depression. They do, and with newer, bigger studies, researchers are finally working out which ones do.” This part really needs some context/sources. Anyone know which studies he’s talking about?

bobcostas55 7y ago
Here's one: https://www.nature.com/articles/s41593-018-0326-7 https://www.nature.com/articles/s41593-018-0326-7
- mindgam3 7y ago
  Thanks

DuskStar 7y ago

I love the quote from the Slate Star Codex article: > ...what bothers me isn’t just that people said 5-HTTLPR mattered and it didn’t. It’s that we built whole imaginary edifices, whole castles in the air on top of this idea of 5-HTTLPR mattering. We “figured out” how 5-HTTLPR exerted its effects, what parts of the brain it was active in, what sorts of things it interacted with, how its effects were enhanced or suppressed by the effects of other imaginary depression genes. This isn’t just an explorer coming back from the Orient and claiming there are unicorns there. It’s the explorer describing the life cycle of unicorns, what unicorns eat, all the different subspecies of unicorn, which cuts of unicorn meat are tastiest, and a blow-by-blow account of a wrestling match between unicorns and Bigfoot. https://slatestarcodex.com/2019/05/07/5-httlpr-a-pointed-review/ https://slatestarcodex.com/2019/05/07/5-httlpr-a-pointed-rev...

lisper 7y ago

> Between them, these 18 genes have been the subject of more than 1,000 research papers, on depression alone. And for what? If the new study is right, these genes have nothing to do with depression. The article doesn't actually say anything about what these 1000 research papers actually claimed to show. It's entirely possible that most of them were negative results, there's no way to tell from the text of the article. It strongly implies that this is the case, but it doesn't actually say so. This is a significant omission for an article whose thesis is that these 1000 papers constituted a "house of cards." [UPDATE] "Sometimes the gene was linked to depression; sometimes it wasn’t. And crucially, the better the methods, the less likely he was to see such a link." Seems to me like this is science working exactly as it is supposed to (except for the possible suppression of earlier negative results, but that's a known broad problem that has nothing to do with this particular gene study).

rossdavidh 7y ago
It worked as it was supposed to...eventually. Instead of double-checking the initial results, hundreds of research papers by many research teams looked into questions that would be very important, if the genes in question were connected to depression. Sure, it is great that it's not (quite) religious dogma, and it eventually gets revisited. But, a couple decades worth of work and ~1000 research papers, many of which took place AFTER the earlier 2005 paper that called into question the link to depression, is by no means what we should shoot for. More fundamentally, there was nothing in the system that motivated anyone to do replication of the original link. The problem is not that the original link turned out to be spurious, and it's not that anyone did anything unethical. More worrisomely, nobody appears to have done anything unethical, but because of the systemic incentives for research grants, tenure, etc., replication of the foundational result wasn't attempted in a serious enough way until many times that much work had been done on something spurious.

pfdietz 7y ago

What's needed is for the Ignoble Prize people to focus on the people who start this sort of crap, not on humorous low value targets. Really call out the people who contributed negatively to the progress of science.