7 ms·

23andme replies to the GAO: GAO Studies Science Non-Scientifically

36 points by jamesbritt 16y ago

roder 16y ago
I'm a 23andme customer (post-DNA day) and after seeing the DNA mixup[1] and watching the youtube video released by the subcommittee on hearings and oversite[2], I am becoming increasingly weary of consumer genetic testing. I am glad to read that 23andme support regulation, because ultimately that is what should be required. Much like the regulation that HIPAA provides, DNA information should be federally protected and regulated. [1] http://www.switched.com/2010/06/08/23andmes-dna-mixup-leaves-96-customers-with-wrong-test-results/ http://www.switched.com/2010/06/08/23andmes-dna-mixup-leaves... [2] http://bit.ly/9MrBpe http://bit.ly/9MrBpe
- jamesbritt 16y ago
  I'd prefer to see Consumer Reports handle this instead of anything like the FDA. I'm an adult; I can decide for myself what to make of the information I obtain.
  - roder 16y ago
    I am also an adult, I also should be able to decide what I want to do with the data. However, when I send out my DNA and receive that data, I would hope that the DNA information should adhere to a scientific and privacy standard respectively. Of course, one could say that the free market could work this all out and everything will be just fine... On that note, please see October 2008 for reference.
  - timr 16y ago
    "I'd prefer to see Consumer Reports handle this instead of anything like the FDA. I'm an adult; I can decide for myself what to make of the information I obtain." I realize that this isn't going to be a popular opinion around these parts, but no, you can't. Smart as you may be, you're utterly ignorant when it comes to this stuff, and you haven't got a chance of beginning to understand the intricacies that go into interpreting the data that these companies are giving you. More importantly, you don't have time to understand. I have a Ph.D. in Biochemistry, and for the most part, I couldn't tell you whether the data in a 23andMe report is meaningful or utter garbage. I perhaps have enough knowledge to go out and find the necessary papers, read and interpret them in light of the literature on genomic analysis, and evaluate risks...but I wouldn't begin to have the time to do it properly for all of the data in a given report. No offense, but you haven't got a prayer. These companies could be fabricating results whole-cloth, and you'd have no way of knowing it. Most importantly, because there's no regulation of the methods used by these companies, neither you nor I have any way of knowing what methods they're using, whether the techniques are precise or accurate, or even if the labwork is done by skilled technicians in a sterile environment. Without these assurances, even if you could understand the data, you would have no guarantee that the data was even gathered correctly. There are some domains where expertise matters more than anything else, and no amount of brainpower makes up for the instincts provided by years of training and experience. This is one such area.
    - moultano 16y ago
      And for some reason you think this is different from any other field? I don't have a prayer when I walk into a car dealership of figuring out whether a car I'm thinking of buying will function properly. There are too many parts for me to conceivably understand. It's very complicated. Thankfully, there are many experts and expert organizations that I trust to make this determination for me. I defer to their authority. How is this any different? The market seems to handle this just fine.
      - jballanc 16y ago
        > And for some reason you think this is different from any other field? Yes, actually. If we must really flog a tortured analogy: This is not like going to your local mechanic friend and asking for car buying advice. It's more like this guy named Karl Benz comes and tells you he's got a great new invention called the internal combustion engine and you should definitely buy it. So you go to your friend the steam engine mechanic and ask him for advice. Sure, he understands the principles, maybe, but this is something completely new! If you must appeal to authority, consider you have two experts in this thread telling you that even they think the topic is too complex to draw useful conclusions. If you don't trust random people on the internet (not that I blame you), then go to the source. From the article: There are valid scientific reasons for different estimates from different companies, such as: companies employ slightly different statistical models for making risk estimates; companies establish different criteria for the inclusion of associations in their reports; new associations are being discovered at a faster rate than companies’ development cycles; companies may test for an imperfectly overlapping set of genetic variants for reasons including the ability of different genotyping technologies to assay certain variants. To be clear, this is not a case of Car and Driver favoring German engineering but JD Powers always skewing towards American manufacturers (completely contrived example, BTW). In this case the statistical models are the science, not just some interpretation of the science. This field is young. Too young. This money and effort would be better spent on basic research. > The market seems to handle this just fine. No, the market gives us homeopathy and snake-oil salesmen. Humans, especially when confronted with areas in which they are not knowledgeable, can be surprisingly irrational!
jballanc 16y ago
First, let me say that I'm very much in favor of the direction that 23andMe is headed. Personalized genetic medicine is the future. Generic small molecule treatments (and by generic, I mean that they're given to patients based on the disease and not based on the patient+disease profile) have pretty much hit their limit of efficacy and biologics are a lot like supersonic airliners: they'd be great in theory if there weren't so many problems with them in practice. I also sympathise with the way that the GAO, the FDA, and a number of other groups that have been weighing in on this issue have been lumping together 23andMe with some of the less reputable players in this burgeoning industry. That said, I still don't agree with what 23andMe is doing. I've nearly completed a Ph.D. in computational biochemistry and have been helping my wife with her thesis in cell signaling, and the only thing I can tell you about the information contained in a genetic profile that 23andMe provides you is that you can't really tell much, if anything, from the profile that 23andMe provides you. Even supposedly straight-forward genetics like the presumably Mendelian pattern of inheritance in CFTR SNPs can get complicated fast. For example: http://www.ncbi.nlm.nih.gov/pubmed/14966131 http://www.ncbi.nlm.nih.gov/pubmed/14966131 Of course, the real problem with 23andMe isn't with 23andMe but with the FDA. The FDA's job, first and foremost, is to keep people safe. IT IS NOT THE FDA's CONCERN TO SEE NEW TREATMENTS DEVELOPED! The result of this is that the medical community and the pharmaceutical industry are caught in a bit of a catch-22. Everyone that I've spoken too (including past CSO's of large-name pharmaceutical companies) recognizes that including genetic profiling of patients in drug trials might reveal that different people benefit from some drugs more than others, and that this is linked to genetics. In fact, you can probably recognize this yourself: When you get a headache, what do you reach for? For me, Ibuprofen does the trick, but Acetaminophen (Paracetemol) does nothing. For my wife, it's the reverse. This is almost definitely linked to genetics. The problem is that genetic profiling for drug trials is expensive and, get this, as of yet unproven to make a difference! Therefore, even if a pharmaceutical company went to the expense of including genetic profiling in their drug trial, the FDA would reject the results based on the fact that genetic profiling is unproven. So, nobody does genetic profiling. So you can't prove the usefulness of genetic profiling. etc. Now, you might think that 23andMe has a chance of solving this issue, by allowing people to take control of their own genetic profiling, but I think they're actually doing more harm than good. Unless you are highly trained in genetics and keep up with recent literature, you're not going to know what to do with that profile. If you bring that profile to a doctor, they will pretty much have to disregard it without a second thought (see FDA argument above). Honestly, with the current state of medical research and pharmaceutical regulation, a 23andMe profile is worth the paper you printed it on. What's worse, the more doctors get pestered with people bringing in their own profiles, the less likely they are to pay heed to any of it, and the distraction that this causes the FDA, worrying about whether they should regulate 23andMe or not, prevents the root issue from being addressed. I liken 23andMe to Quest Diagnostics. If you've visited a doctor recently, you're probably familiar with Quest. But when was the last time you went to Quest asking for a blood test without being prompted to do so by a physician? Honestly, you'd probably learn more on your own looking at the results of a cholesterol test or CBC than by perusing your own genetic profile. Until the U.S. (or, more likely these days, the E.U.) makes a major push for research into personalized genetic medicine, 23andMe will be little more than a novelty.
bkrausz 16y ago
I definitely treated my 23andme profile as a piece of entertainment, and don't think people should take it too seriously. I found someone who may be a distant cousin, and a few interesting traits, so I'd consider it a slightly more informed (and similarly priced) palm reading. I'm really glad I approached it like that, because I was one of the people who were in the DNA mixup. They told me I was a Tay-Sachs[1] carrier, which has a potentially vital impact on my relationship decisions (and was also a bit concerning given that neither of my parents are carriers). I'm really glad they caught and fixed it, and that I didn't take it seriously enough to lose sleep over the results. [1] - http://en.wikipedia.org/wiki/Tay%E2%80%93Sachs_disease http://en.wikipedia.org/wiki/Tay%E2%80%93Sachs_disease
jacquesm 16y ago
23andme engage in infotainment with an additional helping of candidates disease thrown in for each and every one of their customers. GPs are already amongst the most overworked people on the planet, the last thing they need is a herd of people with waving print-outs they (the people) don't understand in support of yet another round of imaginary diseases. It's the perfect product for the hypochondriac.
carbocation 16y ago
23andme uses GWAS SNPs, about 1 million of them. You have a 3 billion base pair haploid genotype, so they sample ~1/3000 base pairs. What does this mean? First, most of the SNPs are noncoding, found not in exons but in introns and intergenic space. Still, you can't dismiss these; some are causal for major gene expression changes (my lab has articles set to be published next week on this topic; more details then). Second, the fact that you're only covering 1/3000 nucleotides tells you that there's no way you can fully specify someone's genetic risk with this data, ever. There is a debate in the community right now about whether common variants (which are on 23andme type chips) or rare variants contribute most of the genetic variability. I think it's clear that common variants are winning this game — but this is on a population level! On an individual level, I don't think I'd ever be comfortable telling someone their risk of X with just GWAS data. For the population, their rare/private mutation in Gene Y has little impact, perhaps, but for try telling that to the 5 people with the Gene Y mutation who will die by age 20. (Extreme hypothetical to try to drive home the broader point.) Let's also not forget that phenotype is a computed expression of genotype+environment. If your genetic risk score from 23andme says you are at high risk of heart disease, yet your grandparents on both sides are 100 and healthy and your LDL-C is 60, should you really be concerned?
- khafra 16y ago
  If you know somebody's medical history and 23andMe results, can you say more things about them at any given level of confidence than you could with just zis medical history?
  - carbocation 16y ago
    Not really, at least not for cholesterol; not right now. [1] Hopefully in the next few years, yes. But genetic data, since our knowledge is limited, does not increase risk discrimination beyond family history. It does, modestly, increase risk classification. Edit - Link didn't work initially. I was not trying to just snarkily link to pubmed - I had a specific paper in mind. Sorry! [1] http://www.ncbi.nlm.nih.gov/pubmed/18354102 http://www.ncbi.nlm.nih.gov/pubmed/18354102

36 points by jamesbritt 16y ago

roder 16y ago

I'm a 23andme customer (post-DNA day) and after seeing the DNA mixup[1] and watching the youtube video released by the subcommittee on hearings and oversite[2], I am becoming increasingly weary of consumer genetic testing. I am glad to read that 23andme support regulation, because ultimately that is what should be required. Much like the regulation that HIPAA provides, DNA information should be federally protected and regulated. [1] http://www.switched.com/2010/06/08/23andmes-dna-mixup-leaves-96-customers-with-wrong-test-results/ http://www.switched.com/2010/06/08/23andmes-dna-mixup-leaves... [2] http://bit.ly/9MrBpe http://bit.ly/9MrBpe

jamesbritt 16y ago
I'd prefer to see Consumer Reports handle this instead of anything like the FDA. I'm an adult; I can decide for myself what to make of the information I obtain.
- roder 16y ago
  I am also an adult, I also should be able to decide what I want to do with the data. However, when I send out my DNA and receive that data, I would hope that the DNA information should adhere to a scientific and privacy standard respectively. Of course, one could say that the free market could work this all out and everything will be just fine... On that note, please see October 2008 for reference.
- timr 16y ago
  "I'd prefer to see Consumer Reports handle this instead of anything like the FDA. I'm an adult; I can decide for myself what to make of the information I obtain." I realize that this isn't going to be a popular opinion around these parts, but no, you can't. Smart as you may be, you're utterly ignorant when it comes to this stuff, and you haven't got a chance of beginning to understand the intricacies that go into interpreting the data that these companies are giving you. More importantly, you don't have time to understand. I have a Ph.D. in Biochemistry, and for the most part, I couldn't tell you whether the data in a 23andMe report is meaningful or utter garbage. I perhaps have enough knowledge to go out and find the necessary papers, read and interpret them in light of the literature on genomic analysis, and evaluate risks...but I wouldn't begin to have the time to do it properly for all of the data in a given report. No offense, but you haven't got a prayer. These companies could be fabricating results whole-cloth, and you'd have no way of knowing it. Most importantly, because there's no regulation of the methods used by these companies, neither you nor I have any way of knowing what methods they're using, whether the techniques are precise or accurate, or even if the labwork is done by skilled technicians in a sterile environment. Without these assurances, even if you could understand the data, you would have no guarantee that the data was even gathered correctly. There are some domains where expertise matters more than anything else, and no amount of brainpower makes up for the instincts provided by years of training and experience. This is one such area.
  - moultano 16y ago
    And for some reason you think this is different from any other field? I don't have a prayer when I walk into a car dealership of figuring out whether a car I'm thinking of buying will function properly. There are too many parts for me to conceivably understand. It's very complicated. Thankfully, there are many experts and expert organizations that I trust to make this determination for me. I defer to their authority. How is this any different? The market seems to handle this just fine.
    - jballanc 16y ago
      > And for some reason you think this is different from any other field? Yes, actually. If we must really flog a tortured analogy: This is not like going to your local mechanic friend and asking for car buying advice. It's more like this guy named Karl Benz comes and tells you he's got a great new invention called the internal combustion engine and you should definitely buy it. So you go to your friend the steam engine mechanic and ask him for advice. Sure, he understands the principles, maybe, but this is something completely new! If you must appeal to authority, consider you have two experts in this thread telling you that even they think the topic is too complex to draw useful conclusions. If you don't trust random people on the internet (not that I blame you), then go to the source. From the article: There are valid scientific reasons for different estimates from different companies, such as: companies employ slightly different statistical models for making risk estimates; companies establish different criteria for the inclusion of associations in their reports; new associations are being discovered at a faster rate than companies’ development cycles; companies may test for an imperfectly overlapping set of genetic variants for reasons including the ability of different genotyping technologies to assay certain variants. To be clear, this is not a case of Car and Driver favoring German engineering but JD Powers always skewing towards American manufacturers (completely contrived example, BTW). In this case the statistical models are the science, not just some interpretation of the science. This field is young. Too young. This money and effort would be better spent on basic research. > The market seems to handle this just fine. No, the market gives us homeopathy and snake-oil salesmen. Humans, especially when confronted with areas in which they are not knowledgeable, can be surprisingly irrational!

jballanc 16y ago

First, let me say that I'm very much in favor of the direction that 23andMe is headed. Personalized genetic medicine is the future. Generic small molecule treatments (and by generic, I mean that they're given to patients based on the disease and not based on the patient+disease profile) have pretty much hit their limit of efficacy and biologics are a lot like supersonic airliners: they'd be great in theory if there weren't so many problems with them in practice. I also sympathise with the way that the GAO, the FDA, and a number of other groups that have been weighing in on this issue have been lumping together 23andMe with some of the less reputable players in this burgeoning industry. That said, I still don't agree with what 23andMe is doing. I've nearly completed a Ph.D. in computational biochemistry and have been helping my wife with her thesis in cell signaling, and the only thing I can tell you about the information contained in a genetic profile that 23andMe provides you is that you can't really tell much, if anything, from the profile that 23andMe provides you. Even supposedly straight-forward genetics like the presumably Mendelian pattern of inheritance in CFTR SNPs can get complicated fast. For example: http://www.ncbi.nlm.nih.gov/pubmed/14966131 http://www.ncbi.nlm.nih.gov/pubmed/14966131 Of course, the real problem with 23andMe isn't with 23andMe but with the FDA. The FDA's job, first and foremost, is to keep people safe. IT IS NOT THE FDA's CONCERN TO SEE NEW TREATMENTS DEVELOPED! The result of this is that the medical community and the pharmaceutical industry are caught in a bit of a catch-22. Everyone that I've spoken too (including past CSO's of large-name pharmaceutical companies) recognizes that including genetic profiling of patients in drug trials might reveal that different people benefit from some drugs more than others, and that this is linked to genetics. In fact, you can probably recognize this yourself: When you get a headache, what do you reach for? For me, Ibuprofen does the trick, but Acetaminophen (Paracetemol) does nothing. For my wife, it's the reverse. This is almost definitely linked to genetics. The problem is that genetic profiling for drug trials is expensive and, get this, as of yet unproven to make a difference! Therefore, even if a pharmaceutical company went to the expense of including genetic profiling in their drug trial, the FDA would reject the results based on the fact that genetic profiling is unproven. So, nobody does genetic profiling. So you can't prove the usefulness of genetic profiling. etc. Now, you might think that 23andMe has a chance of solving this issue, by allowing people to take control of their own genetic profiling, but I think they're actually doing more harm than good. Unless you are highly trained in genetics and keep up with recent literature, you're not going to know what to do with that profile. If you bring that profile to a doctor, they will pretty much have to disregard it without a second thought (see FDA argument above). Honestly, with the current state of medical research and pharmaceutical regulation, a 23andMe profile is worth the paper you printed it on. What's worse, the more doctors get pestered with people bringing in their own profiles, the less likely they are to pay heed to any of it, and the distraction that this causes the FDA, worrying about whether they should regulate 23andMe or not, prevents the root issue from being addressed. I liken 23andMe to Quest Diagnostics. If you've visited a doctor recently, you're probably familiar with Quest. But when was the last time you went to Quest asking for a blood test without being prompted to do so by a physician? Honestly, you'd probably learn more on your own looking at the results of a cholesterol test or CBC than by perusing your own genetic profile. Until the U.S. (or, more likely these days, the E.U.) makes a major push for research into personalized genetic medicine, 23andMe will be little more than a novelty.

bkrausz 16y ago

I definitely treated my 23andme profile as a piece of entertainment, and don't think people should take it too seriously. I found someone who may be a distant cousin, and a few interesting traits, so I'd consider it a slightly more informed (and similarly priced) palm reading. I'm really glad I approached it like that, because I was one of the people who were in the DNA mixup. They told me I was a Tay-Sachs[1] carrier, which has a potentially vital impact on my relationship decisions (and was also a bit concerning given that neither of my parents are carriers). I'm really glad they caught and fixed it, and that I didn't take it seriously enough to lose sleep over the results. [1] - http://en.wikipedia.org/wiki/Tay%E2%80%93Sachs_disease http://en.wikipedia.org/wiki/Tay%E2%80%93Sachs_disease

jacquesm 16y ago

23andme engage in infotainment with an additional helping of candidates disease thrown in for each and every one of their customers. GPs are already amongst the most overworked people on the planet, the last thing they need is a herd of people with waving print-outs they (the people) don't understand in support of yet another round of imaginary diseases. It's the perfect product for the hypochondriac.

carbocation 16y ago

23andme uses GWAS SNPs, about 1 million of them. You have a 3 billion base pair haploid genotype, so they sample ~1/3000 base pairs. What does this mean? First, most of the SNPs are noncoding, found not in exons but in introns and intergenic space. Still, you can't dismiss these; some are causal for major gene expression changes (my lab has articles set to be published next week on this topic; more details then). Second, the fact that you're only covering 1/3000 nucleotides tells you that there's no way you can fully specify someone's genetic risk with this data, ever. There is a debate in the community right now about whether common variants (which are on 23andme type chips) or rare variants contribute most of the genetic variability. I think it's clear that common variants are winning this game — but this is on a population level! On an individual level, I don't think I'd ever be comfortable telling someone their risk of X with just GWAS data. For the population, their rare/private mutation in Gene Y has little impact, perhaps, but for try telling that to the 5 people with the Gene Y mutation who will die by age 20. (Extreme hypothetical to try to drive home the broader point.) Let's also not forget that phenotype is a computed expression of genotype+environment. If your genetic risk score from 23andme says you are at high risk of heart disease, yet your grandparents on both sides are 100 and healthy and your LDL-C is 60, should you really be concerned?

khafra 16y ago
If you know somebody's medical history and 23andMe results, can you say more things about them at any given level of confidence than you could with just zis medical history?
- carbocation 16y ago
  Not really, at least not for cholesterol; not right now. [1] Hopefully in the next few years, yes. But genetic data, since our knowledge is limited, does not increase risk discrimination beyond family history. It does, modestly, increase risk classification. Edit - Link didn't work initially. I was not trying to just snarkily link to pubmed - I had a specific paper in mind. Sorry! [1] http://www.ncbi.nlm.nih.gov/pubmed/18354102 http://www.ncbi.nlm.nih.gov/pubmed/18354102