39 ms·
My deadly disease was just a 23andme bug
- rohu1990 13y agoWhat could have happened on rest of life, if this guy just believed their result ? This is how you can ruin life of some one by coding !
- cmsimike 13y ago23andme should not be used as a replacement for a doctor. It would have been a shock to see the results, yes, but the next step is confirmation from a doctor.
- flebron 13y agoThat's for a false positive, sure. But who goes to the doctor to confirm a false negative, that 23andme says there's no reason to worry about? The list of negative results is going to be huge, in most cases.
- cmsimike 13y agoThe simple answer is that 23andMe is for informational purposes only. I don't go any more or less to the doctors now that I've results from 23andMe. It is cool information, yes, but that's it.
- logfromblammo 13y agoFor most people, not having any symptoms is the (possibly false) negative that prevents them from scheduling a doctor visit. Seeing a catalog of 10000 potential risk factors that I don't have and didn't even know about isn't likely to change that. If I have flu-like symptoms, knowing that I don't have any additional risk factors for lupus won't keep me from seeking treatment for my flu-like symptoms, because I could just have the flu.
- mseebach 13y ago> But who goes to the doctor to confirm a false negative The same people that goes to doctors and dentists for regular check-ups without suspecting anything wrong.
- revelation 13y agoWe know not one iota of what there is to know about the genome. Any report they create contains trillions of false negatives; that is the nature of genome research in its current state.
- Symmetry 13y agoIs someone with a false negative from 23andme in a worse position than someone who hasn't had a genetic test?
- BHSPitMonkey 13y agoThe service in no way tells people they have "no reason to worry" about their health. Even if your report came back "clean", that would only be telling you that your genetics don't suggest any particular risk factors spelled out in your DNA, not that you're healthy.
- geetee 13y agoThat's a risk you take with getting your DNA tested. Are you prepared to handle the results (right or wrong)?
- mkopinsky 13y agoNo, the (necessary) risk you take is that you find out true information that you didn't want to know. The risk that you shouldn't have to take is that the testing company sends you BS results.
- DanBC 13y agoAll tests carry risks of BS results. People taking any test need to be aware of the risks of false negatives and positives.
- drcode 13y agoTHINK OF THE CHILDREN!
- jrockway 13y agoProbably nothing. Even if you could tell someone the exact instant in time they would die, it's bad news, so they'd deny it up until that time. People with chronic disease know they're going to die because the disease makes their body work incorrectly, which is easy to notice and be depressed about.
- revelation 13y agoHuh? A lot of the negatives they inform you about aren't really actionable (they are in your genome, after all), but if they were, 23 and me isn't a doctor; you would still go see a real one to receive any treatment.
- aabalkan 13y agoWhy your character encoding is broken on your blog post?
- deleted 13y ago[deleted]
- b409ba0801cd21 13y agoActually the encoding is UTF-8 but the server is not telling clients, so clients are often guessing it is something like Windows-1252.
- deleted 13y ago[deleted]
- martin-adams 13y agoIt's a mutation of the HTML's DNA
- interstitial 13y agoHaplo-text mark-up language.
- aaronem 13y agoMore like looking at it in the wrong reading frame.
- mntmn 13y agoSorry, fixed!
- kevrone 13y agoWow, this actually makes me pretty impressed with 23andme. Yeah, ok, maybe their analysis front-end is borked, but hell, the science seems pretty sound if he was able to get the "right" answer from the original sample.
- aaronem 13y agoAbout what percentage, would you say, of 23andme's clients are both equipped and inclined to educate themselves in bioinformatics, to the degree OP did, to the extent required to replicate 23andme's sequence analysis and identify errors in their results?
- sachdevap 13y agoAbout what percentage, would you say, of individuals paying income tax are both equipped and inclined to understand all the subtleties of income tax returns? Yet people do pay income taxes and do not look into the possibility of extra amounts paid most often. Only individuals who really care, are willing to dig deep into it. In my opinion, people who use this service are going to use it because they are genuinely curious about their future health and if it leads to a few false positive, I could live with that and get it checked elsewhere for confirmation.
- aaronem 13y agoIt's interesting you should choose taxes in particular for your analogy; by trivial extension, you cast 23andme as someone who operates as a tax accountant not merely in ignorance, but in outright defiance, of the professional licensure requirements of the field. Libertarian-utopian notions of needless regulatory immurement aside, such requirements exist for reasons amply evident in a study of their origins; in short, without a licensure requirement in such a highly specific and technical realm, incompetents and charlatans of every stripe abound, to the ultimate and severe detriment of all those who would seek the advice of genuine professionals in the field.
- kevrone 13y agoNo I get what you're saying. I'm not suggesting that the burden is somehow on the customer to figure out their genome, what I'm saying is that the problem described in the blog is not because of some inherent fuzziness in the science, but just a problem with the web-app. All systems have bugs, even the ones which lives depend on. It's cool that the author was able to bring the issue to 23AM's attention, but it's also cool that they were able to correct it.
- newnewnew 13y agoThank goodness there are thousands of people being paid to prevent me from spitting in a tube and getting noisy data about my own body! I should only be able to get data about my body after running a gauntlet of specialists with millions of dollars in total education, for my protection, of course.
- robk 13y agoFor a hundred bucks, I'm quite pleased with the results I get. Of course if I was flagged for something serious I'd go to a doctor immediately to get it checked and verified.
- jrockway 13y agoThis just increases your risk of a complication. If you have a disease with no symptoms, it's probably not time to try and treat it. Treatment has risks, after all. (Think about all the people that have some minor problem, get minor surgery, and then pick up MRSA in the hospital while they're recovering. Now they have a major problem.)
- bmelton 13y agoI thought that was covered by the "go to the doctor" bit. The doctor would, if even remotely competent, verify the presence of a disease before beginning treatment. For all the hoopla I keep hearing be attributed to malice regarding 23andMe's results, honestly, I think the results, at the least, encourage people to go to the doctor. My father-in-law was diagnosed with cancer a couple of years ago, but because he was regular with checkups, they were able to catch it early, and he's been in remission for over a year. A false positive by 23andMe regarding one's predisposition towards cancer should motivate them to the doctor more often, which would indicate whether they were or were not cancer positive -- but in the event of either, I'd wager that more frequent trips to the doctor would be a net benefit.
- tedunangst 13y agoFor everybody who believes their doctor, there's somebody who doesn't and will just find a new doctor to confirm their self-diagnosis. "I was right, my doctor was wrong" stories get posted to HN fairly frequently.
- bmelton 13y ago"fairly frequently" is still, in this context, within the general margin of error for any statistically sound sample size. Yes, people disagree with their doctors, and yes, sometimes those doctors are wrong, but by and large, the number of people with multiple doctors in agreement, where that agreement is still wrong, are statistical outliers. Further, and maybe it's just me here, I've learned a lot in this thread already, and I'm not an expert on the subject, but I just fail to see how the idea that I have a higher predisposition to cancer than the statistical norm, even if that information might be wrong, can be harmful. I mean, I understand the objective, and the idea of preventing the sale of snake oil is one that I see as vlid in this context, but this isn't that. 23andMe doesn't make any curative claims whatsoever, as much as to announce that "hey, there might be snakes." They're far more scientifically sound than a mood ring, but even if they aren't, I just don't personally see the harm in that.
- tptacek 13y agoNow consider that this is a marketing message that wants to have it both ways: it alerts clients to genetic "risks", which are very likely to be subjected to secondary testing, but does not want to be liable for the accuracy of its negative results, which are very unlikely to be challenged. It would be better if 23AM presented medical genomic information in a neutral way, with external links to descriptions of genomic variations but no assertion of diagnostic significance at all, or even a mention of how to obtain a diagnostically significant result. Unfortunately, the way 23AM is packaged (see the website), they are incentivized to do the opposite; positive results are recognized by their clients as valuable, and, whether 23AM likes it or not, so are the negative results.
- karcass 13y ago23AM just got a smackdown letter from the FDA about that: http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2013/ucm376296.htm http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2...
- mkopinsky 13y agoThat letter was linked from the article itself, and is what prompted the fact that this discussion is happening now.
- Amadou 13y agoCompare it with Carfax and similar "car history" reporting websites. A positive result is nearly always meaningful, but a negative result means nothing more than a lack of hits in a database that is well known to be incomplete.
- shazow 13y agoHere is a company who is trying to make genetics more accessible, approachable, and comprehensible. > It would be better if 23AM presented medical genomic information in a neutral way, with external links to descriptions of genomic variations but no assertion of diagnostic significance at all, or even a mention of how to obtain a diagnostically significant result. What you're suggesting would effectively revert the consumer experience about half a step above "go to lab X to sequence your DNA, then hop on over to SNPedia.com to figure out what it all means." > Unfortunately, the way 23AM is packaged (see the website), they are incentivized to do the opposite... Perhaps this is more subjective than worth talking about, but that's not the impression I had when using and working on the product. All the more reason they need to be careful, for sure. Disclaimer: I worked at 23andMe's frontend team briefly, a few years ago. While I have mixed feelings about their engineering practices and other things, I do believe in and support their mission.
- rafeed 13y agoI'd never want to get my DNA tested by a service like this. It may be cool, but there's too many things at play here. - privacy - security of data - wrongful use of data - spread of misinformation - mind fucking that ensues after reading your results - probably a lot more The only benefit is for those who are so curious about what genes they carry, what mutations they have, and who they may or may not be related to. Really, why risk having your mind potentially fucked by knowing something about yourself that you're not ready to handle yet? Sorry if I seem overly critical about a service like this, but it just doesn't seem worth it to me.
- deleted 13y ago[deleted]
- Jemaclus 13y agoI signed up for 23andme.com a few months ago, mainly because I'm adopted and I don't know anything about my family history, so I thought it would be neat to check out. The one and only thing that bothers me so far is their definition of privacy. Here's what I mean: One of the features they provide is ancestry help. They compare your DNA with all other users' DNA and then give you a list of potential relatives. They say "hey, you guys share .6% of the same DNA, you might be 3rd to 5th cousins." That's cool, right? Especially for someone who is adopted. The problem is that it takes 2 weeks longer to do the ancestry stuff than it does to do the medical stuff. The medical results showed up in my account about 4 weeks after I sent off my spit-filled test tube. The ancestry results showed up a full month later. In the meantime, between medical results and ancestry results, I'm getting "friend requests" from other users. There are no messages attached to these users, but I presume they believe, somehow, that they are related to me. How can this be? I hadn't yet received my own ancestry results. I emailed 23andme.com support, and here's what they said: > Thank you for contacting 23andMe. > If you've been genotyped through our service, you can share your genome with other 23andMe users and compare yourself with them using our various features. >This is of special interest to members who are interested in their family's data. > Due to the way our computation process works, some of your relatives can actually see you appear in their list of DNA Relatives before all your results are uploaded. Let me repeat that last part: > Due to the way our computation process works, some of your relatives can actually see you appear in their list of DNA Relatives before all your results are uploaded. So get this: my name and some level of my DNA results are shown to complete strangers, and not only that, but they are shown before I even get the information. That bothers me. That bothers me a lot. I haven't yet decided what to do about it, but believe me, I've spent a great deal of time considering this since it happened.
- bparsons 13y agoSeems like the problem was addressed immediately. This seems like a success story of 23andMe.
- AsymetricCom 13y agoSeems like there's a lot of pointless and distracting comments on this story. Thanks for your contribution.
- sneak 13y agoIf I were to ever use 23andme, I'd use a fake name, a disposable mailing address, a fake email address, and never, ever discuss it with anyone. Your genome doesn't change, and who knows where that data leaks to later? Who knows what healthcare providers will vacuum up such leaks? Who knows what kind of discriminatory practices will be undertaken by service providers, insurers, or even employers in the future? Haven't y'all ever watched Gattaca?! Keep your private information private. Nobody else will do it for you. EDIT: Oh, and USA PATRIOT too.
- jisaacso 13y agoYou can potentially predict a male's surname from a DNA sample. The accuracy is roughly proportional to the frequency of the name. http://www.sciencemag.org/content/339/6117/321.abstract http://www.sciencemag.org/content/339/6117/321.abstract
- endtime 13y agoI think I can do that without DNA...
- deleted 13y ago[deleted]
- rtpg 13y agoconsidering that pre-existing conditions can no longer be considered for health-care service, there's less things to worry about. Employers, on the other hand...
- gms7777 13y ago> considering that pre-existing conditions can no longer be considered for health-care service, there's less things to worry about. They can't be considered now. There's nothing to say that a few years down the line that won't change.
- chimeracoder 13y ago
- zallarak 13y ago23andme is a consumer genetic testing company. The testing is not meant to be clinically applied. I work for a company that is similar to 23andme, different in the sense that the test can be clinically applied; the catch is that everything you report must be very robustly researched and you must comply with more regulation (research, lab, medical, security, etc.) which drives up the cost but gives you much more actionable and medically relevant information. I think there are benefits to both approaches of genetic testing; 23andme has a lot of data it collects and can do interesting statistical studies/reporting that other more research-oriented companies cannot. However, like I said before, you can't view a test like that as medically actionable. In the end though, the more testing there is (as long as the messaging is clear and consumers do their homework), the better oFf the world will be.
- peter303 13y agoUnexpressed disease genes has been an issue from the beginning of human sequencing. The 3rd human sequenced James Watson had almost 30 unexpressed disease genes, including retinitus pigmentosa. No one understands why this occurs.
- clavalle 13y agoCrazy idea: How about, instead of getting this report ourselves, we have it sent to our primary care physician. That way we don't have to stress that there is some likelihood that we might get some dreaded disease but our doctor could talk to us if the risk is high enough or if we start having symptoms that match one of our genetic risk factors?
- LogicalBorg 13y agoOr the doctor could have just told him that he had the disease without questioning it and with no chance for him to look at his own data and disprove it, giving him an even worse outcome than he had here. Doctors aren't all-knowing or always right. And most of them know nothing special about genetics.
- BHSPitMonkey 13y agoThe distributed network of human physicians probably has a higher error rate than the software described in the article.
- jessriedel 13y agoGreat, you're welcome to do that on your own. I'd just like to read the report myself.
- ksk 13y agoThe potential of pharmaceutical companies pushing meds onto unsuspecting people using this data is really frightening. Hopefully 23andme wont turn into an advertising subsidized business.
- IanDrake 13y agoI'm surprised at the negativity here. There is nothing conclusive about this test and from what I've seen it does a pretty good job. So one guy got some bad indicators that proved to be nothing, who cares? My father did 23 and me, without giving any family medical history and it concluded that my father was at a much higher risk factor for things that his mother was ultimately afflicted with. So, it works to some extent. On the flip side, long story short, I had a CT scan done on my chest that checked out fine, but the doctor that reviewed it said I might have and unrelated problem - Patent Ductus Syndrome. Nothing heart surgery and a lifetime of supply of Coumadin couldn't fix. A few months later I got an Echocardiogram which conclusively said the doctor was wrong. Quick, stop doing CT scans!!!??? Are we're all so afraid of our own shadow that heroes at the FDA need to protect us?
- bdcravens 13y agoWe expect our employers to triple-check everything they say in the fear that they might say something that could possibly be construed as sexist or racist or harassing. Our teachers aren't allowed to offer our children honest critiques for fear that they'll crush their poor little self-esteems. A genetic diagnostic can completely crush a person's outlook on life. This is suicide-level stuff potentially. As for your doctor, do you still happily use that doctor and recommend them? The scan wasn't the problem; the service provider was. Ditto for 23AndMe. They don't get a free pass because they're a "tech" company.
- FiloSottile 13y agoYeah, all doctors, as humans, and systems, as machines, make mistakes. What counts is statistics, and negligence. Are stats about this test/doctor worryingly bad compared to the average? Are the mistakes caused by negligence? If not, I'll happily use that doctor/service again. I have far more problems with the growing perceived need to have employers triple-check everything they say, teachers avoid honest critiques and distrust any new technology that kills N people when the old one killed 10N (re: Tesla fires).
- mcherm 13y ago> We expect our employers to triple-check everything they say in the fear that they might say something that could possibly be construed as sexist or racist or harassing. Our teachers aren't allowed to offer our children honest critiques for fear that they'll crush their poor little self-esteems. Some people do these things. But I believe that they are overreacting, and from your phrasing so do you. So why are you proposing that health data be held to such absurd (and unhealthy) standards? > A genetic diagnostic can completely crush a person's outlook on life. This is suicide-level stuff potentially. Then the person has serious mental issues. I'm not saying that hearing you are susceptible to a serious disease should slide off like water from a duck's back, but if a person feels that learning a medical diagnosis may cause them to commit suicide, then under no circumstances should they engage a company like 23AndMe that clearly states up front that they will NOT be providing counseling. As far as I can tell, no one INCLUDING 23AndMe is proposing that they get a "free pass". In this case @mntmn reported their error to them and they corrected it, not just for him but for ALL future patients. That is better than most doctors do when THEY find an error.
- adeptus 13y agoI'd love to use a service like 23andme, but am seriously concerned about privacy issues. Do they have any privacy conscious competitors? Any upcoming ones?
- pbhjpbhj 13y agoI wonder what financial value is placed on their database and how well secured it is.
- jisaacso 13y agoThis will likely be downvoted to hell, but personally I'm a fan of the service 23andme provides. It has aggregated scientific knowledge across a large number of domains to provide an end-to-end pipeline for ancestry prediction, kinship analysis and phenotype prediction. It's a service built on learning algorithms to correlate features (DNA SNPs) with diseases. As with any learning algorithm, results should be interpreted with care. That doesn't lessen the fact that predictions it provides can be helpful. As an aside, 23andme has a solid API. It's opened up (with consent) a huge DNA dataset for developers to mine. I'm excited for the science that can be learned from this data.
- mkopinsky 13y agoI think many of the commenters here are fans of 23AM, and are hoping they get their regulatory shit together so they can provide an even more valuable and validated service.
- aaron695 13y agoIn saying "This will likely be downvoted to hell" you are basically saying you are smarter than us. Hence why you'll be downvoted.
- wissler 13y agoIt is good to know of real-world misdiagnoses made via 23andMe. Now, can we see some real-world statistics by the government-licensed medical profession? I wonder what a side-by-side analysis of the errors 23andMe has made vs. those the medical community has made might look like. I wonder how the real-life consequences might stack up. As far as I know, 23andMe has never accidentally amputated the wrong limb, so it's at least go that going for it. Interesting tidbit: you can always double-check the reports 23andMe gives to you by getting followup tests someplace else.
- smackfu 13y agoIsn't this the whole point of the FDA looking into 23andMe? To evaluate whether their accuracy matches up to their marketing claims?
- nickff 13y agoWissler seems to want a comparison of the differences in results between the using 23AM and not using it. The FDA measures efficacy in absolute terms, not relative to the counter-factual.
- coldskull 13y agoHypothetically, if i were in OP's shoes and 'really' scared, I would be going to a genetic doctor instead of conducting my own research. (i would still look up stuff, but that would be too risky to depend on)
- kohanz 13y agoThe frustration directed towards the FDA in the 23andMe threads is not surprising, but short-sighted, IMHO. Sure, there are likely inefficiencies in the way the FDA operates, in a micro-sense, but in the macro-view, this sort of stuff needs to be regulated. Many simple take-home tests that are arguably much more time-tested, reliable, and accurate than 23andMe's DNA services are FDA-regulated. If your basic take-home pregnancy test is FDA regulated, why should 23andMe be exempt? Or would you prefer a world where all of these tests were unregulated? Not everyone does their research. Not everyone is intelligent enough to understand the implications of medical tests, drugs, and devices - regulation protects the vulnerable among us. As an aside, I've worked in medical devices for most of my career. All other things remaining equal, engineering teams in these companies are no different, talent-wise and quality-wise, than most teams you'd see in other industries. Take the sloppiest engineering team you've worked with, now imagine them working on a clinical product with the potential to cause real harm to people (physical or psychological). It is the regulation (FDA and others) that ensures that these teams "raise their quality game", so to speak. It is not perfect by any means, but I don't dare imagine the alternative. Yes, the regulation is a massive pain. I experience it first-hand. It slows things down, has massive associated costs and can even stifle innovation. The general trade-off, however, is worth it, IMHO.
- anigbrowl 13y agoIf your basic take-home pregnancy test is FDA regulated, why should 23andMe be exempt? Or would you prefer a world where all of these tests were unregulated? That's an outstanding comparison, one I wish I had thought of.
- gojomo 13y agoPlenty of assessment errors occur via other forms of health research, self-diagnosis, and professional diagnosis, too. That's why the more serious a result is, the more it should be examined and re-checked. This incident seems to me like a big win for the 23andme model. They made an error – but followup by a single individual means the same error won't happen again for anyone in their international clientele. Errors in other health systems don't get globally corrected so quickly.
- patfla 13y agoHmm, a not irrevelant data point. http://allthingsd.com/20131125/fda-tells-23andme-to-stop-marketing-dna-kits-23andme-says-its-sorry-for-being-slow/ http://allthingsd.com/20131125/fda-tells-23andme-to-stop-mar...
- JosephBrown 13y agoYou are a VERY good science writer btw.
- SwellJoe 13y agoI love 23andme. I don't make major medical decisions based on it (yet), and I know it has limitations (it didn't predict the pancreatic cancer that killed my dad, and it didn't find indicators for a number of other conditions in my family tree...). But, it's a fun thing to play with, it has connected me up with a number of folks in my family tree that I didn't know about, and it resolved some questions about the ancestry of my family. I'm an American mutt from poor white trash stock...no idea where everybody came from until 23andme. My sister had tried to do a family tree but ran out of steam just a couple of generations back. Anyway, it's probably good they're being reined in from making medical claims. I don't think the tech is quite there yet. But, as a tool alongside many others, it's cool. And, I love that they're doing original research with their surveys...it's a super cool idea to combine lots of crowdsourced data and DNA results of thousands of people to find markers for diseases and traits. I plan to keep recommending 23andme to folks who can afford it and folks who can grasp that a DNA test isn't a reliable indicator of disease and that it is merely a probability indicator.
- deleted 13y ago[deleted]
- imahboob 13y agoFor more read this http://www.labcritics.com/2013/11/26/fda-asks-23andme-to-stop-its-services/ http://www.labcritics.com/2013/11/26/fda-asks-23andme-to-sto...