17 ms·

When they warn of rare disorders, these prenatal tests are usually wrong

146 points by phsource 5y ago

hprotagonist 5y ago
Behold, the curse of Reverend Bayes: https://en.wikipedia.org/wiki/Bayes%27_theorem#Drug_testing https://en.wikipedia.org/wiki/Bayes%27_theorem#Drug_testing
sklargh 5y ago
I recall a period in the early 2000s when unindicated whole-body CAT-scans were being advertised on television. That got knocked down pretty quickly but wow a lot of folks picked up a big chunk of their lifetime radiation allowance because of that. These tests seem to operate under a similar model, disregard the risks of unnecessary testing in return for information of limited utility that may cause material harm.
- bdzr 5y ago
  I think you're conflating "these tests cause harm" e.g. radiation and "the information gleaned from these tests could cause the patient to make poor decisions". Having a regulatory body make this value judgement for people has quite a bit of disadvantages. See "DON’T TRY THIS AT HOME: THE FDA’S RESTRICTIVE REGULATION OF HOME-TESTING DEVICES" https://scholarship.law.duke.edu/cgi/viewcontent.cgi?article=3914&context=dlj https://scholarship.law.duke.edu/cgi/viewcontent.cgi?article....
  - Enginerrrd 5y ago
    This isn't really a fair criticism. I could be wrong, but I believe your comment reflects a bit of naivete about the current state of evidence-based medicine. To evaluate the value of performing a diagnostic test as an intervention, you DO have to look at final actual patient outcomes at an appropriate end target which includes sending people unnecessarily down different treatment paths, including additional testing with additional risks. And most importantly is that, in fact, mere knowledge of diagnostic results has been PROVEN to cause harm in many scenarios. Now... if a patient WANTS that test, I think it should be available. But whether or not it should be performed routinely without prompting is an appropriate question for regulatory bodies.
treis 5y ago
This seems to miss the point entirely. Even for their worst example the odds of the fetus having it go from 0.005% to 7%. That's valuable information even if it's not perfect or somewhat hard to understand.
- inglor_cz 5y ago
  This would be valuable for running some extra tests (possibly more expensive, but more accurate), but not for, say, decision to abort the kid, which is what usually "hangs in the air" after such a test result.
  - sjckciodjcr 5y ago
    NIPT is not supposed to be used for termination decisions. A positive is meant to be “your baby might have this, test further with amniocentesis”.
    - inglor_cz 5y ago
      The article in NYT nevertheless states: "A 2014 study found that 6 percent of patients who screened positive obtained an abortion without getting another test to confirm the result." Maybe people aren't informed enough. It is my experience that some doctors tend to cut conversations short and some people are shy/insecure enough not to pry answers out of them. In this case, that would be a tragedy, given that statistically 5 of those 6 aborted fetuses were healthy. Edit: I found the following comment in the comment section of this article, which appears to address the same issue: I am a physician with a PhD in Biomedical Informatics. Most patients who receive these tests do not see a maternal fetal medicine doctor or genetic counselor, and no one actually explains that the tests they are receiving are “screening” or “diagnostic.” Your opinion that this article does a disservice to patients reflects your unrealistic assumption that most of the doctors ordering these tests are actually communicating effectively with patients (or frankly, even understand the tests themselves). In my experience, they usually aren’t/don’t. Articles like this “fill the gap” on patient education when doctors are unable to explain math and risk (i.e., most of the time).
inglor_cz 5y ago
Interesting. We have been undergoing IVF with my wife since 2019. (Covid made a huge mess of those plans...) One of our embryos tested as a possible positive (but only slightly) for aneuploidy of one chromosome. The doctor, a veteran of IVF, looked at the results and said "my experience is that this is either a very small mosaic error, which tends to be utterly invisible in real life, or a computer artifact. I have never seen embryos with those borderline results develop any serious problems later. Things would be different if the aneuploidy signals were clear, but definitely do not discard this embryo".
- isoprophlex 5y ago
  Good luck, keep up your hope. I hope things work out for you.
Neil44 5y ago
Before my daughter was born I sometimes felt like it was the doctors job to scare us with every worse case scenario possible. It was quite stressful and upsetting.
- lostlogin 5y ago
  I'm not certain that perk of parenthood ends at birth.
  - Auracle 5y ago
    When my then-pregnant wife called our fertility clinic with a worry the nurse calmed her down but also basically told her “welcome to the rest of your life.”
- kingkawn 5y ago
  The point of the profession is to find and address bad outcomes before they happen.
- middleclick 5y ago
  I had rather my doctor be upfront about all possible scenarios than to be try and nice about them and save possible information.
neonate 5y ago
https://archive.is/LEWoE https://archive.is/LEWoE http://web.archive.org/web/20220102044133/https://www.nytimes.com/2022/01/01/upshot/pregnancy-birth-genetic-testing.html http://web.archive.org/web/20220102044133/https://www.nytime...
gwern 5y ago
This article is a confused mess. It's something of a Gish gallop in conflating all the different issues they could come up with, while leaving out all the necessary vocabulary (C-f "Bayes" "posterior" "decision theory" [Phrase not found]) making it almost impossible to consider each issue in adequate detail. It mixes up poor communication (reporting false-positive/negative rates as if posterior probabilities, & exaggerated confidence thereof), arbitrary-seeming decision thresholds (but their hyperventilating over '85% wrong' notwithstanding, many are probably too conservative, if anything, given how devastating many of these problems are, there should be more false positives to trigger additional testing, not less), costs of testing (sure why not but little is presented), tests which they claim just bad and uninformative (developed based on far too little _n_, certainly possible), implicit calls for the FDA to Do Something and ban the tests (not an iota of cost-benefit considered nor any self-reflection about whether we want the FDA involved in anything at all these days)... Sometimes in the same paragraph. Plenty of valid stuff could be written about each issue, but they'd have to be at least 4 different articles of equivalent length to shed more light than heat.
- deleted 5y ago
  [deleted]
- bscphil 5y ago
  They even missed "base rate", which is the way I usually see this explained to ordinary people without stats backgrounds. Really disappointing.
  - SpicyLemonZest 5y ago
    They don't use that specific term, but the Down syndrome infographic does a pretty solid job at explaining the base rate issue.
- hn_throwaway_99 5y ago
  > implicit calls for the FDA to Do Something and ban the tests (not an iota of cost-benefit considered nor any self-reflection about whether we want the FDA involved in anything at all these days)... This is true in so many areas of journalism but lately seems especially egregious in the NYT. And I don't really blame them, as the incentives for any individual reporter are just too great - having the government make a major policy change based on your article is basically the brass ring for an investigative reporter. I basically can only use these types of articles as a jumping off point for my own research, as I usually find the moralizing conclusion the article comes to as unsupported.
csee 5y ago
> “The chance of breast cancer is so low, so why are you doing it? I think it’s purely a marketing thing.” This mindset is ingrained in every doctor I speak to, but I think it's just so wrong. Take DiGeorge syndrome. You have a 1/4000 chance of having it, and the test carries an 81% chance of a false positive. The above doctor calls this "marketing"? Foolishness. That's an incredibly useful test. The downside is small, and the upside is asymmetrically large. We need far, far better screening for all sorts of things. Adult cancer and heart screens once a year, prenatal screening, and on. We do a good job with breast and prostate screens, but for rarer conditions our current approach of waiting for the disease to be symptomatic makes no sense. Part of that will be driving the cost down. There is so much market need for a legitimate version of Theranos and I'm glad there are some companies working on these things.
- lostlogin 5y ago
  > We do a good job with breast and prostate screens Do we? Unless I'm missing something, breast cancer is a huge killer and PSA tests are deeply imperfect. I am very much not expert in these areas.
  - deleted 5y ago
    [deleted]
sjckciodjcr 5y ago
This article seems a bit deceptive. We are going through NIPT soon and our doctor went over false positive and false negative rates for the common screens. Our doctor has pointed out some of the screens (esp for rare conditions) are not that accurate. The only procedure with high accuracy, amniocentesis, has a slight risk of miscarriage (our provider quoted 0.3% ) so its still statistically better to take NIPT and then only consider amniocentesis with a positive result since there is no risk from NIPT. You are supposed to treat a positive on NIPT as “there’s a chance your baby has this, need a more accurate procedure to confirm”. It sounds like their ob gyn wasn’t able to explain results to them or they didn’t understand the probabilities. To be fair our provider didn’t even suggest tests for the disorders in the article, probably because of the false positive rates and rarity. Sounds like these extra screens shouldn’t be offered.
- SpicyLemonZest 5y ago
  "These extra screens shouldn't be offered" seems like exactly the point the article is trying to make.
  - deleted 5y ago
    [deleted]
halpert 5y ago
How did this article, written by someone who clearly lacks an understanding of basic statistics, make it into the Upshot? They try to make it seem like the test is wrong 85% of the time, but that's not necessarily the case. All we know from the article is that 85 / 100 positive results are false positives, which means the test could actually be quite accurate. If the test correctly identifies 100% of real cases, then that sounds like an excellent test. Just as an example, if 1/4000 people have the disease, and the test identifies 100% of these cases, then around 0.14% of test takers will get a false positive.
- SpicyLemonZest 5y ago
  Their infographics convince me that they understand the statistics. But one of the key issues here is that the statistics are radically counterintuitive in a way that most people don't understand - the patients, the testing companies, and even some medical staff all incorrectly believe that a positive test for a rare condition means you probably have the condition.
  - halpert 5y ago
    Their graphics say the tests are “84% wrong.” Do you really feel that’s an accurate description? That doesn’t feel like an accurate description to me, and their usage of “wrong” in this context highlights that they don’t understand the distinction and importance of true positives, false positives, true negatives, and false negatives when measuring accuracy.
    - SpicyLemonZest 5y ago
      I really feel it's an accurate description. If you get a positive result on the test, there's a 16% chance your fetus has a 1p36 deletion and an 84% chance they don't.
      - halpert 5y ago
        As you said “if you get a positive result”. It’s true, if you ignore the 99.9% of the time the test is correct (true negative result), then you can say the test is 84% wrong.
csours 5y ago
Edit: They kind of do this farther down in the article. Considering this as a UX challenge - imagine a grid of 10,000 dots (100x100). Draw one box around the base rate - the rate at which you expect to find the problem in the population. If the base rate is 1%, then the box is 10x10 = 100 dots. Then color in the dots for the test positive rate (not false positive, just all positive tests) False positives would be the colored dots outside the box. Next to that, put strikes through the dots corresponding to your expected false negative rate.
siganakis 5y ago
My wife and I went through this a couple of years ago, with a 10 week NIPT calling a rare trisomy (chr 9), which is always fatal within a few weeks of birth. It was absolute hell. The key problem here is the waiting and uncertainty. You have the NIPT at 10w, but you can’t have the amniocentesis until several weeks later. When that came back fine, there were questions about whether it was a “mosaic” meaning only a small proportion of cells are effected. We were only really in the clear after the 20 week ultrasound. That’s a lot of weeks to be consumed by wondering about whether to terminate the pregnancy, or wait it out for more information. I have a masters in bioinformatics (in genomics!) and my knowledge of stats and the science was next to useless in the face of these decisions. I know of couples who simply couldn’t deal with this uncertainty and chose to terminate on the basis of this test alone. Fortunately for us our child was fine and is a perfectly healthy 18 month old now, but I wouldn’t do the rare trisomy test again.
- bjt2n3904 5y ago
  So glad to hear that things turned out well for you and your family.
- raymondh 5y ago
  Thank you for sharing this.
- tinbad 5y ago
  Having gone through two twin pregnancies (where the odds of these tests being correct are especially low) we declined all of them. Anecdotally, I know of several parents who had a positive test for genetic disorder, went ahead with the pregnancy anyway and children were perfectly healthy. Until these tests are close to 100% reliable I don’t see the point.
  - CorrectHorseBat 5y ago
    The point is that it is a screening test. A positive test will be followed by a more invasive test that has a lower false positive rate.
don-code 5y ago
I am not a parent, but the criticism of the article appears to be around a misunderstanding of statistics, or at least how to apply them. While I agree that criticism is completely correct, it overlooks the human nature of the people receiving the tests. At an already-stressful point in someone's life, it seems almost like bad bedside manner for the medical community, even if in an automated fashion, to tell people that there might be a complication looming. This _does_, however, seem like a framing issue, more than a utility issue. If the tests are 100% accurate at detecting true positives, they're a great aid. But rather than framing the tests as a be-all, end-all source for information, why not frame them as "a test that suggests whether or not you should get other tests"? That simple wording change would save a great deal of added stress on someone starting or growing a family.
- isoprophlex 5y ago
  I totally agree with this. Managing perceptions and expectations is super important here. Having been on the receiving end of a false positive, I'd still do the test again for a hypothetical future pregnancy. Even though it was hell for a couple of days.
dougmwne 5y ago
Wow, what an embarrassing mess. Front page feature of bad statistics and bad medicine.
sterlind 5y ago
I've heard that in the early days of HIV, the tests were (e.g.) 95% accurate, and when patients saw their positive results and the supposed 5% chance it's wrong they'd sometimes kill themselves. They revised the tests so the first test would say Inconclusive rather than Positive, and ask them to repeat it. This saved some lives. Maybe this a UX failure? Shouldn't the test designers present the results like this, even to doctors?
- adjkant 5y ago
  Absolutely a UX failure here, one that it seems some doctors translate for patients while others are left in the dark on. From the way people are responding on here about the use of statistics in the article, it's clear that a big portion of the techo community I think is undervaluing that often UX is far more important than it is treated.
- Auracle 5y ago
  I had a friend tell me their daughter tested positive for something and they didn’t do any follow-up testing because someone else they knew also had a positive result and their baby was fine, so “the test is worthless.” Luckily their daughter seems to be healthy herself, but it clearly wasn’t explained well to them by their doctor.
tambeb 5y ago
A tweet about this very article caught my eye yesterday, and I'm glad HN's taken notice too. https://twitter.com/JohnFPfaff/status/1477382805583716353?t=UAFtsfEu43n_J2-fwA2JsA&s=09 https://twitter.com/JohnFPfaff/status/1477382805583716353?t=... 'For a disease w a 1-in-20,000 risk, a test w a false positive rate of 1% and a false negative rate of 0%—an insanely accurate test—would identify 1 correct case and 200 false positives every time. Or would be wrong 99.5% of the time. This isn’t “bad tests.” This is… baserates.'
divbzero 5y ago
Isn’t that often true with screens in general? The threshold often allows a good number of false positives in order to minimize false negatives. The goal is to know when to seek further diagnostics. Communicating that to patients can be a challenge but it doesn’t mean the screens were designed incorrectly.
jasonhansel 5y ago
IMHO, some of those criticizing the article for failing to understand statistics are missing the point. The point is that people who get a "positive" result on these tests are often put through terrifying levels of anxiety when there is no actual problem; this anxiety is often exacerbated because they aren't informed of the false positive rate. This clearly has a harmful emotional effect on people, and explaining the false positives in Bayesian terms, or reframing it in terms of sensitivity and specificity, doesn't undo that damage. That potential harm needs to be explained to patients, and it needs to be weighed carefully against the potential benefits of the test (as is done for PSA tests for prostate cancer, which also have a high false positive rate). Given that potential for harm, it's not unreasonable to ask that these tests be more tightly regulated. To quote the OP: > In interviews, 14 patients who got false positives said the experience was agonizing. They recalled frantically researching conditions they’d never heard of, followed by sleepless nights and days hiding their bulging bellies from friends. Eight said they never received any information about the possibility of a false positive, and five recalled that their doctor treated the test results as definitive. (Edit: clarified)
- midjji 5y ago
  If you get a positive for a horrid cancer with a 90 percent false positives you should be afraid. Its lunacy for tests to be regulated beyond requiring rough false positive false negative rates, and if anything smacks of "I dont understand statistics and therefore have to protect my children from understanding statistics." The article is most likely written by some anti abortion idiot.
  - teruakohatu 5y ago
    > if you get a positive for a horrid cancer with a 90 percent false positives you should be afraid. No you should not panic, anymore than you should celebrate and buy a yatch if you think you have 10% chance of winning a lottery. The doctor should phrase it as "the test indicates you have a 10% possibility of cancer. The majority of people who test positive do not have cancer. Further testing is required to confirm or rule it out." Going with the expected value of 10% of a horrid thing is still bad, or 10% of $100m is still $10m, is not applicable to a single non repeated event.
taeric 5y ago
This is an example of a problem that is so hard to explain. The vast majority of folks getting these tests will get a true negative. Such that for most people, this is not an issue. So I get that it takes effort to make people care. That said, I do feel that pulling in abortions to the debate is specifically to trigger a set of readers. But to what aim? They have not established that the tests could be better. Just that when they say yes, they are still not perfect.
tmnstr85 5y ago
My 2nd daughter was flagged during our 20 week for something having to do with the way her skull was forming and they wanted to do a series of genetic test. They charged us through the wazoo and everything came back negative. She arrived 3.5 weeks early and contracted bacterial meningitis shortly after birth. We found her code blue in the crib. She ended up having a bilateral craniotomy to relieve the empyema that had formed. CP, CVI, global TBI - every day is hell on earth. This was 2019, so the nightmare of the last few years started early for our family. We've had a number of medical professionals drop hints at the fact there might be something wrong from a rare disorder perspective but we're in a league of our own and that is hindsight - the damage and trauma are non-stop. Anyone trying to shickle a few dollars from the medical system to provide "pre-natal diagnosis" without sound science - they can come burn in the same hell I live in every day.
- jacobolus 5y ago
  I am sorry for your horrible situation. I don’t really understand what your implication is. Are you wishing you had not done the genetic tests? Suggesting you should have aborted the 20-week fetus based on unusual skull appearance? Suggesting that the meningitis was a result of some malpractice? Mad at medical professionals who are now “dropping hints” without saying something more substantive? The problem with these tests highlighted (confusingly) by the NY Times article is that they test for extremely rare conditions but have a high enough false positive rate (by my napkin math, on order 0.05% = 1/2000 false positive rate) that they end up dramatically over-diagnosing these conditions. That doesn’t necessarily mean the people designing the tests were lacking in “sound science” or were behaving maliciously. Their tests just need to be almost perfect to not overdiagnose rare abnormalities, and perfection in medical tests is a very high bar. Arguably we shouldn’t do tests for such rare conditions unless the false positive rate can be dropped by another order of magnitude or more, and if we do such tests communication to those who test positive should be much clearer. On the other hand, since genetic abnormalities are so life altering, even a 10% or 20% chance of an accurate positive might be grounds to abort a very early fetus. It sounds like your specific problem was that the genetic tests did not return a positive result, even though it seems plausible that your child had some genetic abnormality. Not picking up every possible genetic condition is a different problem from the false-positive problem highlighted by the article.
rflrob 5y ago
There's a lot of sibling comments going on about whether the value they're looking at is the right one. What the Times is showing as their headline number is Positive Predictive Value (True positive/(TP+FP)), which depends on the prevalence in the population. The "methods section" here is a little vague, but given the low prevalence I'm willing to accept on face value that it's basically accurate (i.e. that it's not assuming that the families getting these tests are not orders of magnitude more likely to be positive for these diseases). If the test result truly said one patient's 'daughter had a “greater than 99/100” probability of being born with Patau syndrome', then that's concerning, but given the fairly narrow quotes around the number, I'd suspect that what is actually on the test result is not inconsistent with the fairly low PPV on these screens.
bell-cot 5y ago
https://xkcd.com/1132/ https://xkcd.com/1132/ seems appropriate. Or perhaps https://xkcd.com/2545/ https://xkcd.com/2545/
dzink 5y ago
The state mandated tests in California are far worse. At least with NIPT tests, if you get a negative, it's fairly certainly a negative. The state tests have all kinds of unnecessary false positives, and if you don't have the NIPT to negate them, you are in for a lot of worry.
gaoshan 5y ago
We were told our son had a high chance of being born with down syndrome. It was quite stressful to hear this as we weren't going to do anything about it regardless (he was born with no issues whatsoever and is now a thriving young adult).
raspasov 5y ago
Watch this for a great explanation about the statistics of testing for rare diseases https://www.youtube.com/watch?v=R13BD8qKeTg https://www.youtube.com/watch?v=R13BD8qKeTg
rayiner 5y ago
Medical professionals are often shockingly bad at statistics. My wife and I were talking about birth control to an RN* after our first child was born. The RN mentioned cooper IUDs were 95% effective. He asked what timeframe that was measured over and she couldn’t answer. Not only did she not know, but she couldn’t even understand why we were asking the question. *) My wife insists that it was a doctor, not an RN, but my brain won’t let me process that possibility.
in_cahoots 5y ago
I was in this exact situation. I received a phone call from my midwives, saying that my son had tested positive for one of these disorders, and that these tests aren’t usually wrong. Fortunately I had done my research and knew that the false positive rate is high. But the entire system is set up to provide a terrible experience. Your results are sent directly to your provider, so you can’t read the fine print yourself. And if you do get access to the results, the wording implies that a null result (not enough DNA collected) actually means you’re likely to have some disorder. In fact the wording here actually got worse in the three years between my two (healthy) births. Ideally these companies should require genetic counseling before you take the test. Parents should understand that these tests are for screening purposes only, and that a definitive diagnosis can’t be gotten until 16-20 weeks. Unfortunately these companies have found a niche- parents wanting to know the sex and health of their children as soon as possible- and have no real reason to improve their practices.

146 points by phsource 5y ago

Behold, the curse of Reverend Bayes: https://en.wikipedia.org/wiki/Bayes%27_theorem#Drug_testing https://en.wikipedia.org/wiki/Bayes%27_theorem#Drug_testing

sklargh 5y ago

I recall a period in the early 2000s when unindicated whole-body CAT-scans were being advertised on television. That got knocked down pretty quickly but wow a lot of folks picked up a big chunk of their lifetime radiation allowance because of that. These tests seem to operate under a similar model, disregard the risks of unnecessary testing in return for information of limited utility that may cause material harm.

bdzr 5y ago
I think you're conflating "these tests cause harm" e.g. radiation and "the information gleaned from these tests could cause the patient to make poor decisions". Having a regulatory body make this value judgement for people has quite a bit of disadvantages. See "DON’T TRY THIS AT HOME: THE FDA’S RESTRICTIVE REGULATION OF HOME-TESTING DEVICES" https://scholarship.law.duke.edu/cgi/viewcontent.cgi?article=3914&context=dlj https://scholarship.law.duke.edu/cgi/viewcontent.cgi?article....
- Enginerrrd 5y ago
  This isn't really a fair criticism. I could be wrong, but I believe your comment reflects a bit of naivete about the current state of evidence-based medicine. To evaluate the value of performing a diagnostic test as an intervention, you DO have to look at final actual patient outcomes at an appropriate end target which includes sending people unnecessarily down different treatment paths, including additional testing with additional risks. And most importantly is that, in fact, mere knowledge of diagnostic results has been PROVEN to cause harm in many scenarios. Now... if a patient WANTS that test, I think it should be available. But whether or not it should be performed routinely without prompting is an appropriate question for regulatory bodies.

treis 5y ago

This seems to miss the point entirely. Even for their worst example the odds of the fetus having it go from 0.005% to 7%. That's valuable information even if it's not perfect or somewhat hard to understand.

inglor_cz 5y ago
This would be valuable for running some extra tests (possibly more expensive, but more accurate), but not for, say, decision to abort the kid, which is what usually "hangs in the air" after such a test result.
- sjckciodjcr 5y ago
  NIPT is not supposed to be used for termination decisions. A positive is meant to be “your baby might have this, test further with amniocentesis”.
  - inglor_cz 5y ago
    The article in NYT nevertheless states: "A 2014 study found that 6 percent of patients who screened positive obtained an abortion without getting another test to confirm the result." Maybe people aren't informed enough. It is my experience that some doctors tend to cut conversations short and some people are shy/insecure enough not to pry answers out of them. In this case, that would be a tragedy, given that statistically 5 of those 6 aborted fetuses were healthy. Edit: I found the following comment in the comment section of this article, which appears to address the same issue: I am a physician with a PhD in Biomedical Informatics. Most patients who receive these tests do not see a maternal fetal medicine doctor or genetic counselor, and no one actually explains that the tests they are receiving are “screening” or “diagnostic.” Your opinion that this article does a disservice to patients reflects your unrealistic assumption that most of the doctors ordering these tests are actually communicating effectively with patients (or frankly, even understand the tests themselves). In my experience, they usually aren’t/don’t. Articles like this “fill the gap” on patient education when doctors are unable to explain math and risk (i.e., most of the time).

inglor_cz 5y ago

Interesting. We have been undergoing IVF with my wife since 2019. (Covid made a huge mess of those plans...) One of our embryos tested as a possible positive (but only slightly) for aneuploidy of one chromosome. The doctor, a veteran of IVF, looked at the results and said "my experience is that this is either a very small mosaic error, which tends to be utterly invisible in real life, or a computer artifact. I have never seen embryos with those borderline results develop any serious problems later. Things would be different if the aneuploidy signals were clear, but definitely do not discard this embryo".

isoprophlex 5y ago
Good luck, keep up your hope. I hope things work out for you.

Neil44 5y ago

Before my daughter was born I sometimes felt like it was the doctors job to scare us with every worse case scenario possible. It was quite stressful and upsetting.

lostlogin 5y ago
I'm not certain that perk of parenthood ends at birth.
- Auracle 5y ago
  When my then-pregnant wife called our fertility clinic with a worry the nurse calmed her down but also basically told her “welcome to the rest of your life.”
kingkawn 5y ago
The point of the profession is to find and address bad outcomes before they happen.
middleclick 5y ago
I had rather my doctor be upfront about all possible scenarios than to be try and nice about them and save possible information.

neonate 5y ago

https://archive.is/LEWoE https://archive.is/LEWoE http://web.archive.org/web/20220102044133/https://www.nytimes.com/2022/01/01/upshot/pregnancy-birth-genetic-testing.html http://web.archive.org/web/20220102044133/https://www.nytime...

gwern 5y ago

This article is a confused mess. It's something of a Gish gallop in conflating all the different issues they could come up with, while leaving out all the necessary vocabulary (C-f "Bayes" "posterior" "decision theory" [Phrase not found]) making it almost impossible to consider each issue in adequate detail. It mixes up poor communication (reporting false-positive/negative rates as if posterior probabilities, & exaggerated confidence thereof), arbitrary-seeming decision thresholds (but their hyperventilating over '85% wrong' notwithstanding, many are probably too conservative, if anything, given how devastating many of these problems are, there should be more false positives to trigger additional testing, not less), costs of testing (sure why not but little is presented), tests which they claim just bad and uninformative (developed based on far too little _n_, certainly possible), implicit calls for the FDA to Do Something and ban the tests (not an iota of cost-benefit considered nor any self-reflection about whether we want the FDA involved in anything at all these days)... Sometimes in the same paragraph. Plenty of valid stuff could be written about each issue, but they'd have to be at least 4 different articles of equivalent length to shed more light than heat.

deleted 5y ago
[deleted]
bscphil 5y ago
They even missed "base rate", which is the way I usually see this explained to ordinary people without stats backgrounds. Really disappointing.
- SpicyLemonZest 5y ago
  They don't use that specific term, but the Down syndrome infographic does a pretty solid job at explaining the base rate issue.
hn_throwaway_99 5y ago
> implicit calls for the FDA to Do Something and ban the tests (not an iota of cost-benefit considered nor any self-reflection about whether we want the FDA involved in anything at all these days)... This is true in so many areas of journalism but lately seems especially egregious in the NYT. And I don't really blame them, as the incentives for any individual reporter are just too great - having the government make a major policy change based on your article is basically the brass ring for an investigative reporter. I basically can only use these types of articles as a jumping off point for my own research, as I usually find the moralizing conclusion the article comes to as unsupported.

csee 5y ago

> “The chance of breast cancer is so low, so why are you doing it? I think it’s purely a marketing thing.” This mindset is ingrained in every doctor I speak to, but I think it's just so wrong. Take DiGeorge syndrome. You have a 1/4000 chance of having it, and the test carries an 81% chance of a false positive. The above doctor calls this "marketing"? Foolishness. That's an incredibly useful test. The downside is small, and the upside is asymmetrically large. We need far, far better screening for all sorts of things. Adult cancer and heart screens once a year, prenatal screening, and on. We do a good job with breast and prostate screens, but for rarer conditions our current approach of waiting for the disease to be symptomatic makes no sense. Part of that will be driving the cost down. There is so much market need for a legitimate version of Theranos and I'm glad there are some companies working on these things.

lostlogin 5y ago
> We do a good job with breast and prostate screens Do we? Unless I'm missing something, breast cancer is a huge killer and PSA tests are deeply imperfect. I am very much not expert in these areas.
- deleted 5y ago
  [deleted]

sjckciodjcr 5y ago

This article seems a bit deceptive. We are going through NIPT soon and our doctor went over false positive and false negative rates for the common screens. Our doctor has pointed out some of the screens (esp for rare conditions) are not that accurate. The only procedure with high accuracy, amniocentesis, has a slight risk of miscarriage (our provider quoted 0.3% ) so its still statistically better to take NIPT and then only consider amniocentesis with a positive result since there is no risk from NIPT. You are supposed to treat a positive on NIPT as “there’s a chance your baby has this, need a more accurate procedure to confirm”. It sounds like their ob gyn wasn’t able to explain results to them or they didn’t understand the probabilities. To be fair our provider didn’t even suggest tests for the disorders in the article, probably because of the false positive rates and rarity. Sounds like these extra screens shouldn’t be offered.

SpicyLemonZest 5y ago
"These extra screens shouldn't be offered" seems like exactly the point the article is trying to make.
- deleted 5y ago
  [deleted]

halpert 5y ago

How did this article, written by someone who clearly lacks an understanding of basic statistics, make it into the Upshot? They try to make it seem like the test is wrong 85% of the time, but that's not necessarily the case. All we know from the article is that 85 / 100 positive results are false positives, which means the test could actually be quite accurate. If the test correctly identifies 100% of real cases, then that sounds like an excellent test. Just as an example, if 1/4000 people have the disease, and the test identifies 100% of these cases, then around 0.14% of test takers will get a false positive.

SpicyLemonZest 5y ago
Their infographics convince me that they understand the statistics. But one of the key issues here is that the statistics are radically counterintuitive in a way that most people don't understand - the patients, the testing companies, and even some medical staff all incorrectly believe that a positive test for a rare condition means you probably have the condition.
- halpert 5y ago
  Their graphics say the tests are “84% wrong.” Do you really feel that’s an accurate description? That doesn’t feel like an accurate description to me, and their usage of “wrong” in this context highlights that they don’t understand the distinction and importance of true positives, false positives, true negatives, and false negatives when measuring accuracy.
  - SpicyLemonZest 5y ago
    I really feel it's an accurate description. If you get a positive result on the test, there's a 16% chance your fetus has a 1p36 deletion and an 84% chance they don't.
    - halpert 5y ago
      As you said “if you get a positive result”. It’s true, if you ignore the 99.9% of the time the test is correct (true negative result), then you can say the test is 84% wrong.

csours 5y ago

Edit: They kind of do this farther down in the article. Considering this as a UX challenge - imagine a grid of 10,000 dots (100x100). Draw one box around the base rate - the rate at which you expect to find the problem in the population. If the base rate is 1%, then the box is 10x10 = 100 dots. Then color in the dots for the test positive rate (not false positive, just all positive tests) False positives would be the colored dots outside the box. Next to that, put strikes through the dots corresponding to your expected false negative rate.

siganakis 5y ago

My wife and I went through this a couple of years ago, with a 10 week NIPT calling a rare trisomy (chr 9), which is always fatal within a few weeks of birth. It was absolute hell. The key problem here is the waiting and uncertainty. You have the NIPT at 10w, but you can’t have the amniocentesis until several weeks later. When that came back fine, there were questions about whether it was a “mosaic” meaning only a small proportion of cells are effected. We were only really in the clear after the 20 week ultrasound. That’s a lot of weeks to be consumed by wondering about whether to terminate the pregnancy, or wait it out for more information. I have a masters in bioinformatics (in genomics!) and my knowledge of stats and the science was next to useless in the face of these decisions. I know of couples who simply couldn’t deal with this uncertainty and chose to terminate on the basis of this test alone. Fortunately for us our child was fine and is a perfectly healthy 18 month old now, but I wouldn’t do the rare trisomy test again.

bjt2n3904 5y ago
So glad to hear that things turned out well for you and your family.
raymondh 5y ago
Thank you for sharing this.
tinbad 5y ago
Having gone through two twin pregnancies (where the odds of these tests being correct are especially low) we declined all of them. Anecdotally, I know of several parents who had a positive test for genetic disorder, went ahead with the pregnancy anyway and children were perfectly healthy. Until these tests are close to 100% reliable I don’t see the point.
- CorrectHorseBat 5y ago
  The point is that it is a screening test. A positive test will be followed by a more invasive test that has a lower false positive rate.

don-code 5y ago

I am not a parent, but the criticism of the article appears to be around a misunderstanding of statistics, or at least how to apply them. While I agree that criticism is completely correct, it overlooks the human nature of the people receiving the tests. At an already-stressful point in someone's life, it seems almost like bad bedside manner for the medical community, even if in an automated fashion, to tell people that there might be a complication looming. This _does_, however, seem like a framing issue, more than a utility issue. If the tests are 100% accurate at detecting true positives, they're a great aid. But rather than framing the tests as a be-all, end-all source for information, why not frame them as "a test that suggests whether or not you should get other tests"? That simple wording change would save a great deal of added stress on someone starting or growing a family.

isoprophlex 5y ago
I totally agree with this. Managing perceptions and expectations is super important here. Having been on the receiving end of a false positive, I'd still do the test again for a hypothetical future pregnancy. Even though it was hell for a couple of days.

dougmwne 5y ago

Wow, what an embarrassing mess. Front page feature of bad statistics and bad medicine.

sterlind 5y ago

I've heard that in the early days of HIV, the tests were (e.g.) 95% accurate, and when patients saw their positive results and the supposed 5% chance it's wrong they'd sometimes kill themselves. They revised the tests so the first test would say Inconclusive rather than Positive, and ask them to repeat it. This saved some lives. Maybe this a UX failure? Shouldn't the test designers present the results like this, even to doctors?

adjkant 5y ago
Absolutely a UX failure here, one that it seems some doctors translate for patients while others are left in the dark on. From the way people are responding on here about the use of statistics in the article, it's clear that a big portion of the techo community I think is undervaluing that often UX is far more important than it is treated.
Auracle 5y ago
I had a friend tell me their daughter tested positive for something and they didn’t do any follow-up testing because someone else they knew also had a positive result and their baby was fine, so “the test is worthless.” Luckily their daughter seems to be healthy herself, but it clearly wasn’t explained well to them by their doctor.

tambeb 5y ago

A tweet about this very article caught my eye yesterday, and I'm glad HN's taken notice too. https://twitter.com/JohnFPfaff/status/1477382805583716353?t=UAFtsfEu43n_J2-fwA2JsA&s=09 https://twitter.com/JohnFPfaff/status/1477382805583716353?t=... 'For a disease w a 1-in-20,000 risk, a test w a false positive rate of 1% and a false negative rate of 0%—an insanely accurate test—would identify 1 correct case and 200 false positives every time. Or would be wrong 99.5% of the time. This isn’t “bad tests.” This is… baserates.'

divbzero 5y ago

Isn’t that often true with screens in general? The threshold often allows a good number of false positives in order to minimize false negatives. The goal is to know when to seek further diagnostics. Communicating that to patients can be a challenge but it doesn’t mean the screens were designed incorrectly.

jasonhansel 5y ago

IMHO, some of those criticizing the article for failing to understand statistics are missing the point. The point is that people who get a "positive" result on these tests are often put through terrifying levels of anxiety when there is no actual problem; this anxiety is often exacerbated because they aren't informed of the false positive rate. This clearly has a harmful emotional effect on people, and explaining the false positives in Bayesian terms, or reframing it in terms of sensitivity and specificity, doesn't undo that damage. That potential harm needs to be explained to patients, and it needs to be weighed carefully against the potential benefits of the test (as is done for PSA tests for prostate cancer, which also have a high false positive rate). Given that potential for harm, it's not unreasonable to ask that these tests be more tightly regulated. To quote the OP: > In interviews, 14 patients who got false positives said the experience was agonizing. They recalled frantically researching conditions they’d never heard of, followed by sleepless nights and days hiding their bulging bellies from friends. Eight said they never received any information about the possibility of a false positive, and five recalled that their doctor treated the test results as definitive. (Edit: clarified)

midjji 5y ago
If you get a positive for a horrid cancer with a 90 percent false positives you should be afraid. Its lunacy for tests to be regulated beyond requiring rough false positive false negative rates, and if anything smacks of "I dont understand statistics and therefore have to protect my children from understanding statistics." The article is most likely written by some anti abortion idiot.
- teruakohatu 5y ago
  > if you get a positive for a horrid cancer with a 90 percent false positives you should be afraid. No you should not panic, anymore than you should celebrate and buy a yatch if you think you have 10% chance of winning a lottery. The doctor should phrase it as "the test indicates you have a 10% possibility of cancer. The majority of people who test positive do not have cancer. Further testing is required to confirm or rule it out." Going with the expected value of 10% of a horrid thing is still bad, or 10% of $100m is still $10m, is not applicable to a single non repeated event.

taeric 5y ago

This is an example of a problem that is so hard to explain. The vast majority of folks getting these tests will get a true negative. Such that for most people, this is not an issue. So I get that it takes effort to make people care. That said, I do feel that pulling in abortions to the debate is specifically to trigger a set of readers. But to what aim? They have not established that the tests could be better. Just that when they say yes, they are still not perfect.

tmnstr85 5y ago

My 2nd daughter was flagged during our 20 week for something having to do with the way her skull was forming and they wanted to do a series of genetic test. They charged us through the wazoo and everything came back negative. She arrived 3.5 weeks early and contracted bacterial meningitis shortly after birth. We found her code blue in the crib. She ended up having a bilateral craniotomy to relieve the empyema that had formed. CP, CVI, global TBI - every day is hell on earth. This was 2019, so the nightmare of the last few years started early for our family. We've had a number of medical professionals drop hints at the fact there might be something wrong from a rare disorder perspective but we're in a league of our own and that is hindsight - the damage and trauma are non-stop. Anyone trying to shickle a few dollars from the medical system to provide "pre-natal diagnosis" without sound science - they can come burn in the same hell I live in every day.

jacobolus 5y ago
I am sorry for your horrible situation. I don’t really understand what your implication is. Are you wishing you had not done the genetic tests? Suggesting you should have aborted the 20-week fetus based on unusual skull appearance? Suggesting that the meningitis was a result of some malpractice? Mad at medical professionals who are now “dropping hints” without saying something more substantive? The problem with these tests highlighted (confusingly) by the NY Times article is that they test for extremely rare conditions but have a high enough false positive rate (by my napkin math, on order 0.05% = 1/2000 false positive rate) that they end up dramatically over-diagnosing these conditions. That doesn’t necessarily mean the people designing the tests were lacking in “sound science” or were behaving maliciously. Their tests just need to be almost perfect to not overdiagnose rare abnormalities, and perfection in medical tests is a very high bar. Arguably we shouldn’t do tests for such rare conditions unless the false positive rate can be dropped by another order of magnitude or more, and if we do such tests communication to those who test positive should be much clearer. On the other hand, since genetic abnormalities are so life altering, even a 10% or 20% chance of an accurate positive might be grounds to abort a very early fetus. It sounds like your specific problem was that the genetic tests did not return a positive result, even though it seems plausible that your child had some genetic abnormality. Not picking up every possible genetic condition is a different problem from the false-positive problem highlighted by the article.

rflrob 5y ago

There's a lot of sibling comments going on about whether the value they're looking at is the right one. What the Times is showing as their headline number is Positive Predictive Value (True positive/(TP+FP)), which depends on the prevalence in the population. The "methods section" here is a little vague, but given the low prevalence I'm willing to accept on face value that it's basically accurate (i.e. that it's not assuming that the families getting these tests are not orders of magnitude more likely to be positive for these diseases). If the test result truly said one patient's 'daughter had a “greater than 99/100” probability of being born with Patau syndrome', then that's concerning, but given the fairly narrow quotes around the number, I'd suspect that what is actually on the test result is not inconsistent with the fairly low PPV on these screens.

bell-cot 5y ago

https://xkcd.com/1132/ https://xkcd.com/1132/ seems appropriate. Or perhaps https://xkcd.com/2545/ https://xkcd.com/2545/

dzink 5y ago

The state mandated tests in California are far worse. At least with NIPT tests, if you get a negative, it's fairly certainly a negative. The state tests have all kinds of unnecessary false positives, and if you don't have the NIPT to negate them, you are in for a lot of worry.

gaoshan 5y ago

We were told our son had a high chance of being born with down syndrome. It was quite stressful to hear this as we weren't going to do anything about it regardless (he was born with no issues whatsoever and is now a thriving young adult).

raspasov 5y ago

Watch this for a great explanation about the statistics of testing for rare diseases https://www.youtube.com/watch?v=R13BD8qKeTg https://www.youtube.com/watch?v=R13BD8qKeTg

rayiner 5y ago

Medical professionals are often shockingly bad at statistics. My wife and I were talking about birth control to an RN* after our first child was born. The RN mentioned cooper IUDs were 95% effective. He asked what timeframe that was measured over and she couldn’t answer. Not only did she not know, but she couldn’t even understand why we were asking the question. *) My wife insists that it was a doctor, not an RN, but my brain won’t let me process that possibility.

in_cahoots 5y ago

I was in this exact situation. I received a phone call from my midwives, saying that my son had tested positive for one of these disorders, and that these tests aren’t usually wrong. Fortunately I had done my research and knew that the false positive rate is high. But the entire system is set up to provide a terrible experience. Your results are sent directly to your provider, so you can’t read the fine print yourself. And if you do get access to the results, the wording implies that a null result (not enough DNA collected) actually means you’re likely to have some disorder. In fact the wording here actually got worse in the three years between my two (healthy) births. Ideally these companies should require genetic counseling before you take the test. Parents should understand that these tests are for screening purposes only, and that a definitive diagnosis can’t be gotten until 16-20 weeks. Unfortunately these companies have found a niche- parents wanting to know the sex and health of their children as soon as possible- and have no real reason to improve their practices.